DISCUSS: Event filtering in Sensu::Handler should be deprecated and then removed

The problem
The Sensu::Handler class currently implements a number of methods for filtering events, e.g. filter_disabled, filter_repeated, filter_silenced, and filter_dependencies. Each of these filters is evaluated every time an event is handled by a Sensu::Handler plugin.
This Sensu::Handler class has been widely adopted by the plugins in the community Sensu Plugins collection. The original intention was for organizations to maintain their own plugins and override the filter method for their own purposes, but thats not what has manifest in our ecosystem.
Because of how these filter methods are implemented inside Sensu::Handler, and owing to the popularity of the community plugins in our ecosystem, it has become very difficult for operators to distinguish between bugs in Sensu::Handler's filtering methods and strange behavior in Sensu Core itself. For example, even rather commonly used check attributes like refresh aren't implemented directly in Sensu Core, but are used only in Sensu::Helper filter methods.
In particular, the filter_repeated method uses logic to compare occurrences, refresh and status in order to determine if the event should be disregarded as a repeat. It's apparently prone to filtering events which should have been sent , often to indicate the outage was resolved. There may be some room for improvements to the existing filtering made possible by making use of last_ok and last_state_change event attributes, but we lack tests to describe the current behavior and can't change much without breaking many plugins.
The proposed solution
As stewards of Sensu, we'd like to see filters to become the primary way for operators to separate signal from noise, instead of relying on this library to cover all cases. Although we think operators can use Sensu filters in a similar capacity now, events that pass all filter evaluation may still be erroneously dropped by handlers that use the filter_repeated event occurrence filtering behavior in this library, and we cannot change this without breaking a few eggs.
To that end we're proposing the following steps:

[x] Make Sensu::Handler event filtering optional at both the top level and the occurrence filtering (filter_repeated) leven via check configuration, defaulting to enabled (next minor revision release)
[x] Document deprecation warnings in Sensu::Handler (next minor revision release)
[x] Switch default for Sensu::Handler event filtering from enabled to disabled (next major revision release)
[ ] Remove event filtering from Sensu::Handler altogether (next major revision + 1)

Related issues

https://github.com/sensu/sensu/issues/1067
https://github.com/sensu/sensu/issues/1092
https://github.com/sensu/sensu/issues/1097
https://github.com/sensu-plugins/sensu-plugin/pull/91
https://github.com/sensu-plugins/sensu-plugin/issues/88
https://github.com/sensu-plugins/sensu-plugin/pull/107

Edit: I've updated the description of this PR to indicate that the scope of my proposed change has narrowed to deprecating the event occurrence logic of the filter_repeated method.
Edit: I've changed this description again to describe the more granular implementation now being proposed in #136.

I also agree in moving this onto the core.  With the current behavior the handler is called on every event as a separate process, which is expensive.  This could be a landmine for users, as with checks set to 'standard' OK events aren't handled, but if something major happens (like a core router dropping) suddenly the majority of an environment could go into alert, effectively fork-bombing the Sensu server with handlers.  This may be a case many users have not tested for.
Sensu core supports filters now, but with the filter logic in the handlers I consider using both dangerous as it would be easy to create a filter setup that blocks all events, so no events are ever handled.  It would be very nice, at the minimum, to have an option in the base handler class to switch the default filter method to a no-op.    Perhaps this flag should be a subtask of this larger story?

@CVTJNII I opened #136 with a proposed implementation of deprecating the filtering behavior, including a per-check configuration to make filtering a no-op as you mentioned.

Cool.  One thing I would like to see is either have the current filter functionality moved into Sensu server or have a filter config block that reproduces this functionality added to the Sensu docs.  In reading the current filter docs I have to admit I'm not sure how to replicate all the current handler filter functionality, so having an example of how to do so would be super useful and I believe would de-duplicate effort otherwise required by users.  I'd like to see this be an explicit step to ensure it doesn't get forgotten and to make the transition easier for users.

@CVTJNII I agree regarding documentation, and I am working on enhancements to provide example filters which reproduce much of the functionality in sensu-plugin that I have proposed to remove.
There are aspects of filtering behavior in sensu-plugin which cannot be readily reproduced in native Sensu filters. Specifically, we recognize that there's currently no way to do things like query the stashes API for silenced checks/clients. Toward that end we've started to discuss silencing as a first class concept in https://github.com/sensu/sensu/issues/1328. We believe implementing this proposal will address that particular concern. Please have a look and let us know what you think?

I've published a post about this change on the Sensu blog: https://sensuapp.org/blog/2016/07/07/sensu-plugin-filter-deprecation.html

@cwjohnston do you see this ultimately ending with the removal of #api_request function?  We have handlers that will work with the event filtering but mostly our deregistration handler uses the #api_request function to remove autoscaled instances.

@zbintliff I don't think we'll be removing #api_request in the foreseeable future, but we can probably deprecate checking silenced stashes in this library before too long.
We're planning to move silencing into Sensu Core as a dedicated API endpoint and providing an filter extension to match (see https://github.com/sensu/sensu/issues/1328).
I think there's still a case to keep the #api_request method in this library for other operations which can't be neatly implemented as native filters, e.g. #filter_dependencies.

I've updated the title and description of this PR to indicate that the scope of my proposed change has narrowed to deprecating the event occurrence logic of the filter_repeated method.

In particular, the filter_repeated method uses logic to compare occurrences, refresh and status in order to determine if the event should be disregarded as a repeat. It's apparently prone to filtering events which should have been sent , often to indicate the outage was resolved.

I'm curious why the proposed built in occurrences filter  or even regular sensu filters would be in any better position to solve those issues.
IMO an occurrences filter needs to never filter resolve events except in the one case where all other events were filtered.  if I handled a create I should handle a resolve.   If I haven't handled a create I shouldn't handle a resolve.  Knowledge of wether all other events were filtered is not computable from information in the stored event AFAICT.  I think we might need to store if a filter was successfully applied in the stored event.

I have updated the built-in silencing proposal description to more accurately represent the required/desired functionality https://github.com/sensu/sensu/issues/1328

Version 1.4.0 with deprecations is now available on rubygems.org.

Hi @fessyfoo,

I'm curious why the proposed built in occurrences filter or even regular sensu filters would be in any better position to solve those issues.

You're right, it is not going to solve those issues immediately. In this case we're choosing not to change the existing implementation with the knowledge that it is flawed, but with the option to iterate on these things in the future with a more decoupled model.
As of this writing we've merged changes in sensu-extensions to enable loading extensions from gems, and we've moved the occurences filter into its own gem and we've opened a ticket for tracking the need for improvement in handling resolve events.
Internally we're tracking tasks for publishing other filter extensions which will demonstrate alternate approaches to event filtering, e.g. using last_ok & last_state_change.

IMO an occurrences filter needs to never filter resolve events except in the one case where all other events were filtered.

We agree here, but I believe that making a determination as to whether all other events were filtered will require access to the API. AFAIK there's actually nothing in the Sensu Core codebase that provides a canonical method for generating the API URI. Adding some reasonable way to access the API from extensions is probably a prerequisite for being able to make meaningful changes to this particular filter and unlocking other possibilities as well. See #122 for some related discussion.

I've opened #170 to disable deprecated filters by default as part of the 2.0 release.

We've released Sensu 0.29 today, which includes sensu-plugin versions 1.4.5 and 2.0.0.
The 2.0.0 release disables event filtering by default. :tada:

There is an issue with keepalive handling after migration to filters: every client has to opt out of the old event and occurrence filters, in their keepalive config.
This is not under control of the server in any way.
This kind of filtering exists on the handler side, why don't we have this tunable at the handler? Eg. instead of client.enable_deprecated_occurrence_filtering, we could have handler.enable_deprecated_occurrence_filtering.

@julian7 sensu-plugin 1.4.5, shipped in Sensu 0.29, adds support for globally disabling the deprecated filtering. See https://github.com/sensu-plugins/sensu-plugin/tree/v1.4.5#important for example configuration. Does this address your concern?

The remaining to-do described here is to completely remove filtering methods in a future major version release. I think this issue has served its purpose for now. Closing.

I just upgraded from 0.26.5 to 1.0.2, which was tricky for a completely different reason. Filtering works very smoothly, even legacy keepalive alerts are nicely filtered. It took a while, but it is totally worth it.


ITS / 18S

Jake asked if it would be possible to compute insertion trees not only for 16S but also for 18S and ITS. I think that would work in principle, however we would need to create reference trees for the according databases (Silva and Unite).
Any comments?

18S should be fine, good to benchmark. I'm not aware of an ITS tree that doesn't suck, and I do not believe UNITE exposes a phylogeny. ITS is not conserved and doesn't work for alignments over modest phylogenetic divergence.

I remember ITS is relatively structured RNA, do you know if people tried to come up with an alignment based on RNA structure? Would that be worth a short or is that obsolete because we are moving to shotgun anyway?

It's not structurally conserved. The alignments are atrocious.

Regarding Silva:
@wasade do you think we could go with http://ftp.arb-silva.de/qiime/Silva_128_release.tgz , decorate the tree (SILVA_128_QIIME_release/trees/99/99_otus.tre) via tax2tree with taxonomic labels from SILVA_128_QIIME_release/taxonomy/18S_only/99/consensus_taxonomy_7_levels.txt and produce a SEPP reference together with the alignment SILVA_128_QIIME_release/rep_set_aligned/99/99_otus_aligned.fasta.gz following Siavash's instructions?

Doesn't the tree they have already contain internal node labels?

nope. I checked all node.names, they are either floats or None (referring to SILVA_128_QIIME_release/trees/99/99_otus.tre)

It is not safe go decorate the taxonomy on and say its the Silva taxonomy
unfortunately
On Nov 13, 2017 1:16 PM, "Stefan Janssen"<EMAIL_ADDRESS>wrote:

nope. I checked all node.names, they are either floats or None (referring
to SILVA_128_QIIME_release/trees/99/99_otus.tre)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/biocore/q2-fragment-insertion/issues/10#issuecomment-343795298,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAc8srmMmrLQzz0s76tIE2oD_tK8J-nUks5s161-gaJpZM4QaQmE
.

Ok, than we might want to go ahead without taxonomic decoration. We need those only for the FeatureData[Taxonomy] output, which you don't like anyway ;-)
In principle, we could also obtain taxonomy by finding the closest reference tip and use this ID to look up linages in classical two column taxonomy files.


Instructions to use search?

get wording from tech services archivists group

this should not go on site, rather documentation. noted for workflow meetings https://docs.google.com/drawings/d/1W4w8qSI6re3RDiRyMWJISMrc0-ugycm_-UK-rbnQ2KM/edit?usp=sharing


Remove hacktoberfest references to PRs / Issues

mentioned in #386
We do hacktoberfest label on the repo level and it covers all PRs and issues within the repo.

resolved by #390


Annotations about DCIS1 cells

Hello, the annotations of the DCIS1 sc data you provided do not include annotations about tumor cells, but only nine annotations: 'B cell', 'EndoVas', 'Epithelial1', 'Epithelial2', 'Epithelial3', 'Fibroblast', 'Myeloid', 'NK/T', 'pDC'. However, annotations about tumor cells are shown in your study. Where can I see them?

This DCIS1 sc data comes from the paper "Spatial charting of single-cell transcriptomes in tissues". Our DCIS1 sc data corresponds to his DCIS2 sc data, including nine annotations: 'B cell', 'EndoVas', 'Epithelial1', 'Epithelial2', 'Epithelial3', 'Fibroblast', 'Myeloid', 'NK/T', 'pDC'. As shown in Supplementary Figure 9a of the paper "Spatial charting of single-cell transcriptomes in tissues", 'Epithelial3' can also be annotated as tumor cells.


Add uniswap simulator

Not yet used but will be to verify that interaction from naive amm
settlement works.
Test Plan
new unit tests

After discussing that we could just "try" to submit the solution on chain is this still needed? If so what are your thoughts around using the actual smart contract implementation instead of copying it locally?

I agree that we won't need this immediately. Will keep the PR open as draft in case we want it later.

What's the latest status on this? Can we close this PR?


Nepali Information addition and update.

Some data editing and adding some more Nepali Information

Please don't open an issue for everything you'd like to see in Faker - this doesn't scale.


Generate storage info for pallet im_online

Fixes part of #8629
Refactor Im Online to use WeakBoundedVec instead of Vec, and generate storage info for that pallet.
Polkadot companion paritytech/polkadot#3744
@shawntabrizi @KiChjang @thiolliere

CI is complaining:
error[E0277]: the trait bound `frame_benchmarking::Vec<<T as pallet::Config>::AuthorityId>: EncodeLike<frame_support::WeakBoundedVec<<T as pallet::Config>::AuthorityId, <T as pallet::Config>::MaxKeys>>` is not satisfied
  --> frame/im-online/src/benchmarking.rs:49:17
   |
49 |     Keys::<T>::put(keys.clone());
   |                    ^^^^^^^^^^^^ the trait `EncodeLike<frame_support::WeakBoundedVec<<T as pallet::Config>::AuthorityId, <T as pallet::Config>::MaxKeys>>` is not implemented for `frame_benchmarking::Vec<<T as pallet::Config>::AuthorityId>`
   |
   = help: the following implementations were found:
             <frame_benchmarking::Vec<T> as EncodeLike<&[U]>>
             <frame_benchmarking::Vec<T> as EncodeLike<VecDeque<U>>>
             <frame_benchmarking::Vec<T> as EncodeLike<frame_benchmarking::Vec<U>>>
   = note: required by `frame_support::pallet_prelude::StorageValue::<Prefix, Value, QueryKind, OnEmpty>::put`

Rebased my branch to make it cleaner and force the checks of the polkadot companion

bot merge

bot merge


Maven wrapper support

This PR adds maven wrapper support to this project to use Maven 3.6.3 explicitly in all cases (and by default if not installed).

👍
Did any of the mvnw support files need modification?

🤷♂ I don't think so, probably not.


Clarify that some plugin settings can be overridden by users

Fixes #124

Codecov Report

Merging #125 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #125   +/-   ##
=======================================
  Coverage   44.03%   44.03%
=======================================
  Files           8        8
  Lines         822      822
=======================================
  Hits          362      362
  Misses        433      433
  Partials       27       27

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 013533b...cc76080. Read the comment docs.


Raise a RegexpError if the \u{} escape is empty

This one had a rewrite after the changes of #2249. The Natalie regexp test now works with the self hosted compiler too.


ApiClient object missing

Hello,
It appears the ApiClient object is missing from the definitions.
More Specifically because swagger doesn't contain the ApiClient object, I don't see any actual way of authenticating the requests via the API.
The documentation calls for
`
import * as docusign from 'docusign-esign';
const apiClient  = new docusign.ApiClient();  // ApiClient doesn't exist
// use jwt authentication through apiClient object
`
I'm following the logic in your  auto-generated  Quickstart project for nodejs, the corresponding code location is on github.
Is there a new way to authenticate? If not could you help fix this? thanks!

Hi @code4ghana ,
The Swagger file includes the specification for the eSignature REST API itself. The ApiClient is how the SDKs keep track of the baseUrl and other information.
To obtain an access_token for the API, create a client_id (what DocuSign calls an integration key), and then use Authorization Code, Implicit, or JWT grant to get an access token.
Include the access token with each API call in an Authorization header.
See https://developers.docusign.com/platform/auth/choose/

Hi @code4ghana ,
The Swagger file includes the specification for the eSignature REST API itself. The ApiClient is how the SDKs keep track of the baseUrl and other information.
To obtain an access_token for the API, create a client_id (what DocuSign calls an integration key), and then use Authorization Code, Implicit, or JWT grant to get an access token.
Include the access token with each API call in an Authorization header.
See https://developers.docusign.com/platform/auth/choose/

Hey @LarryKlugerDS
thanks for the response! The ApiClient class also has methods that help with the jwt flow for you, so it was useful being able to abstract the auth-dance to it. The demo code explicitly uses that object so I was just surprised when I could not find it, perhaps the demo could be updated to not use it since it's not meant to be be used.
Most of this was prompted by me wanting to use typescript so I :

npm i docusign-esign
npm i  @types/docusign-esign
Getting missing/undefined object errors (e.g. EnvelopeDefinition) and realizing that the two libraries are out of sync (I believe the types library is out of date).
Decided to regen the newest types with Swagger, which lead to its own rabbit-hole.

My issue is solved, but I'll leave this bug open for you to decide whether any tasks should come out of it!
thanks again!

Hey @LarryKlugerDS
thanks for the response! The ApiClient class also has methods that help with the jwt flow for you, so it was useful being able to abstract the auth-dance to it. The demo code explicitly uses that object so I was just surprised when I could not find it, perhaps the demo could be updated to not use it since it's not meant to be be used.
Most of this was prompted by me wanting to use typescript so I :

npm i docusign-esign
npm i  @types/docusign-esign
Getting missing/undefined object errors (e.g. EnvelopeDefinition) and realizing that the two libraries are out of sync (I believe the types library is out of date).
Decided to regen the newest types with Swagger, which lead to its own rabbit-hole.

My issue is solved, but I'll leave this bug open for you to decide whether any tasks should come out of it!
thanks again!


Vulnerability roundup 57: libsndfile-1.0.28: 4 advisories

search, files

[ ] CVE-2018-19432 (nixos-unstable, nixos-18.09)
[ ] CVE-2018-19661 (nixos-unstable, nixos-18.09)
[ ] CVE-2018-19662 (nixos-unstable, nixos-18.09)
[ ] CVE-2018-19758 (nixos-unstable, nixos-18.09)

Scanned versions: nixos-unstable: 44b02b52ea6; nixos-18.09: c840631fe20. May contain false positives.

See also #47122

Opened Upstream issue asking for new releases https://github.com/erikd/libsndfile/issues/436 .

Still no new release from upstream. Project looks abandoned.


Investigate the use of shiki as the library for syntax highlighting

Proposal
Use shiki, https://shiki.style/, as the syntax highlighting library for mystmd. It uses the same syntax highlighting engine as vscode and has very good tooling.

It uses TextMate grammars for its language definitions. Which is the same definitions that are used by vscode and GitHub's linquist among others.

The output it produces is very good.
there is a large number of high-quality language definitions already defined.
It also allows reusing vscode color themes.

It is supports being run in a compilation step for static generation and at runtime in the end users browser.
It has a extensive API for defining transformations and decorations

Additional notes
It is used in a lot of open source projects

Astro
Nuxt
vitepress

I recall seeing this mentioned somewhere ... turns out it was you! https://discord.com/channels/1083088970059096114/1083088970059096117/1295937103846314004

Yep! I finally have some free time to work on some contributions to a few projects, so I wanted to open the floor to see if this is something that would be desired.

I recall seeing this mentioned somewhere ... turns out it was you! https://discord.com/channels/1083088970059096114/1083088970059096117/1295937103846314004

I also forgot to add that it is really easy to add custom languages to shiki as the consumer of the library. I think that might be useful if someone wanted to have syntax highlight for a custom languages (such as if it was a book about language design/compilers and they made up a simple language) (See https://shiki.style/guide/load-lang)

Thanks for this idea @Nova38 ! I wanted to share a few quick questions to help us triage this idea a bit. I promise they're not coming from a place of skepticism - just trying to understand cost/benefit!

What's the benefit of using this over the current system we use for syntax highlighting? (in functionality, maintenance cost, long-term outlook, UX, etc?)
How much work would it be to implement and then maintain this?
Why should this be the most important thing to use maintainer time implementing/reviewing/etc?

The syntax highlighter is used in the myst-theme package here:
https://github.com/jupyter-book/myst-theme/blob/main/packages/myst-to-react/src/code.tsx#L106

Here are some of my initial thoughts

What's the benefit of using this over the current system we use for syntax highlighting? (in functionality, maintenance cost, long-term outlook, UX, etc?)

An example of what is possible, here is it intigrated with typescripts Shiki Twoslash: Static Code Samples for JS Projects - https://shikijs.github.io/twoslash/

The UX of adding new transfomations is a lot more consise

Functionality:

Uses .tmlanguage definitions, which has more languages that prism/highlight and its definitions are better.
Allows access to preform transforms using hast  https://github.com/syntax-tree/hast
It provides a cleaner api than the current one for adding decorations Decorations | Shiki - https://shiki.style/guide/decorations

import { codeToHtml } from 'shiki'
const html = await codeToHtml(code, {
theme: 'vitesse-light',
lang: 'ts',
decorations: [
{
  // line and character are 0-indexed
  start: { line: 1, character: 0 },
  end: { line: 1, character: 11 },
  properties: { class: 'highlighted-word' }
}
]

It has some built in transformations that could improve code readability such as the @shikijs/colorized-brackets | Shiki - https://shiki.style/packages/colorized-brackets and Shiki Magic Move Demo - https://shiki-magic-move.netlify.app/ Usng

Longterm outlook

Language definitions are likely to keep being developed that are compatible as it is used by a lot of tools
Built on top of some of the libraries used by vscode (oniguruma)
Integrated in to multiple large projects/frameworks (Nuxt, Astro, vitepress, etc... )'
They have a rough roadmap Future | Shiki - https://shiki.style/guide/future
Some of the core developers also are core developers on things like vitest, nuxt, etc...

Maintenance cost

Shiki has a diffrent way of parsing what they call as 'meta' strings for customizing the front ends.

@shikijs/transformers | Shiki - https://shiki.style/packages/transformers
// [!code highlight]
We could still use the css and other parts of their transform and just add the classes in the transform that we get from the myst directive props.
I am pretty sure that any of these have to be manually enabled so we don't want them to work it should be simple to just not add them

How much work would it be to implement and then maintain this?

Honestly I am not sure how long it would take to implement. It doesn't look like it would take much to add it. To maintain it after the initial setup would probably be minimal. I can make a POC to see what the code will look like. I Think that it might be less work maintaining the current code of the code highlighter.

Why should this be the most important thing to use maintainer time implementing/reviewing/etc?

It could be used to further expand what mystmd can do with code annotations

Is there a way to override an existing component and inject different project specific component?

@Nova38 no not on the renderer/theme/react side of things, this is part of the plugin framework that has yet to be fleshed out.
+1 for shikijs being pure ESM though. We've had to wrangle with react-syntax-highlighter at times due to the way it is packaged, trying to be a cjs/esm distribution.

I am not sure that I entirely follow the question. Are you wanting how it by itself can be adapted into a plugin itself, how it could fit in by themselves exposing parts of it that other plugins could then be able to expand on it and add new features to the code blocks, or is it something else?

Do you mean about how it allow things like decorations to code blocks be added by plugins. If so then there is actually a really nice api that could definitely be hooked into by the plugin system. They have some documentation about it here https://shiki.style/guide/decorations. I have also included an example they have of using that API directly.
import { codeToHtml } from 'shiki'

const code = `
const x = 10
console.log(x)
`.trim()

const html = await codeToHtml(code, {
  theme: 'vitesse-light',
  lang: 'ts',
  decorations: [
    {
      // line and character are 0-indexed
      start: { line: 1, character: 0 },
      end: { line: 1, character: 11 },
      properties: { class: 'highlighted-word' }
    }
  ]
})

And here is the rendered result:

A good example of using what this can allow can be seen in the first party twoslash plugin.  https://shiki.style/packages/twoslash
The integrating it into the plugins system could be done by allowing  plugins could generate some of the objects passed into the decoration array or by allowing the definition of something like their transformation API (https://shiki.style/packages/transformers)

@Nova38 no not on the renderer/theme/react side of things, this is part of the plugin framework that has yet to be fleshed out.
+1 for shikijs being pure ESM though. We've had to wrangle with react-syntax-highlighter at times due to the way it is packaged, trying to be a cjs/esm distribution.

Also I was wondering, would it be better to include it in the myst-to-html or its own exetention instead on the react myst theme as it can render to html without needing to load the libraries in the browser. That way it will reduce the amount of JavaScript on the end users page and simplify the pipeline? In particular shiki's transform syntax uses a different style of comments than mystmd used to pass arguments. If we wanted to reuse some of the existing ones being able to convert the mystmd args/context into those comments/meta string would probably be better done statically. Their decorations API on the other hand is also powerful enough that we would just need to generate the list of decorations we wanted.
https://github.com/jupyter-book/mystmd/tree/main/packages/myst-to-html

The syntax highlighter is used in the myst-theme package here:
https://github.com/jupyter-book/myst-theme/blob/main/packages/myst-to-react/src/code.tsx#L106

Just wanted to note that we have very limited bandwidth on the project right now, so there's a good chance that discussions like this will take a while to play out. We appreciate all of this research and consideration here, there's just too much to do and not much time to work on the project :-)
To your last point, I think that this issue might be relevant to the point you're making:

https://github.com/jupyter-book/mystmd/issues/1698


Multiple parallel streams

Again, I'm trying to integrate Meteor with Highlandjs so this may seem like an odd question. Basically I want to know if you can have two streams going side by side, so I can do something like .map(function(stream1values, stream2values) { ...
Heres' the motivation:
sub = Meteor.subscribe( ... ) will subscribe to data from the server. It returns an object so you can call sub.stop() when you want to clean up.
I created a function that will do this cleanup for me.
# will stop a meteor subscription when rerun or stopped
subscribe = _.curry((f, source) ->
  sub = null
  source.consume((err, x, push, next) ->
    if err
      push(err)
      next()
    else if x is _.nil
      sub?.stop?()
      push(null, x)
    else
      sub?.stop?()
      sub = f()
      push(null, x)
      next()
  )
)

sub.ready() is a "reactive" function that will tell you if the subscription is ready, i.e. the server is done sending the data to the client. I created a function that will convert this meteor reactivity into a highland stream:
autorunStream  = (f) ->
  stream = _()
  c = Tracker.autorun ->
    results = f()
    stream.write(results)

  stream.observe().through(onDone(() ->
    c.stop()
  ))
  return stream

Thus, we can now do this readyStream = autorunStream(-> sub.ready())
So here's what I have so far:
search.fork()
  .debounce(500)
  .through(subscribe((query) ->
    Meteor.subscribe('search', query)
  )).map (query) ->
    # return the results of the query

However, I want to render a loading template while the subscription isn't ready, and render the results once it is ready. What I want to be able to do is something like this:
renderStream = search.fork()
  .debounce(500)
  .through(subscribe((query) ->
    Meteor.subscribe('search', query)
  )).map (query, ready) ->
    if ready
      # return an html string of the results based on the query
    else
      # return an html string that says loading...

I'm not sure this is possible or even a good way of doing it. The idea is to have the query stream and the ready stream side by side. Its as if they're merged (the map will run when either changes), but the values are separated and passed as separate arguments.
At the end of the day, renderStream needs to be a stream of renderable strings.
Thanks for the help

First of all, autorunStream shouldn't be using observe. An observer cannot affect the source stream. Since you observe stream but return the observer, you'll never ever get any data (since nothing will resume stream). onDone does data passthrough anyway, so you don't even need the observer. It's done (which is a terminal operation) that needed the observe so that it doesn't resume the source stream too early.
Secondly, you'll want to always end streams (unless what you have is logically an infinite stream).
  c = Tracker.autorun ->
    results = f()
    stream.write(results)
    stream.end() # Should end the stream.

Onto the main question. I can't think of an easy way to implement what you're asking for, but I don't think it's necessary. The easiest way is to have subscribe also provide the subscription, so
sub?.stop?()
sub = f()
push(null, [x, sub]) # I made an array, but it could be an object if you want
next()

then use map to return a stream that first emits your loading text, then emit your data once sub.ready() returns.
map(function (arry) {
    var query = arry[0],
        sub = arry[1];

    var data = autorunStream(sub.ready.bind(sub))
        .map(renderData.bind(null, query));

    return _(function (push, next) {
        push(null, "Loading...");
        next(data); // Delegate to the data stream.
    });
});

function renderData(query) {
    // Do render here.
}

Then you use switch to always stay with the latest query.
var renderStream = search.fork()
    .debounce(500)
    .through(subscribe(Meteor.subscribe.bind(Meteor, 'search')))
    .map(function (arry) {
        ...
    })
    .switch();

Hmm. So I've come up with this concept of "joining" streams where you and stack multiple streams next to each other. Here's the basic concept in the form of a test:
_ = highland
a = _()
b = null
a.through(join(function(x){
  b = _()
  b.write(x*2)
})).through(multi(_.each, function(x,y){
  console.log(x,y)
}))
a.write(1)
// => 1, 2
b.write(3)
// => 1, 3
b.write(4)
// => 1, 4
a.write(3)
// => 3, 6
b.write(7)
// => 3, 7

join maps over a source stream creating child streams. These child streams are ended when the next value from the source is emitted (switchOnNext). It returns a multiStream which is an array of the values in both streams whenever either stream changes.
multi simply assumes that a stream has an array of values meant to be passed as arguments to some transform.
Anyways, here's what I've got so far:
join = _.curry((f, source) ->
  multiStream = _().through(stopOn(source))

  val1 = null
  val2 = null

  emit = ->

    if val1 instanceof Array and val2 instanceof Array
      multiStream.write(val1.concat(val2))
    else if val1 instanceof Array
      multiStream.write(val1.concat([val2]))
    else if val2 instanceof Array
      multiStream.write([val1].concat(val2))
    else
      multiStream.write([val1, val2])

  # not observe?
  source.fork()
    .map(f)
    .through(switchOnNext)
    .each (x) ->
      val2 = x
      emit()

  source.fork().each (x) ->
    val1 = x
    emit()

  return multiStream
)

@multi = (transform, f) ->
  (source) ->
    source.through(transform((arr) ->
      f.apply(null, arr)
    ))

takeUntil = _.curry((stream, source) ->
first = true
done = false
source.consume( (err, x, push, next) ->
if first
stream.pull ->
if not done
push(null, _.nil)
done = true
first = false
if err
push(err)
next()
else if x is _.nil
done = true
push(null, x)
else if not done
push(null, x)
next()
)
)
switchOnNext = _.curry((source) ->
stop = null
source.flatMap (maybeStream) ->
if _.isStream(maybeStream)
stream = maybeStream
if stop
stop.end()
stop = null
stop = _();
stream.through(takeUntil(stop))
else
_([maybeStream])
)
onDone = _.curry((f, source) ->
source.consume((err, x, push, next) ->
if err
push(err)
next()
else if x is _.nil
f()
push(null, x)
else
push(null, x)
next()
)
)
stopOn = (stream, source) ->
stream.through(onDone(-> source.end()))
a function creates a stream and we get back 2 parallel streams

I don't have time to look closely right now, but a few things that I noticed:

You can only call write/end on a stream that was constructed with no generator (i.e., _()). That's the only case where we guarantee that it works. Your multiStream was not constructed this way.It's actually the child stream of the _() that was created by your stopOn function.

It's generally not needed (or encouraged) for you to use the write/end API when writing transforms. It's mostly meant for interop with Node streams. In some cases though (like switchOnNext), it is easier to use write, and that's fine.

fork is probably fine in this case, but you may not don't want to call each synchronously like that.

What you probably want is something similar to zipAll, except without waiting for all zipped streams to emit a value (L2141) and without reseting the array every time you emit some thing new (L2153). Though maybe it is worth emitting copies of the buffer instead of the buffer itself as a defensive measure. Note that a zipAll-like implementation would make stopOn no no longer necessary.

Maybe call this snapshot? For viewing the current state of a stream of streams?

Then, you can implement join by using snapshot. Generally, when writing transforms, you want to keep their scope as small as possible, so that they're easier to compose.

Oh and your switchOnNext seems weird. If you get a thing that's not a stream, you turn it into a stream that doesn't get stopped once the next thing gets push? If you just want to coerce every element into a streams, this works just as well
stream.map(_) // Apply the constructor == coerce
    ...

You can also just blindly call the constructor and it'll do the correct coercion for you.
stream = _(maybeStream);
if (_.isStream(maybeStream)) {
    console.log(stream === maybeStream) // => true
}

Hmm, I was under the impression that this could create a new stream _() that stops when the source stream stops .through(onStop(source)).

stopOn = (stopStream, source) ->
  stopStream.through(onDone(-> source.end()))

@join = _.curry((f, source) ->
  multiStream = _().through(stopOn(source))
  # ...
)

Or are you saying that this would be better instead?
@join = _.curry((f, source) ->
  multiStream = _()
  # ...
  return multiStream.through(stopOn(source))
)

2 and 3. So you're right that I don't actually want to call each, but the funky thing is that multiStream is disconnected from the source and the parallel stream in terms of consumption.
  multiStream = _().through(stopOn(source))

  # parallel stream
  source.fork()
    .map(f)
    .through(switchOnNext)
    .map (x) ->
      val2 = x
      emit()

  # source stream
  source.fork().map (x) ->
    val1 = x
    emit()

Suppose I call multiStream.each(_.log). Nothing should happen because the source stream consumed.

How this stream ended without calling switchOnNext or something like that?

So this turns out to be a rather elegant solution :)
join = _.curry((f, source) ->
  source.map (x...) ->
      f.apply(null, x).map (y...) ->
        return x.concat(y)
    .through(switchOnNext)
)

multi = (transform, f) ->
  (source) ->
    source.through(transform((arr) ->
      f.apply(null, arr)
    ))

Here's how it tests out:
_ = highland
a = _()
b = null
a.through(join(function(x){
  b = _()
  b.write(x*2)
  return b
})).through(multi(_.each, function(x,y){
  console.log(x,y)
}))

a.write(1)
// => 1 2
a.write(2)
// => 2 4
b.write(5)
// => 2 5
a.write(5)
// => 5 10

It even works for piling on more streams!
corceArray = (x) ->
  if x instanceof Array
    x
  else
    [x]

join = _.curry((f, source) ->
  source.map (x) ->
      x = corceArray(x)
      f.apply(null, x).map (y) ->
        y = corceArray(y)
        return x.concat(y)
    .through(@switchOnNext)
)

multi = (transform, f) ->
  (source) ->
    source.through(transform((arr) ->
      f.apply(null, arr)
    ))

_ = highland
a = _()
b = null
c = null
a.through(join(function(x){
  b = _()
  b.write(x*2)
  return b
})).through(join(function(x,y){
  c = _()
  c.write(x+y)
  return c
})).through(multi(_.each, function(x,y,z){
  console.log(x,y,z)
}))
a.write(1)
// 1 2 3
a.write(2)
// 2 4 6
b.write(3)
// 2 3 5
b.write(5)
// 2 5 7
c.write(10)
// 2 5 10
a.write(4)
// 4 8 12

Nice! I was thinking too general-case where you don't have the luxury to create a new stream every time the source emits a new value. This was probably me misunderstanding the use case a bit.

Hmm, I was under the impression that this could create a new stream _() that stops when the source stream stops .through(onStop(source)).
...
Or are you saying that this would be better instead?

Yep. That's the correct way to do it. Before, you were essentially writing to the wrong thing. What you had was equivalent to
var src = _();
var multiStream = src.through(onStop(source));
multiStream.write(...);

What you actually want is to write to src.

but the funky thing is that multiStream is disconnected from the source and the parallel stream in terms of consumption.

Yeah, you do have to call each. I just meant you might want to do it lazily.
multiStream = _(function (push, next) {
// Call `each` in here. This won't be called until you need the data.
// Then `emit` can use `push` instead of `write`
});

How this stream ended without calling switchOnNext or something like that?

That stream ends when nil is pushed here.


Broken Images in graphql docs

Description:
Broken Images in graphql docs

Fixed in https://github.com/wso2/docs-apim/pull/830


Add content on remotes, documentation and the workshop project

Deployed to http://hbz.github.io/swib16-workshop
Will partially resolve #1

I committed some small improvements. Generally, I really like the structure and how exercise and lessons built on each other.
We should think about facilitating the understanding of different types slides by better distinguishing them visually, see #14.
Another small thing I noticed: regarding the process in https://hbz.github.io/swib16-workshop/#/15/4 we should make clear that this is one possible way of many for structuring a Kanban and that it is the approach we chose.

Please merge if you are ok with my adjustments.

I find the 'non-developer' term a bit odd, I think because it's kind of dev-centric by defining all other roles as non-dev, but I see your point. Will merge for now, maybe we'll find something better later.


Added Support for reading and writing UTF-8 arrays

Added support for reading and writing UTF-8 arrays from C++ client. Aimed to enhance interoperability with Java clients.

Hello mstier,
Can you rebase and resubmit the PR? Can you also add a test for this so that we can merge it? Thanks for the PR.

verify

Hi @mstier ,
To be able to accept your pr we need you to sign our contributor agreement. You can find it here.
https://hazelcast.atlassian.net/wiki/spaces/COM/pages/6357071/Hazelcast+Contributor+Agreement

Linux test PASSed.

Windows test PASSed.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 3 committers have signed the CLA.:x: devOpsHazelcast:x: Markus Stier:x: sancarMarkus Stier seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Can one of the admins verify this patch?

@mstier Are you still working on this PR? We need to add a test to verify the changes and also we need to rebase the PR, can you do them?

Fix is merged by https://github.com/hazelcast/hazelcast-cpp-client/pull/596


Add Russian locale

I translated everything into Russian. But I don't know if I need to include a translation file somewhere in the code.
There are some inaccuracies in the translation. The word modal has no direct translation into Russian. That's why I used the closest word in the meaning of pop-up message (всплывающие сообщение).

Perfect, thank you. :)


Add the "type" and "property" to the SciData class

In the SciData framework, in the @graph section, there is the the scidata.type (usually "property value") and scidata.property:
https://github.com/stuchalk/scidata/blob/master/examples/sections/framework_plus.jsonld#L32-L33
Would like to include a method that allows for including these values in the SciData object.

I have decided to create a discussion topic to address this suggestion...

@stuchalk sounds good, thanks!
Should I hold off on trying to implement this?
I am already working on it now (finished with "type", working on "properties") but can pause, especially if we don't want to add these.

#54

@stuchalk based on #54 I am fine to close this issue since will go with the ids instead.
Sound good?

As I said in the discussion, if you feel you don't need these metadata elements and can use the 'ids' right now then close this issue.  We can always reopen it later if needed...

Sounds good and I think I can right now :+1:


Need help resolving prerelease NuGet packages

I am having a very difficult time getting NuGet prerelease packages to resolve properly and would really appreciate some help. :smile:
What I would like to do is install the latest available of a package, including prereleases. In the example below, I expected Paket to find Microsoft.AspNet.Mvc 6.0.0-beta6. In the future when v7.0 is released, then I would expect that to be resolved.
source https://nuget.org/api/v2
nuget Microsoft.AspNet.Mvc >= 6.0.0 prerelease

$ .paket/paket.exe install
Paket version <IP_ADDRESS>
Resolving packages:
Paket failed with:
        Could not find compatible versions for top level dependency:
     "Microsoft.AspNet.Mvc >= 6.0.0 (from y:\src\temp\paket.dependencies)"
   Available versions:
     -
   Try to relax the dependency or allow prereleases.

When I use a pessimistic version range with beta explicitly set e.g. ~> 6.0.0-beta it is able to resolve. Am I using the right syntax? Based on the examples, I had thought the former would work.
Thanks!

This is a common error. And I will change the semantics soon so that people
don't run into this anymore.
The issue is that prerelease versions are smaller than 6.0 so you filter
these out.
To solve your issue use only prerelease without any version filter. Or use
something like >= 5 prerelease.
On Aug 13, 2015 12:16 AM, "Carolyn Van Slyck"<EMAIL_ADDRESS>wrote:

I am having a very difficult time getting NuGet prerelease packages to
resolve properly and would really appreciate some help. [image: :smile:]
What I would like to do is install the latest available of a package,
including prereleases. In the example below, I expected Paket to find Microsoft.AspNet.Mvc
6.0.0-beta6
https://www.nuget.org/packages/Microsoft.AspNet.Mvc/6.0.0-beta6. In the
future when v7.0 is released, then I would expect that to be resolved.
source https://nuget.org/api/v2
nuget Microsoft.AspNet.Mvc >= 6.0.0 prerelease
$ .paket/paket.exe install
Paket version <IP_ADDRESS>
Resolving packages:
Paket failed with:
Could not find compatible versions for top level dependency:
"Microsoft.AspNet.Mvc >= 6.0.0 (from y:\src\temp\paket.dependencies)"
Available versions:
-
Try to relax the dependency or allow prereleases.
When I use a pessimistic version range with beta explicitly set e.g. ~>
6.0.0-beta it is able to resolve. Am I using the right syntax? Based on
the examples, I had thought the former would work.
Thanks!
—
Reply to this email directly or view it on GitHub
https://github.com/fsprojects/Paket/issues/976.

Thank you very much for the quick reply! I'm slapping my forehead for not realizing sooner what was going on. :blush:

no as I said it was a common misunderstanding and it is now fixed so that people get what they expect.


ImportError: cannot import name 'TypedDict' from 'typing'

It seems that this sdk cannot run with python 3.7, because package "typing" does not supported python 3.7 but python 3.8+.

you are right


withParam expanded step names with . in item key or value causes map nesting issues

Pre-requisites

[X] I have double-checked my configuration
[X] I can confirm the issues exists when I tested with :latest
[ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?
When using withParam, the expanded step name includes keys and values from the item json object. These key and value
string can contain .. This causes issues with the environment map that's passed when evaluating expression. This is because the environment map splits strings on ..
Example
expanded step name: with-param-example(0:url:github.com)
expected reference:
steps["with-param-example(0:url:github.com)"].status == "Blah"

actual reference:
steps["with-param-example(0:url:github"]["com)"].status == "Blah"

I encountered this problem when trying to work with hook expressions for withParam steps. For my use case, I was checking the status of the step for a hook (I would be able to figure out the expanded step name using the item.Url as it's unique).
Hook expression: steps[filter(sprig.keys(steps), { {{ item.Url }} in # })].status == "Succeeded"
However, this breaks due to splitting keys on . when crafting the environment map for evaluating, so additional work is needed to navigate the env map.
Version
v3.4.1
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: steps-with-param-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: use-with-param
        template: whalesay
        arguments:
          parameters:
          - name: message
            value: "{{item.Url}}"
        withItems:
          - {"Url": "https://github2.com"}
          - {"Url": "https://github1.com"}
        hooks:
          test:
            expression: 'steps[filter(sprig.keys(steps), { {{ item.Url }} in # })].status == "Succeeded"'
            template: whalesay
            arguments:
              parameters:
                - name: message
                  value: "hook ran!"
  - name: whalesay
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

Logs from the workflow controller
N/A

Logs from in your workflow's wait container
N/A

Tangentially related to https://github.com/argoproj/argo-workflows/discussions/10573 , as these workarounds wouldn't really be needed if it was easy to reference the expanded step status from its own hook. Something like:
hooks:
  onFail:
    expression: 'hook.current_step_status == "Failed"'

Although, I'd imagine this expanded step . issue could cause problems depending on how solutions are designed+developed.

Bump.

+1


Bgp use case

Please go through this link for use case
https://github.com/open-traffic-generator/snappi-ixnetwork/blob/bgp_use_case/docs/deviceUsecase/deviceBgpUsecase.md

@winstonliu1111 comment related to https://github.com/open-traffic-generator/snappi-ixnetwork/pull/451#pullrequestreview-765585405
Scenario 9 is a 'Not Supported' use case. I have put it here so everyone in same page. I have removed IxNetwork mapping which is confusing. And put comment that we will raise error from snappi_ixnetwork.


Refine and update the old README

The README is a bit outdated. We should refine it.
Ideas:

First section should be something like "what epinio is, and what problem does it solves", and the links to docs, sites and other references.
remove the contents section (too long and not really needed)
remove the old references (i.e.: linkerd)
update configurations and services
put some quickstart references (how to install from charts for a standalone installation, on rancher and how to install the CLI). Eventually add links to the docs)
some quick examples commands?
Reach us section !important!
Update License, keep contributing (OSS)
Drop buildpacks reference

### Tasks
- [x] new first section (what is it, what problem is solved)
- [x] remove table of contents
- [x] remove old references (linkerd)
- [ ] update configuration and service information
- [x] add quickstart references
- [x] quick example commands
- [ ] highlight section `reach us`
- [ ] update license, contribution information
- [ ] drop buildpack refs

Notes:

We already have a what does it solve section - https://github.com/epinio/epinio#what-problem-does-epinio-solve
We already have a quick start reference - https://github.com/epinio/epinio#usage
This contains the example commands as well


lxml error

when i run skiptracer.py i get this:
Traceback (most recent call last):
File "skiptracer.py", line 4, in
import plugins.proxygrabber as pg
File "C:\Users\maxen\OneDrive\Documents\maison\1Maxence\skiptracer-master\plugins\proxygrabber.py", line 4, in
from lxml.html import fromstring
ImportError: No module named lxml.html

have you installed the dependencies from the requirements.txt yet? that was not enough to get it working, i found i had to manually install lxml with the following:
sudo apt-get install python3-lxml
sudo apt-get install libxml2-dev libxslt-dev python-dev

have you installed the dependencies from the requirements.txt yet? that was not enough to get it working, i found i had to manually install lxml with the following: sudo apt-get install python3-lxml sudo apt-get install libxml2-dev libxslt-dev python-dev

i got the same error as Om3gaa, i did what you've suggested, (sudo apt-get install python3-lxml and sudo apt-get install libxml2-dev libxslt-dev python-dev) and i got the same error when i tried running skiptracer.py
also if you didn't know this program uses python 2.7 so i went from python3 to python-2.7 and it still gave me an error


Inability to search facility users from Assign Pickers list for Open orders in Fulfillment App

Expected behavior
Users should be able to utilize a search feature to efficiently locate facility users from the assign pickers list.
Steps to reproduce the issue
Open the fulfillment app and navigate to the section for assigning pickers to open orders.
Attempt to locate a specific facility user from the assign pickers list.
Notice that there is no search functionality available to filter and locate users based on specific criteria.
Scroll through the list manually to find the desired facility user.
Environment details
App version : MAIN-F2FA0F7
App testing environment : DEV
Additional information
https://jam.dev/c/ee55cdcf-f4c9-4a5f-aca3-6a7ec6ceea15

Duplicate issue has been raised and fixed.
https://github.com/hotwax/fulfillment-pwa/issues/506
Hence, closing this issue.


Add content in About App

About app screen is empty. I woul like to work on this issue @cdhiraj40 . I would use material design for UI.
I will get review the written content from you once.

hello @Kajal13081, you are indeed correct. we have talked about this and we actually want to add it. I think you can propose a way the page look or maybe you can wait until we are ready with the about page. maybe you can make PRs for other issues till then? Thanks :)

Yes sure. I'll work on other opened issues for now.


Enhancement: Support Upsert (over Merge) when calling the 'Merge' operation.

Other providers do not support 'Merge' statement (ie: SqLite). When the developers call the 'Merge' operation, the RepoDb.Core must be able to automatically switch its context to use the 'Insert/Update' operation instead.
It should be a requirement to use the existing operations below.

Exists
Insert
Update

Also, the upsert feature must be configurable. The IDbSetting object is the candidate object to handle the settings for the Database Provider specific context.

This will be available on the builds > v1.9.11.

This is now available at RepoDb releases >= v1.10.0.


Support Ruby 3.2

File.exists? and Dir.exists? have been removed in Ruby 3.2 in favor of File.exist? and Dir.exist, respectively.
File.exists? has been around since before Ruby 1.0, and Dir.exists? has been a method since Ruby 1.9.0, so backwards compatibility is not a concern here.

It's actually exist? that you want to demonstrate historical support for, not exists? which is being removed. You're switching to exist?. So you want to demonstrate this switch is valid going back to some impressively old implementation, right? (e.g. https://github.com/ruby/ruby/blob/v1_9_0_0/file.c#L4445 and https://github.com/ruby/ruby/blob/v1_9_0_0/dir.c#L1906).

What options do we have here for CI, so we can validate the test suite is sufficient and passes? GitHub Actions? CircleCI? Team City? Etc.?

GitHub Actions is what everyone uses.

Ah right. Thanks for catching that. I've updated the PR description with the correct references.


adding procedures back into table of contents

Ah, at the time that I changed the formatting of those headings, I didn't realize that the TOC depended on them. I'm not sure how the reverted formatting plays in the product doc, but if a further modification is needed, we'll reassess later.


Adding PHP

Thanks for the contribution, just need you to sign the CLA.

Thanks for the contribution, just need you to sign the CLA.


Constantly alternating object ID in pipelines using shortterm tracking algorithm

Regression in 2021.1 release compared with 2020.4 regarding object detection label IDs and bounding boxes in pipelines with gvatrack(shortterm tracking).  In 2020.4, the label IDs are smooth and constant, where in 2021.1, the labels are constantly alternating almost every second between different IDs.


Import bug when trying to export model to ONNX

System Info
Name: transformers
Version: 4.31.0
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email<EMAIL_ADDRESS>License: Apache 2.0 License
Location: /opt/conda/lib/python3.10/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: auto-gptq, optimum, sentence-transformers
---
Name: onnxruntime
Version: 1.15.1
Summary: ONNX Runtime is a runtime accelerator for Machine Learning models
Home-page: https://onnxruntime.ai
Author: Microsoft Corporation
Author-email<EMAIL_ADDRESS>License: MIT License
Location: /opt/conda/lib/python3.10/site-packages
Requires: coloredlogs, flatbuffers, numpy, packaging, protobuf, sympy
Required-by: chromadb
Note: you may need to restart the kernel to use updated packages.

Running on GCP - Vertex AI Workbench with A100 84GB GPUs.

Who can help?
Export of transformers model to ONNX/TFLite: @michaelbenayoun
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction
from optimum.onnxruntime import ORTModelForCausalLM
from transformers import AutoTokenizer
model_checkpoint = "TheBloke/Vicuna-13B-1-3-SuperHOT-8K-GPTQ"
save_directory = "onnx/"
ort_model = ORTModelForCausalLM.from_pretrained(model_checkpoint, export=True)
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
ort_model.save_pretrained(save_directory)
tokenizer.save_pretrained(save_directory)
Expected behavior
Trying to export to ONNX a LLaMA-based Vicuna model that is supposed to be able to be exported according to: https://huggingface.co/docs/optimum/exporters/onnx/overview
I get library/dependencies errors when trying to run my code. Is there anything that I'm doing wrong or the errors that I'm getting are showing up because Vicuna model is not supported. Would appreciate any help I can get

What's the error you get?

Hi @nicoFerreira95 , GPTQ uses custom CUDA/triton kernels from external libraries to handle the mixed int4/fp16 compute, it is expected that the ONNX export fails.


Compiling copies RDLC-files in the root folder

I put the reports and corresponding RDLC-files in a Report-Folder. When I compile the solution they will copied to the root-Folder of the solution. I would be nice if this didn't happen.

Are you sure that rdlc file is copied? If compiler cannot find rldc file, it creates a new, default one. All paths to files specified in reports and control add-ins definitions are not relative to the location of the al file but are relative to the root of the project. If you moved both your report al and rdlc files to the Report folder, you have to update RDLCLayout property and include that folder in the path.

Hi @anzwdev,
you are right. This newly generated RDLC-Files only contained the Field-Definition. I changed from RDLCLayout = './<filename>.rdlc'; to RDLCLayout = './Reports/<filename>.rdlc'; in every Report and everything works as expected.


How to approach pricing at the deepest option level?

While building a provider for Reaction Commerce, I stumbled onto an issue that I think many other developers will have with other APIs.
The structure for product variants is as follows:
iPhone 12 (main product)
  |
  |
  |--- Black (variant)
  |       |
  |       |--- 64GB (sub-variant) - $699
  |       |--- 128GB (sub-variant) - $799
  |       |--- 256GB (sub-variant) - $899
  |
  |
  |--- Purple (variant)
  |       |
  |       |--- 64GB (sub-variant) - $799
  |       |--- 128GB (sub-variant) - $899
  |       |--- 256GB (sub-variant) - $999

With this structure, the sub-variants (or the sub-sub-variants, or whichever deepest level) is the one that bears a price. There is no price at the variant level (unless it's the deepest), nor at the main product level.
As I understand it, the approach of Next Commerce is to have the price at the variant level (which is expected to ideally be colors), and then however many MultipleChoiceOptions we want, but which won't influence the price of the item.
Is that correct? If so, how would you go about making a variant structure like Reaction's work with Next Commerce without having to touch too many core files?

Turns out I was just misunderstanding how this whole MultipleChoiceOption worked. Maybe it should be better documented, but at least we got it to work on our provider, so I'm closing this.


Sort version numbers as version numbers.

Currently the version numbers are sorted as strings (e.g. 0.1, 0.10, 0.11, 0.2) rather than as version numbers (e.g. 0.1, 0.2, 0.10)
See for instance https://inspector.pypi.io/project/whey/

Duplicate of https://github.com/pypi/inspector/issues/3.


[docs] Improve CSS Grid documentation

As discussed in #9513 starting at https://github.com/mui-org/material-ui/issues/9513#issuecomment-486299939 the current documentation was a little confusing and not really located in a logical place.
I'm not sure there is a logical place for this information, but it seems like the end of the document is better than in the middle of information talking about grid stuff that MUI does support.
I made the fact that Material UI doesn't provide any CSS Grid functionality explicit.
I also took out the warning about CSS Grid only being supported in the latest browsers.  As of right now it has 90% browser coverage at https://caniuse.com/#feat=css-grid.

[x] I have followed (at least) the PR section of the contributing guide.

Thanks


Out of Date Powershell Cmdlets

• Connect-AzAccount should be Connext-AzRMAccount
• Get-AzSubscription should be Get-AzureRmSubscription
• Select-AzSubscription should be Select-AzureRmSubscription
• $ResGrp = New-AzRsourceGroup $resourceGroupName -location 'East US' should be $ResGrp = New-AzureRmResourceGroup $resourceGroupName -location "East US"
• Set-AzDataFactoryV2 should be Set-AzureRmDataFactoryV2
• Set-AzDataFactoryV2LinkedService should be Set-AzureRmDataFactoryV2LinkedService
• Set-AzDataFactoryV2Dataset should be Set-AzureRmDataFactoryV2Dataset
• Set-AzDataFactoryV2Pipeline should be SetAzureRmDataFactoryV2Pipeline
• Invoke-AzDataFactoryV2Pipeline should be Invoke-AzureRmDataFactoryV2Pipeline

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5f8f06db-58d4-75b9-f150-9a1b4767cf75
Version Independent ID: 54c26e53-460a-2f6e-b4f2-86f1e8581435
Content: Copy Activity in Azure Data Factory
Content Source: articles/data-factory/copy-activity-overview.md
Service: data-factory
GitHub Login: @linda33wj
Microsoft Alias: jingwang

@camoisan  This document has updated PowerShell cmdlets.
This document has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure PowerShell.
Hope this helps.

I didn't realize this is fantastic! Thank you for letting me know :)

@camoisan We will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.


test: fix POLLHDRUP related failures for AIX

POLLHDRUP is not implemented on AIX. Therefore UV_DISCONNECT will
never be set on events. This causes the socket to never be closed and
the tests to be stuck inside pollset_poll indefinitely, resulting in a
timeout.
This fixes the following tests:
poll_duplex
poll_unidirectional
Updated docs to let end users know that UV_DISCONNECT can be set, but
is unsupported on AIX.
Fixes: https://github.com/libuv/libuv/issues/844

LGTM

LGTM

LGTM. CI: https://ci.nodejs.org/view/libuv/job/libuv-test-commit/40/

[% 100|+ 280|-   1|T   0|S   5]: Done.
FAIL: test/run-tests
======================================================
1 of 1 test failed
Please report to https://github.com/libuv/libuv/issues
======================================================
make: The error code from the last command is 1.

https://ci.nodejs.org/job/libuv-test-commit-aix/nodes=aix61-ppc64/46/console
Down to one failure on aix!!

Cheers Imran! Landed in 337e9fd. :tada:


Add Silicon Valley Academy

Add Silicon Valley Academy SVASchool.org

@tonycrespome Pull request merged. Thank you!


Sample output in markdown for Bayes notebooks 1 and 2

Updated Bayes notebooks 1&1 with "Sample Outputs" plots in markdown

dope dope dope


TimeoutException error logged when KafkaClusterIT

When running io.kroxylicious.proxy.KafkaClusterIT the following error gets logged

2023-02-03 14:52:07 ERROR org.apache.kafka.controller.QuorumController:445 - [Controller 0] maybeBalancePartitionLeaders: unable to start processing because of TimeoutException.
Formatting /tmp/kafka9519135150772298755/broker-0 with metadata.version 3.3-IV0.

We should investigate and fix this.

can it be assigned to me?

@aaditya9899 sure, done.

Thankx and just a little help, can I know to proceed with it. I mean where's the code base and what i should be looking for?

@aaditya9899 well it's a error that's coming from Apache Kafka. It might be because of how we're running the Kafka broker for the tests. The first thing to do is understand what better what is timing out and why. You could do that by putting a breakpoint on QuorumController:445 and trying understand the context at that point (e.g. from the callstack). But this would require some decent Kafka knowledge. I'm beginning to think may this this an easy-starter issue.

Thankx for this info let me first try then

where to find the quorum controller?

@aaditya9899 The QuorumController is part of the Apache Kafka project and can be found on GitHub.
I think I've linked to the place @tombentley was discussing but the line numbers don't match exactly and I'm not sure which branch Tom was thinking of.

Just in case it could help to resolve this issue. I saw in the logs that the error happens only in the shutdown process:
2023-03-01 13:35:26 INFO  org.apache.kafka.raft.KafkaRaftClient:2287 - [RaftManager nodeId=0] Beginning graceful shutdown 2023-03-01 13:35:26 INFO  org.apache.kafka.raft.KafkaRaftClient:2413 - [RaftManager nodeId=0] Graceful shutdown completed 2023-03-01 13:35:26 INFO  org.apache.kafka.queue.KafkaEventQueue:404 - [Controller 0] QuorumController#beginShutdown: shutting down event queue. 2023-03-01 13:35:26 ERROR org.apache.kafka.controller.QuorumController:445 - [Controller 0] maybeBalancePartitionLeaders: unable to start processing because of TimeoutException. 2023-03-01 13:35:26 DEBUG org.apache.kafka.queue.KafkaEventQueue:401 - [Controller 0] KafkaEventQueue#close: Event queue is already shutting down. 2023-03-01 13:35:26 INFO  org.apache.kafka.queue.KafkaEventQueue:424 - [Controller 0] closed event queue.
but I cannot find the root cause of the issue

Closing stale - we can reopen if we see these exceptions again


Allow building using Podman

Podman reports free memory using with line starting with memFree in KB. This change accounts for that.

/lgtm


Discrepency between csjs examples and csjs-extractify

In all the examples in the main csjs repo all styles are stored in plain js files X-styles.js whereas the examples in this package are X.csjs.js.
It'd be nice to have some consistency so as to not have to rewrite modules when getting to extracting the css into a file for production.

Closing due to inactivity.


[Feature Request] Support conditional tool calling

Required prerequisites

[X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
[ ] Consider asking first in a Discussion.

Motivation
Supporting conditional tool calling means we can build a FSM with state and active_tools or a simple DAG for active_tools. This will be very helpful for workforce.
Solution
No response
Alternatives
No response
Additional context
No response

research topic, would @yiyiyi0817  and @WHALEEYE  be interested in this one?

@yiyiyi0817 @zhiyu-01 do some pre-study in this week, @WHALEEYE join the development from next week
Use agent to manage status/tools

Guohao: we can define the FSM in tool manager, manager can return the activated tools by using the FSM

@WHALEEYE thins about the combination with workforce

A paper related to this work: https://ar5iv.labs.arxiv.org/html/2402.11770

Related to PR #1141


Spurious error messages

Sometimes when the first predicate fails, error messages are included for subsequent predicates, regardless of whether or not they pass or fail.
schema = Dry::Validation.Form do
  required(:age) { int? & gt?(18) & lt?(25) }
end

schema.call('age' => 26).messages
# {:age=>["must be less than 25"]}

schema.call('age' => 17).messages
# {:age=>["must be greater than 18", "must be less than 25"]}

In the second case, the message "must be less than 25" doesn't make sense with an age of 17.

I believe in the second case, "must be less than 25" is a validation hint, i.e. the message for a validation predicate that wasn't actually run in the last call. The aim for these hints is to give the user enough information to make sure they can submit correct input without having to go through additional extra rounds of validation failures. For example, if the user only saw "must be greater than 18", then they might go ahead and enter "35" instead, which'd annoyingly trigger yet another error. The hint message prevents this.
I do believe that @solnic has some plans to make the error messages more flexible, perhaps allowing you to disable hints altogether, so I'll leave this open for his comment, but I thought I could at least clarify things for you in the meantime :)

Ok, thanks for the explanation. When you look at it from the point of view of hints, it does make sense.
However, as error messages, I think they can be very confusing for the user, especially if there are a lot of them, and the user doesn't know which one indicates the actual problem.
For instance, consider a message hash like:
{
  :order_id=>[
     "must be an active order",
     "must be owned by you",
     "must be in transit",
     "must be shipping via FedEx",
     "must be less than 10kg",
     "must not be for red pyjamas"
      ...
  ]
}

Suppose that it turns out that the only problem in this case is that the order is not active.  But suppose also that the user doesn't happen to know that the order is not active.
The user is going to get very frustrated going through their order trying to figure out what is wrong with it, because everything looks correct to them.  ("Maybe it's not actually in transit yet?  But the web site says it's in transit!  Hmm...it's only a couple of shirts - surely they can't weigh more than 10kg...")
In effect, including all of the error messages when the first predicate fails can (in certain cases) be the equivalent of saying: "something is wrong - try again".

Yes, I plan to make it configurable and smarter too. Not sure when, but probably in 0.10.0. You can already opt-out from hints by using Result#message_set which has messages and hints stored separately. I'll be expanding this API and functionality a lot in the future. One idea I have is to be able to assign priorities to individual predicates so that the system can skip certain messages when other errors with higher priority are present. And of course a simple way of excluding hints from error messages will be provided too.
For now, you could check this out:
result = schema.(input)

result.message_set
result.message_set.messages # I should probably rename this to `errors` though
result.message_set.hints

Oh I forgot to mention one important thing - the example you gave is not an issue, we don't generate hints for high-lvl rules exactly because of the reasons you gave, and this type of checks would be definitely high-lvl rules.

In the second case, the message "must be less than 25" doesn't make sense with an age of 17.

Actually, why do you think that? w/o this hint the use doesn't have the full info about the requirements.

Thanks for the information!  It's good to know I can treat errors and hints separately.

In the second case, the message "must be less than 25" doesn't make sense with an age of 17.

Actually, why do you think that? w/o this hint the user doesn't have the full info about the requirements.

If it is understood by the user to be a hint, then yes it does make sense.  If it is understood by the user to be an error message, then it doesn't make sense.

If it is understood by the user to be a hint, then yes it does make sense. If it is understood by the user to be an error message, then it doesn't make sense.

Ah yes, and I agree, although for the time being and the limited use cases people have this worked fine, but as I said - this is not the final version of this feature, right from the start I wanted to have a way to treat the separately. Ie in the 0.9.x I added support for adding different messages for hints like this:
en:
  errors:
    gt?:
      hint: make sure it's greater than %{num}
      failure: must be greater than %{num}

So that's an improvement already :) In 0.10.0 I'll expand MessageSet functionality too, so it's easier to work with it. We probably also want more configurable #messages API (it already supports options ie locale: :pl so we can add support for more options like hints: false etc.

All of that sounds most excellent - thank you!


Failed to set TABLE row, expected array

We've been wrestling with this one a bit - am getting message "Failed to set TABLE row, expected array"  when calling BAPI_SALESORDER_CREATEFROMDAT2. It would be great if in a downstream version -- if possible -- the error message could include which table parameter it is complaining about. There are a bunch of tables for this BAPI and we're having a hard time figuring out which is the offending item  : )
We're calling BAPI_SALESORDER_CREATEFROMDAT2 with the following data:
            [ORDER_HEADER_IN] => Array
                (
                    [DOC_TYPE] => Y0OR
                    [SALES_ORG] => 3310
                    [DISTR_CHAN] => 00
                    [DIVISION] => 00
                    [PURCH_NO_C] =>
                    [COLLECT_NO] =>
                )

            [ORDER_PARTNERS] => Array
                (
                    [0] => Array
                        (
                            [PARTN_ROLE] => AG
                            [PARTN_NUMB] =><PHONE_NUMBER>
                        )

                )

            [ORDER_ITEMS_IN] => Array
                (
                    [0] => Array
                        (
                            [ITM_NUMBER] => 00010
                            [MATERIAL] => BC535R
                        )

                    [1] => Array
                        (
                            [ITM_NUMBER] => 00020
                            [MATERIAL] => B0020017
                        )

                    [2] => Array
                        (
                            [ITM_NUMBER] => 00030
                            [MATERIAL] => B0020017
                        )

                )

            [ORDER_CONDITIONS_IN] => Array
                (
                    [0] => Array
                        (
                            [ITM_NUMBER] => 00020
                            [COND_TYPE] => Y0M1
                            [COND_VALUE] => 11.111
                        )

                )

            [ORDER_TEXT] => Array
                (
                    [0] => Array
                        (
                            [TEXT_ID] => Y004
                            [LANGU] => EN
                            [FORMAT_COL] => *
                            [TEXT_LINE] =>
                        )

                )

            [ORDER_SCHEDULES_IN] => Array
                (
                    [0] => Array
                        (
                            [ITM_NUMBER] => 00010
                            [REQ_QTY] => 1
                        )

                    [1] => Array
                        (
                            [ITM_NUMBER] => 00020
                            [REQ_QTY] => 2
                        )

                    [2] => Array
                        (
                            [ITM_NUMBER] => 00030
                            [REQ_QTY] => 1
                        )
                )
        )

The interface is listed here fyi:
http://www.se80.co.uk/sapfms/b/bapi/bapi_salesorder_createfromdat2.htm
ORDER_HEADER_IN is an is an EXPORTING structure. The rest are TABLES parameters. Thus I can't seem to figure out whats wrong : (
Ran a full trace -- doesn't seem to suggest the location of the issue?

The error messages are indeed not ideal, I'll have a look at improving them. Most of them already state the parameter name when an error occurs, but some of the TABLE methods do not currently.
The RFC_ILLEGAL_STATE return code in RfcDestroyFunctionDesc should not be an issue. There's definitely a problem worth investigating there, but I think your problem lies somewhere else.
I'll have a look at the BAPI. I'm using it in one of my projects, so I should find a clue...

Can you upload the full trace somewhere (make sure to whipe credentials and the like)?

You bet - thanks much for the help. Here you go.
rfc09564_09388.salesorder.usb.failed.txt

Actually, I can't reproduce your error. Using this code:
<?php

$config = include "tests/sapnwrfc.config.inc";

$c = new \SAPNWRFC\Connection($config);

$f = $c->getFunction('BAPI_SALESORDER_CREATEFROMDAT2');
$r = $f->invoke([
    'ORDER_HEADER_IN' => [
                'DOC_TYPE' => 'Y0OR',
                'SALES_ORG' => '3310',
                'DISTR_CHAN' => '00',
                'DIVISION' => '00',
                'PURCH_NO_C' => '',
                'COLLECT_NO' => '',
    ],
    'ORDER_PARTNERS' => [
        [
            'PARTN_ROLE' => 'AG',
            'PARTN_NUMB' => '0020040938',
        ]

    ],
    'ORDER_ITEMS_IN' => [
        [
            'ITM_NUMBER' => '00010',
            'MATERIAL' => 'BC535R',
        ],
        [
            'ITM_NUMBER' => '00020',
            'MATERIAL' => 'B0020017',
        ],
        [
            'ITM_NUMBER' => '00030',
            'MATERIAL' => 'B0020017',
        ]
    ],
    'ORDER_CONDITIONS_IN' => [
        [
            'ITM_NUMBER' => '00020',
            'COND_TYPE' => 'Y0M1',
            'COND_VALUE' => 11.111,
        ]
    ],
    'ORDER_TEXT' => [
        [
            'TEXT_ID' => '004',
            'LANGU' => 'EN',
            'FORMAT_COL' =>'*',
            'TEXT_LINE' => '',
       ]
    ],
    'ORDER_SCHEDULES_IN' => [
        [
            'ITM_NUMBER' => '00010',
            'REQ_QTY' => 1.0,
        ],
        [
            'ITM_NUMBER' => '00020',
            'REQ_QTY' => 2.0
        ],
        [
            'ITM_NUMBER' => '00030',
            'REQ_QTY' => 1.0
        ]
    ]
]);

var_dump($r);

I only get errors about the sales document not existing, etc. (which was expected), but the source line that produces the error you mentioned executes cleanly.
I'll poke around a little more though. Can you show me a code snippet that reliably reproduces the error for you?

Thanks I'll try your code snippet in the morning and get back to you. Best,
On Wed, Sep 14, 2016 at 2:03 AM -0400, "Gregor Kralik" <notifications<EMAIL_ADDRESS>wrote:
Actually, I can't reproduce your error. Using this code:
getFunction('BAPI_SALESORDER_CREATEFROMDAT2');
$r = $f->invoke([
    'ORDER_HEADER_IN' => [
                'DOC_TYPE' => 'Y0OR',
                'SALES_ORG' => '3310',
                'DISTR_CHAN' => '00',
                'DIVISION' => '00',
                'PURCH_NO_C' => '',
                'COLLECT_NO' => '',
    ],
    'ORDER_PARTNERS' => [
        [
            'PARTN_ROLE' => 'AG',
            'PARTN_NUMB' => '0020040938',
        ]

    ],
    'ORDER_ITEMS_IN' => [
        [
            'ITM_NUMBER' => '00010',
            'MATERIAL' => 'BC535R',
        ],
        [
            'ITM_NUMBER' => '00020',
            'MATERIAL' => 'B0020017',
        ],
        [
            'ITM_NUMBER' => '00030',
            'MATERIAL' => 'B0020017',
        ]
    ],
    'ORDER_CONDITIONS_IN' => [
        [
            'ITM_NUMBER' => '00020',
            'COND_TYPE' => 'Y0M1',
            'COND_VALUE' => 11.111,
        ]
    ],
    'ORDER_TEXT' => [
        [
            'TEXT_ID' => '004',
            'LANGU' => 'EN',
            'FORMAT_COL' =>'*',
            'TEXT_LINE' => '',
       ]
    ],
    'ORDER_SCHEDULES_IN' => [
        [
            'ITM_NUMBER' => '00010',
            'REQ_QTY' => 1.0,
        ],
        [
            'ITM_NUMBER' => '00020',
            'REQ_QTY' => 2.0
        ],
        [
            'ITM_NUMBER' => '00030',
            'REQ_QTY' => 1.0
        ]
    ]
]);

var_dump($r);

I only get errors about the sales document not existing, etc. (which was expected), but the source line that produces the error you mentioned executes cleanly.

I'll poke around a little more though. Can you show me a code snippet that reliably reproduces the error for you?

-
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I updated the exception messages to be more clear.
Consider this code, passing a wrong parameter type to ORDER_PARTNERS:
// [...]
$r = $f->invoke([
   // [...]
    'ORDER_PARTNERS' => [
            // ORDER_PARTNERS is a table, but we are not passing rows here
            'PARTN_ROLE' => 'AG',
            'PARTN_NUMB' => '0020040938',
    ],
  // [...]
]);

// [...]

Instead of only
Failed to set TABLE row, expected array
the message is now
Failed to set TABLE row, expected array for parameter ORDER_PARTNERS
I'll release a new version in the next days, so @ThaDafinser can (hopefully?) provide new builds for Windows.

Happy to hear it works.
I'll definitely have a look at why it doesn't work with references though. I rarely use them, so there are no test using references that could have caught this issue. I'll open another ticket to track progress...

So the problem with this is that you have to define an array, but watch out how you define it
This will not work because the array does not have an index.
		$customersData = [ 			'LOW'       =>  $sap->populateKunnrWithZeros($customer->kunnr), 			'HIGH'      =>  $sap->populateKunnrWithZeros($customer->kunnr) 		];
This will work because the array has an index.
		$customersData[0] = [ 			'LOW'       =>  $sap->populateKunnrWithZeros($customer->kunnr), 			'HIGH'      =>  $sap->populateKunnrWithZeros($customer->kunnr) 		];


[curl] Add feature idn2 and disable dependency lber2

add curl feature idn2 to avoid to find the system idn2 library and headers.
Since lber2 is not included in vcpkg and is not installed on some systems, it may cause installation issues, disable curl feature lber2.

Fixes #14798.
Note: already test all features on x86-windows and x64-windows-static successfully.

Hi JackBoosY!
I see that this pull request is still a work in progress. Do you have an ETA for when it's going to be ready? Right now the port curl and the 39 ports that depend on it are broken on Linux (I don't understand why the CI didn't catch this :slightly_frowning_face: ). If you tell me that your ETA is more a few days, I could do a 2-liner pull request that allows the port to work again on Linux while waiting for the cleaner solution you are developing, i.e. doing
vcpkg_fixup_pkgconfig(SYSTEM_LIBRARIES pthread dl c) -> vcpkg_fixup_pkgconfig(SYSTEM_LIBRARIES pthread dl c lber ldap).
 Port-Version: 3->Port-Version: 4`
is an ugly (or more precisely, an incorrect) but effective way to make curl work again. :slightly_smiling_face:

Hi @klalumiere, it should be fixed on my latest commit, can you test it?
The reason for this issue is that some curl dependencies are not added as other dependencies (add option to CMakeLists.txt and use find_package/find_library to find them), and I've disabled them.

Also, @martti-cloudkey can you please test this PR?
Thanks.

The problem with lber has been replaced by a problem with ldap :slightly_frowning_face:
-- Installing: /home/klalumiere/Downloads/vcpkg/packages/curl_x64-linux/share/curl/curl-config
-- Fixing pkgconfig file: /home/klalumiere/Downloads/vcpkg/packages/curl_x64-linux/lib/pkgconfig/libcurl.pc
-- CHECK_LIB_ldap_RELEASE:CHECK_LIB_ldap_RELEASE-NOTFOUND
CMake Error at scripts/cmake/vcpkg_fixup_pkgconfig.cmake:233 (message):
  Library "ldap" was not found! If it is a system library use the
  SYSTEM_LIBRARIES parameter for the vcpkg_fixup_pkgconfig call! Otherwise,
  correct the *.pc file
Call Stack (most recent call first):
  scripts/cmake/vcpkg_fixup_pkgconfig.cmake:299 (vcpkg_fixup_pkgconfig_check_files)
  ports/curl/portfile.cmake:187 (vcpkg_fixup_pkgconfig)
  scripts/ports.cmake:136 (include)

Error: Building package curl:x64-linux failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `./vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: curl:x64-linux
  Vcpkg version: 2020.11.12-unknownhash

Additionally, attach any relevant sections from the log files above.
[21:04:48] klalumiere@WKS-000791:~/Downloads/vcpkg$ git status
On branch dev/jack/14798
Your branch is up to date with 'origin/dev/jack/14798'.

nothing to commit, working tree clean

@KaibaLopez Can you just modify VCPKG_ROOT/port/curl/portfile.cmake line 71:
...
    OPTIONS ${FEATURE_OPTIONS}
        -DCURL_DISABLE_LDAP=OFF
        ${UWP_OPTIONS}
...

And try again?
Thanks.

@klalumiere  Any feedback with my comment?

@KaibaLopez Can you just modify VCPKG_ROOT/port/curl/portfile.cmake line 71:
...
    OPTIONS ${FEATURE_OPTIONS}
        -DCURL_DISABLE_LDAP=OFF
        ${UWP_OPTIONS}
...

And try again (do not use my changes)?
Thanks.

I had exactly the same problem with -DCURL_DISABLE_LDAP=OFF (that is, ./vcpkg install curl fails). However, adding -DCURL_DISABLE_LDAP=ON did solve the problem (I think there was a confusion with the double negative :slightly_smiling_face:).

@klalumiere Yep, my mistake. It should be ON. I will continue fix this issue today.

Need restore my changes about lber2.

@klalumiere Can you remove DCURL_DISABLE_LDAP=ON, install libldap2-dev and libsasl2-dev then try again?
Thanks.

So as you saw on #14865, I'm not able to reproduce the problem I had anymore on master. However, I still have problem with the branch JackBoosY:dev/jack/14798. I built in a Docker container running the image rikorose/gcc-cmake. I installed vim with
apt-get update && apt-get install vim --no-install-recommends

and that's all. Without any changes to the branch, I ran ./vcpkg install curl:x64-linux-dynamic with the triplet x64-linux-dynamic from the doc, i.e.
set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic) # This changed from static to dynamic

set(VCPKG_CMAKE_SYSTEM_NAME Linux)

It fails, with the output
CMake Error at scripts/cmake/vcpkg_fixup_pkgconfig.cmake:235 (message):
  Unhandled string
  "/root/vcpkg/packages/curl_x64-linux-dynamic/lib/pkgconfig/../../lib/libssl.so
  /root/vcpkg/packages/curl_x64-linux-dynamic/lib/pkgconfig/../../lib/libcrypto.so
  /root/vcpkg/packages/curl_x64-linux-dynamic/lib/pkgconfig/../../lib/libz.so"
  was found! If it is a system library use the SYSTEM_LIBRARIES parameter for
  the vcpkg_fixup_pkgconfig call! Otherwise, correct the *.pc file or add the
  case to vcpkg_fixup_pkgconfig
Call Stack (most recent call first):
  scripts/cmake/vcpkg_fixup_pkgconfig.cmake:299 (vcpkg_fixup_pkgconfig_check_files)
  ports/curl/portfile.cmake:188 (vcpkg_fixup_pkgconfig)
  scripts/ports.cmake:136 (include)

Error: Building package curl:x64-linux-dynamic failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `./vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: curl:x64-linux-dynamic
  Vcpkg version: 2020.11.12-unknownhash

Additionally, attach any relevant sections from the log files above.

I'll try removing -DCURL_DISABLE_LDAP=ON next, although it doesn't seems to be related to the error. I'll try installing the libs you suggested after. It might take a while. Building is super slow in a container :slightly_frowning_face:

Actually... Shouldn't the string /root/vcpkg/packages/curl_x64-linux-dynamic/lib/pkgconfig/../../lib/libssl.so be replaced by -lssl in the pkgconfig? This seems to be what is done elsewhere in the portfile.cmake. Oh, I see, e.g. portfile.cmake:139
vcpkg_replace_string(${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-dbg/libcurl.pc "${CURRENT_INSTALLED_DIR}/debug/lib/libssl.a" "-lssl")

There should also be
vcpkg_replace_string(${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-dbg/libcurl.pc "${CURRENT_INSTALLED_DIR}/debug/lib/libssl.so" "-lssl")

Couldn't we use a regular expression instead? Something in the spirit of
vcpkg_replace_string(${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-dbg/libcurl.pc "${CURRENT_INSTALLED_DIR}/debug/lib/lib(.*)\\.so" "-l\1")

We could add a function
function(vcpkg_replace_string_regex filename match_string replace_string)
    file(READ ${filename} _contents)
    string(REGEX REPLACE "${match_string}" "${replace_string}" _contents "${_contents}")
    file(WRITE ${filename} "${_contents}")
endfunction()

and then use it as I was suggesting, but without my mistake :laughing:
vcpkg_replace_string_regex(${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-dbg/libcurl.pc
    "${CURRENT_INSTALLED_DIR}/debug/lib/lib(.*)\.so" "-l\\1")

I had practically the same error than here without -DCURL_DISABLE_LDAP=ON, the only difference being of course the line number, i.e. `ports/curl/portfile.cmake:187 (vcpkg_fixup_pkgconfig)

cc @ras0219 for review this PR. I'm not sure my changes is correct.

Merging master did fix the build on Linux with the triplet x64-linux-dynamic. :partying_face:
The systematic warning when we build on a system different than Windows is a bit unfortunate. :slightly_frowning_face: . Could this warning instead be added to cURL's CMakeLists.txt? I believe that once inside the CMakeList.txt, we might have the information necessary to know if libldap2-dev and libsasl2-dev will be required for the build. We could print the warning only if we detect that they are necessary, and that they can't be found. :thinking:

@klalumiere Adding the message to CMakeLists.txt is not a good choice, because I'm not sure if these two packages are needed, and curl builds successfully in my ubuntu without that.

@klalumiere Adding the message to CMakeLists.txt is not a good choice, because I'm not sure if these two packages are needed, and curl builds successfully in my ubuntu without that.

I'm not sure I understand. Right now the warning is inside the portfile.cmake. How is it better to put the warning inside the portfile.cmake than to put it in the CMakeLists.txt? As I mentioned above, the advantages of putting it inside the CMakeLists.txt is that you can search (find_library or find_package) for e.g. ldap2 and, if it is not found, print the warning. What are the advantages of putting the warning inside te portfile.cmake?
It also works on my machine (Ubuntu 20.04). dpkg --list libldap2-dev tells me that this package is installed, but dpkg --list libsasl2-dev informs me that there is no such package on my machine. It probably would be worth it to verify what happen inside a Docker container where none of the packages are installed. If none are required, we can juste remove the warning altogether.

@klalumiere Normally, the warning message like that always put into portfile.cmake, such as pixel, glfw3, spdk etc.
We need to detect and prepare these dependencies before configuration, rather than dynamically checking during configuration. Otherwise vcpkg will report error logs instead of error message.

I retried to install curl inside the Docker image rikorose/gcc-cmake with your latest changes. I tried with the triplets x64-linux and x64-linux-dynamic. Both work, and I made sure that neither libldap2-dev nor libsasl2-dev are installed. Hence, I don't think that the warning is required: it works without libldap2-dev and libsasl2-dev.
Good point about the error logs instead of error message. I think the way spdk does it is smart, i.e. it uses FIND_PATH to print (an error in this case) only if there's actually a problem.

@JackBoosY Why is this "requires:discussion"? It looks like a normal bugfix to me?

@BillyONeal Discussion for feature lber2 in curl's source. I'm not sure which method we should choose to solve that.


index documents

currently the deno_mongo lib does not support the create-index command, this will need to be implemented in order to support indexes

implemented in 0.0.21 of the mongo client and was added to adapter in #27


SVG class used twice displays as black instead of fill color

I have an svg with 3 different classes that have 3 different fill color properties. I am using two of the classes for two different paths/polygons and those paths/polygons display as fill black rather than the styled color. When I remove the second usage of the class the first displays fine, also if I change the second usage to the same fill color but a different class name it works fine.
Here is my SVG
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 21.1.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
     viewBox="0 0 3600 3600" style="enable-background:new 0 0 3600 3600;" xml:space="preserve">
<style type="text/css">
    .st0{fill:#CF4545;}
    .st1{fill:#FFFFFF;}
    .st2{fill:#922727;}
</style>
<path class="st0" d="M1845.4,46.2L3585,1798.6c0,0-1780.2,1785.2-1794.6,1742.2c-14.3-43-1807.6-1746.6-1764.7-1760.9
    C68.7,1765.6,1845.4,46.2,1845.4,46.2z"/>
<path class="st1" d="M1812.2,1065l734.1,734.1c0,0-734.1,745.9-740,728.1c-5.9-17.8-763.6-740-745.9-745.9
    C1078.1,1775.4,1812.2,1065,1812.2,1065z"/>
<path class="st0" d="M1839.1,1394l396.7,396.7c0,0-396.7,403.1-399.9,393.5c-3.2-9.6-412.7-399.9-403.1-403.1
    C1442.4,1777.9,1839.1,1394,1839.1,1394z"/>
<path class="st2" d="M2235.7,1790.7l-399.9,393.5l-403.1-403.1C1432.8,1781.1,2254.8,1790.7,2235.7,1790.7z"/>
<polygon class="st2" points="3585,1798.6 2538.2,1798.6 1812.9,2527.6 1057.6,1787.4 25.8,1779.9 1807.2,3561.4 "/>
</svg>

The two paths with class="st0" display as black rather than #CF4545
The path and polygon with class="st2" display as black rather than #922727
The path with `class="st1" displays white correctly

Just ran into something that sounds similar to this issue. There's an upstream dependency on svgo which defaults to only updating the first element with a matching class when inlining styles. Not sure that's a sensible default here but you could try changing it by configuring your vue.config like this:
svgRule
  .use('babel-loader')
  .loader('babel-loader')
  .end()
  .use('vue-svg-loader')
    .options({
      svgo: {
        plugins: [{ inlineStyles: { onlyMatchedOnce: false } }]
      }
    })
  .loader('vue-svg-loader');

Thanks @pat-flew! That did indeed fix the issue I was seeing.

didnt work for me though...

didnt work for me though...


Update golangci-lint and config

What type of PR is this?
/kind ci
What this PR does / why we need it:
Update the linter configuration as well as the tool to its latest version.
Which issue(s) this PR fixes:

None
Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

None

/lgtm


Feature request for Cypress test runner plugin

Is your feature request related to a problem? Please describe.
It would be beautiful to have a plugin for Cypress runner, as it supports component testing in 4 frameworks.
Describe the solution you'd like
Just as we can use Jest + RTL for component testing, we can use Cypress for component testing. It would be desirable to do mutation analysis with it as well.
Additional context
Add any other context or screenshots about the feature request here.
Side by side 40+ RTL and Cy CT tests: https://github.com/muratkeremozcan/tour-of-heroes-react-cypress-ts
yarn test vs yarn cy:run-ct-fast

👍

👍🏻

Anyone working on something that will make the cypress tests run a little more efficiently?
We have ballpark 450 Cypress component tests running and each run of all of these takes approximately 15 minutes. Stryker seems to be running all tests for all mutations for all files when set up with command runner:
yarn cypress run --component --headless -C cypress.component.config.ts
and this causes the stryker run to timeout (> 6h) in a github workflow - even using incremental mode. Also, it seems all mutations that was not killed, are timing out - even if they should have survived:

In the above case, the mutation is always returning false. This should have been a surviving mutation - not a timed out one.
Will a dedicated runner target tests to mutated files?

👍

👍