Add MMR functionality to elasticsearch retriever

Allows MMR functionality only for the case where we have access to the embedding function. Also allows for users to request for fields from elasticsearch store. These are added to the document metadata.

cc @joemcelroy

Thanks for contribution! Would be good to add a mention that this is possible in documentation too. Happy to add it if you dont manage to.

@joemcelroy I'm unable to see the real reason behind the vercel deployment failure. I get a 404 when i click on the link. I suspect it is the integration test that i added. I followed the instructions at (Recording HTTP interactions with pytest-vcr) in https://github.com/langchain-ai/langchain/blob/master/libs/langchain/tests/README.md . Should i actually record the interactions and push it as part of the pull request?

fixed vercel 👍


⚠ T:Riza Community Forum has degraded performance

In 8b50ea7, T:Riza Community Forum (https://community.triza.dev) experienced degraded performance:

HTTP code: 200
Response time: 1631 ms

Resolved: T:Riza Community Forum performance has improved in ecc34ac.


kitchen-docker not working with InSpec

Hi Team,
'service' resource type of InSpec isn't working properly with kitchen-docker.
It's working fine with kitchen-vagrant.
I am trying to check whether apache2 is running or not.
Hence, my InSpec Test is as follows:
describe service('apache2') do
it { should be_enabled }
it { should be_running }
end
And the above test is failing.
However, when running with kitchen vagrant, the test is passing.
Can you help me understand the difference in implementation of kitchen-vagrant and kitchen-docker and is the issue with kitchen-docker or InSpec?

Sorry  for that's a bug in inspec.
https://github.com/chef/inspec/issues/931


Add IO::Stapled

Adds IO::Stapled class which staples together two unidirectional IOs to form a single bidirectional IO. It basically delegates all read methods to reader and all write methods to writer.
This class is an extension to unidirectional IOs (like IO::Memory, IO.pipe) when you need true bidirectional IOs, for example as a replacement for socket streams.
This is particularly useful for speccing code that would normally work with a socket. You don't need to create an actual socket but can just staple together two IO::Memory.

Added gets delegator and removed win32 check :+1:

Is there any use case other than "mocking sockets in specs"?

@ysbaddaden You could for example wrap the standard streams with HTTP or any other protocol that expects read and write on the same IO ;)

@ysbaddaden I think the idea is to use it for specs, to stub a socket. But if we are going to do that, maybe IO::Stapled should just have two IO::Memory internally, and maybe be called IO::Mock or something like that.

Yes, the main purpose of this is obviously mocking socket connections, but it can also be generally useful for multiplexing two unidirectional read/write streams in one IO.
Limiting to IO::Memory won't do because you can't really communicate through it. This would limit the usefulness as replacement for socket connections. For many use cases you'll need a pipe (hence there is also a class method to set that up).

Hm, maybe wrapping two pipes is a good use case...


Inline modal

I want to be able to specify the target for a modal as a DOM reference on the same page. This (sort of) works, but always loads the first element due to how modal content is substituted into the global modal turbo frame.

Not relevant to kpop 3


[release-branch.go1.21] Replace RtlGetNtVersionNumbers with RtlGetVersion

This is a cherry-pick of CL 571015, as upstream won't backport it to Go 1.21.
The cherry-pick is not completely clean, it needed some manual conflict resolution.

Actually I'm seeing this in the build logs:
D:\a\_work\1\s\go\src\syscall\exec_windows.go:322:2: undefined: rtlGetNtVersionNumbers

I just noticed going through my email that this was merged into dev/auto-sync/microsoft/release-branch.go1.21 rather than the actual release branch, probably not intended. 😄

I just noticed going through my email that this was merged into dev/auto-sync/microsoft/release-branch.go1.21 rather than the actual release branch, probably not intended. 😄
(Also #1179)

Wops, good catch. Fixed.


Readgroups Endpoint

The access patterns for the reads API would be improved by adding a read groups endpoint (#650). Currently the readgroupset endpoint returns a repeated list of readgroups matching the search criteria. This is confusing because the readgroupset record is modified at query time.
This PR adds the /readgroups/search endpoint, which exposes the available read groups using familiar strict matching on name, bio_sample_id, dataset_id, and read_group_set_id. Read group set messages no longer contain the list of readgroups since they can be retrieved using readgroups/<read_group_id>. This simplifies the read group set endpoint by moving the bio_sample_id filter to the appropriate entity, ReadGroup, as a single read group set might contain reads for many samples.

I would be very careful when using required when defining a protobuf message - please take at the following note as required is forever:

:exclamation: Required Is Forever You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field – old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead. Some engineers at Google have come to the conclusion that using required does more harm than good; they prefer to use only optional and repeated. However, this view is not universal.

btw, I am +1 on adding the end point, it's only the search criteria that needs to be worked through.

Given the current data model, bio_sample_id, name, and dataset_id are all named fields in the ReadGroup message. This means they can be searched using these fields without restricting to a read group set, I believe the original schema designers were clever regarding this and left a path for accessing read groups at this granularity.
We could move ahead with presenting a more sensible offering based on the existing message fields as the current metadata is hampered without this change. If other search request fields become desired, we can add them to the request as needed.
The use case is specifically filtering read groups by their bioSampleId, although it seems sensible to provide name filtering as well. @jeromekelleher: for performing a search reads request the read group IDS have been included in the ReadGroupSet message itself. That common access pattern is unchanged.

The point is to allow restricting to a readgroupset, not
require it.  readgroupset are the most common (based on the
largest cancer genomics research project to date) data grouping
for analysis.
David Steinberg<EMAIL_ADDRESS>writes:

Given the current data model, bio_sample_id, name, and dataset_id are all named
fields in the ReadGroup message. This means they can be searched using these
fields without restricting to a read group set, I believe the original schema
designers were clever regarding this and left a path for accessing read groups
at this granularity.
We could move ahead with presenting a more sensible offering based on the
existing message fields as the current metadata is hampered without this
change. If other search request fields become desired, we can add them to the
request as needed.
The use case is specifically filtering read groups by their bioSampleId,
although it seems sensible to provide name filtering as well. @jeromekelleher:
for performing a search reads request the read group IDS have been included in
the ReadGroupSet message itself. That common access pattern is unchanged.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.*

@diekhans So do you think making the dataset_id required and the read_group_set_id optional makes more sense?

Hi David,
Think of the bigger picture, what are we really interested in?  We want to analyze clean quality-controlled groupings of data (reads, variants, etc.) for clinical purposes (i.e. disease identification, personalized medicine, etc).  We want to search by metadata and not drill too closely on the bottom level unless really necessary, and usually that is performed by reaching back to the sequencing center that submitted them, in case there are significant issues with the cloud-uploaded data.  We have worked too long with with (g)VCFs and BAMs, which slow us down and become sub-optimal for the type of large-scale analysis that we want to target.  We want to query a system and through macros like this (yes I know it looks rough, but omitted the other details for clarity to drive home the point):
prog GermlineAndCancer {
  qc.reads <- using.all.reads(qc="validated", filter="*carcinogenic*")
  variants.germline <- germline.pipeline.with.qc(qc.reads)
  variants.cancer   <- cancer.pipeline.with.qc(qc.reads)
  report.%date% <- saveAsHTML(getOverlapsAndRare(variants.germline, variants.cancer<EMAIL_ADDRESS>}

run.daily( "GermlineAndCancer" )  // Daily-streaming data will eventually trigger new key, rare variants.

So the key question is what type of information and data-structures would most easily and correctly, get us to perform such queries in the most intuitive way.  That is really the key goal of GA4GH that matters the most.
Hope it helps,
Paul

Thanks Paul, that's very interesting. The decision on whether to include the read_group_set_id in the request seems like the last contention. Based on @diekhans observation, there is a good case for including it as an optional field of the request, which is reflected in the most recent commit. Can I get your +1?

+1!

Hi David,
The issue is not that it is optional, but that a Readgroup can belong to multiple ReadGroupSets.  If it belongs to multiple ones, does that mean we have to search all ReadGroupSets that this ReadGroup belongs to?
A ReadGroup can be a sample, but I want to combine with other samples that are high quality to perform a specific analysis.   Then I want to find other ReadGroupSets where this QC'ed sample was analyzed in, so that I do not duplicate results which I can use for comparison.  How natural would such a query be to perform?
Think about this scenario.  A baby is born today with a predisposition to a specific cancer subtype.  I want sequence that baby every year for the rest of his/her life including the parents at the time of birth.  More refined samples of the cancer's molecular evolution will be biopsied on a more frequent basis that reflect the aggressiveness of the stage of the cancer.  These might be many ReadGroupSets made up of multiple ReadGroups which I might - or might not - want to average out for my comparison.
We are doing deep sequencing these days because we are limited in the capacity to quickly determine the active site in the form of  a 3D confirmation of protein sequences, and their interactions.  Thus we are left with what we can inspect most quickly, which are gene variations that become most probabilistically significant.  Next-Generation Sequencing has always been an indirect analysis for understanding the some types of biological processes.
So in order to correlate with diseases and possible drug targets - and their downstream interactions - we want to perform queryable analysis on a large scale.
Do you see my dilemma?  We can add a ReadGroupSet id to a ReadGroup as optional, but the data-structure is back to the form of a BAM which we are trying to get away from.  We have been having the same mindset towards NGS for the past 7 years - and reverting to it periodically the past two years here in GA4GH - but the goal is to make the wire-protocol fit large-scale, global multi-repository analysis.
If we add it now and large datasets get loaded this way at different repositories, and several years down the line we change the design to be more inter-connected to suite new analysis techniques, does that mean we have to reload the data, or would we be able to perform schema evolution with minimal updates to the data to fix it?
We have to start with the analysis goals, and tailor the data-structures to fit it.
Hope you can see where I am coming from.
Thank you,
Paul

Complex queries for readgroups should be performed using the sequencing
metadata.
Paul Grosu<EMAIL_ADDRESS>writes:

Hi David,
The issue is not that it is optional, but that a Readgroup can belong to
multiple ReadGroupSets. If it belongs to multiple ones, does that mean we have
to search all ReadGroupSets that this ReadGroup belongs to?
A ReadGroup can be a sample, but I want to combine with other samples that are
high quality to perform a specific analysis. Then I want to find other
ReadGroupSets where this QC'ed sample was analyzed in, so that I do not
duplicate results which I can use for comparison. How natural would such a
query be to perform?
Think about this scenario. A baby is born today with a predisposition to a
specific cancer subtype. I want sequence that baby every year for the rest of
his/her life including the parents at the time of birth. More refined samples
of the cancer's molecular evolution will be biopsied on a more frequent basis
that reflect the aggressiveness of the stage of the cancer. These might be many
ReadGroupSets made up of multiple ReadGroups which I might - or might not -
want to average out for my comparison.
We are doing deep sequencing these days because we are limited in the capacity
to quickly determine the active site in the form of a 3D confirmation of
protein sequences, and their interactions. Thus we are left with what we can
inspect most quickly, which are gene variations that become most
probabilistically significant. Next-Generation Sequencing has always been an
indirect analysis for understanding the some types of biological processes.
So in order to correlate with diseases and possible drug targets - and their
downstream interactions - we want to perform queryable analysis on a large
scale.
Do you see my dilemma? We can add a ReadGroupSet id to a ReadGroup as optional,
but the data-structure is back to the form of a BAM which we are trying to get
away from. We have been having the same mindset towards NGS for the past 7
years - and reverting to it periodically the past two years here in GA4GH - but
the goal is to make the wire-protocol fit large-scale, global multi-repository
analysis.
If we add it now and large datasets get loaded this way at different
repositories, and several years down the line we change the design to be more
inter-connected to suite new analysis techniques, does that mean we have to
reload the data, or would we be able to perform schema evolution with minimal
updates to the data to fix it?
We have to start with the analysis goals, and tailor the data-structures to fit
it.
Hope you can see where I am coming from.
Thank you,
Paul
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.*

@david4096 will create a demo program to show this approach before merging.

I greatly welcome the idea of a demo, as it is my preferred approach.  It will give us the opportunity to hash out and test ideas together for integrating the best ideas from everyone.

Hi David,
I'm assuming the demos will be presented via Github.  While you are building the demo try to see if generating temporary federated tables might help you in joining multiple data sources together.  These would work as short-lived dynamic databases, where the transitory tables optimize the memoization similar to a dynamic programming approach.  With time we can have a cache scheduler where the tables become more permanent if they become highly accessed, and would have the possibility of talking to each other.  It is basically a way of balancing the wire-protocol load versus these temporary data-hubs' throughput.
These federated tables should be auto-generated by the queries performed, and thus can live ephemerally but would aid in speeding up the dynamic query accessing multiple sites and/or data-sources.
Hope it helps,
Paul

@pgrosu That's very interesting, I agree! I currently have enough resources to approach this in such a way so as to verify the basic access patterns, however, I am interested in whether you have some scaffolding to work with in approaching the federation and optimization problems.
I'll let you know if I run into any issues putting together a most limited demonstration!

@david4096 I have ways to test things on a multi-node cluster where the distributed repositories would be simulated by throttling the bandwidth among running processes.  There are tools like Pipe Viewer, or RequestsThrottler where one can tweak the bandwidth throughput among parallel running processes representing subsets of repositories.
I would not rush it, as it is better to take the time to do it right.  Let's all work together to find the optimal approach, as there will be multiple iterations we will have to go through.  It'll be fun :)
Feel free to ping me anytime if you need anything. It would be nice to work together to come up with an elegant solution for this, which I know would benefit everyone as well.
Cheers,
~p


Update README.md

Summary
Fixed a typo

Hi, thanks & looks good. Are you able to click through and accept the CLA? Even for a small change like this, we can't merge until the CLA bot is happy. Thanks!

Oh, disregard. I see the bot is happy now. Thanks!


Array.from_cpointer with CPointer.create violates memory safety

Array's methods and Array.from_cpointer basically trust the user outright; but we can make a null pointer publicly with Pointer.create, and generally blow up
https://stdlib.ponylang.org/builtin-Pointer/#create
https://stdlib.ponylang.org/builtin-Array/#from_cpointer
actor Main
  new create(env: Env) =>
    let arr = Array[U8].from_cpointer(Pointer[U8], 1, 1)
    env.out.print((try arr(0)? else 1 end).string())

I genuinely thought that Pointer.create returned a tag. In fact when I copied pointer code to Mare, I did it as a tag: https://github.com/jemc/mare/blob/44aee9554115e2b17d54c4fe477481dcc34b8ffc/src/prelude/c_pointer.mare#L15-L18
In the docs in Mare I even explained that this is the reason why it must be tag.
But you're right - in Pony it returns a ref. I swear I thought it was tag all along but I just checked the git blame and can't find any change in the recent past - it appears that it's always been this way.
This would be a breaking change, but I believe it must return tag for the safety reasons discussed in this ticket.

What value is there in creating an array that you only have a tag reference to?

I think the point was that Pointer.create would return a tag, and Array.from_cpointer would remain unchanged and return ref

ah, thanks for the clarification @jasoncarr0

If someone wants to working on fixing this, they'll probably run into the fact that Array and String internally rely on being able to create a ref null pointer, and changing this to tag will break that. So to fix those breakages, we'll need to add ._unsafe() after those constructor calls.
The ._unsafe() call can be used within the builtin package but nowhere else. If this breaks other packages, they'll need to find other ways to change their code to either use tag pointers only, or wait for a "real" ref pointer instead of using a null pointer as a placeholder.

I've taken a stab at this. There's one issue with the net and process packages requiring usage of a ref or iso variants of Pointer. Since _unsafe() is private, it's not visible from those packages. Here are the cases where it happens:

NetAddress.name requiring an iso pointer
https://github.com/ponylang/ponyc/blob/eb6fbbb16600249e1d90cd3e6674ebe75fecc5b0/packages/net/net_address.pony#L89-L90

_ProcessWindows.create requiring a ref pointer

Both of these cases follow the same pattern: passing a NULL pointer to a C function, which fills it with data. This pointer is then used to create a string or array using from_cpointer. Using a tag pattern for the first use-case works, but we can't use a tag pointer to get an array or a string.
To get around this limitation, an option is to create the String or Array on the pony side, and use cpointer() to pass it as  a pointer to C, but I don't know the implications of doing this, since the pony String / Array was allocated by Pony, and will be tracked by the garbage collector.

Interestingly enough, pony_os_nameinfo used to return the two pointers as a tuple rather than using the addressof pattern, but in this commit (https://github.com/ponylang/ponyc/commit/dca188cfde26bdc61f861a29f31e2c90d3aeb0b7) it was changed, and as the commit message implies ("returning a struct from a C FFI on Windows is complicated"), this wasn't safe on all platforms and in fact this is now illegal in Pony for that reason (https://github.com/ponylang/ponyc/pull/2012).
So we need to find an efficient approach to get these two filled pointers out from the C side. I have a few ideas rattling around, but I'm not sure which one is the most efficient.

Some other options:

Allow ref pointers to be null, and account for it in all public interfaces. That is, Pointer can optionally point to an allocation.
Follow Rust in a null-pointer optimization with guaranteed ABI: There Option<&Foo>, which is either a reference to Foo or None, is defined to have the exact same layout as a pointer to Foo. I can see a lot of difficulty with this route though.

For the case of ponyint_win_process_create (used by _ProcessWindows.create) the C API refactor is more straightforward: we need to make it return the error_message pointer instead of using addressof for it, then use addressof for the value that is currently the return value.

We discussed on today's sync call that we like approach :three:	the best.
To be more specific, we want to change Array.from_cpointer and String.from_cpointer so that it will keep accepting a Pointer ref, but it will check first for the null pointer case. If the pointer is a null pointer, then it will return a zero-length Array or String, ignoring the length that you asked for.
This closes the safety hole, still behaves in a not-surprising way, and won't break any code that works today. Great idea, @jasoncarr0!

@jemc Yes, I think it makes sense. If I understand point 3 correctly, the default Pointer constructor will still be ref, and we just add an extra to String / Array to prevent unsafe behaviour.
Also, you're right that I misunderstood the original code. I tried my approach of constructing a string on the pony side but I couldn't get it to work, makes sense now!


Error while running deepdream demo

While running deepdream demo as specified in https://applatix.com/open-source/argo/docs/argo_v2_example_ml_workflow.html, I am running into the following error:
$ argo submit ~/tmp/dream.yaml
2018/02/15 16:30:37 Workflow manifest tmp/dream.yaml failed submission: template 'deep-dream-demo' inputs.parameters[0].name: 'IMAGE_URL' is invalid: name must consist of alpha-numeric characters or '-', and must start with an alpha-numeric character (e.g. My-name1-2, 123-NAME)

Tried changing IMAGE_URL to IMAGE-URL from various places in dream.yaml, but then it fails with error:
$ argo submit ~/tmp/dream.yaml
2018/02/15 16:38:31 Workflow manifest /Users/vinaykakade/tmp/dream.yaml failed submission: templates.download-image inputs.parameters.IMAGE_URL was not supplied

Let me know if I am missing something here, environment details below:
Environment:

Argo version:

$ argo version
argo: v2.0.0
  BuildDate: 2018-02-06T21:38:42Z
  GitCommit: 0978b9c61cb7435d31ef8d252b80e03708a70adc
  GitTreeState: clean
  GitTag: v2.0.0
  GoVersion: go1.9.1
  Compiler: gc
  Platform: darwin/amd64

Kubernetes version :

$ kubectl version -o yaml
clientVersion:
  buildDate: 2018-02-09T21:51:06Z
  compiler: gc
  gitCommit: d2835416544f298c919e2ead3be3d0864b52323b
  gitTreeState: clean
  gitVersion: v1.9.3
  goVersion: go1.9.4
  major: "1"
  minor: "9"
  platform: darwin/amd64
serverVersion:
  buildDate: 2018-01-26T19:04:38Z
  compiler: gc
  gitCommit: 925c127ec6b946659ad0fd596fa959be43f0cc05
  gitTreeState: clean
  gitVersion: v1.9.0
  goVersion: go1.9.1
  major: ""
  minor: ""
  platform: linux/amd64

I wanted an excuse to learn a bit about Argo so I've had a quick stab at fixing this.  I think you just need to consistently avoid underscores.

Yes, argo tightened down what it considered valid names in a later release, which broke this demo. @mthx PR fixed this.


SVG dumping and multi row fluctuations

Hi, awesome project! Exciting to try it out.
I have a couple questions, how to dump SVG intermediate after all the stroke path post processing (ie. left align etc) similar to how the original repo did it.
Also, I was wondering how to add row amplitude to the lines or apply some path modification for multiple lines.
imagine human is writing on blank piece of paper, there is always some slant or skew applied to several rows since we cant write straight very well (at least I can't!).
Any idea what an effective approach would be to add in this type of signal adaptation in the preprocess stage?

I realized just uncommenting self._align() would reintroduce these global slant/offsets.
And probably just need to add back in the svg writer to the _draw function in the HandGCode class

Hi, thanks for your kind words.
I completely rewrote how the generated strokes get processed and the documentation is outdated as of version 0.4.0.
Instead of Text -> Neural Network -> SVG -> GCode (-> Postprocessing) my refactored code instead takes the output of the Neural network and samples the strokes directly into a GCode file with all postprocessing steps applied either before converting the stroke coordinates to GCode or directly before writing the commands to the file.
If you just want to preview the GCode, i suggest using a GCode viewer.
If you want to create an SVG instead of GCode using my program, I highly suggest you to look into src/lib/handwriting/commands.py
You can create your own set of commands to write an SVG.
Something like
page->start: "<svg><path data=\""
travel->move: "M {x} {y} "
write->move: "L {x} {y} "
page->end: "\" /></svg>"
Regarding custom effects/post-processing:
You can write custom code to add your own post-processing. Please have a look into the file src/lib/handwriting/gcode.py in the function generate(self, settings).
You can write your own function (i.e. def _postprocess(self, settings, passalong)) and let it run between self._sample() and self._draw().
The passalong object in particular contains all the sampled strokes for each word. You can modify the strokes to slant/get bigger/... The strokes list should contain (nested) lists of coordinates for each word (split by space character) in the sampled text .
If you dont want to post-process the stroke data this way, you may be able to prime the neural network differently to get slightly different handwriting styles for each word/line. I would suggest looking into src/lib/handwriting/gcode.py within _sample(self, settings, passalong). In particular i would look at the prime and biases variables, but for more information about those please refer to the original project.
Good luck :)


Correction for config filename

The LLVM_CONFIG parameter should be set in the config.cmake file for building with the emscripten llvm.
Thanks for contributing to TVM!   Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from others in the community.

@tqchen please see if I'm right.

@tqchen please see if I'm right.

I'm mistaken. There was no config.mk file as mentioned in the tutorial, so I was confused there. Sorry about that.


Add cmake flag -DHPX_WITH_FAULT_TOLERANCE=ON (OFF by default)

to ignore connection reset signals of connections, so HPX does not terminate if a node crashes.

What is the rationale of making this functionality optional? Wouldn't it be better to be always able to handle this kind of exception?

The functionality is optional, because I'm not sure if ignoring a "connection_reset" signal due to suddenly failing remote endpoints is the right way of handling this -- it is a workaround to avoid HPX raising an exception, and terminating, and allowing to explore recovery (which we are working on).

The functionality is optional, because I'm not sure if ignoring a "connection_reset" signal due to suddenly failing remote endpoints is the right way of handling this -- it is a workaround to avoid HPX raising an exception, and terminating, and allowing to explore recovery (which we are working on).

In the end this should be the default, it shouldn't even be possible to disable this. I would have no objections to go ahead with merging this PR if that's the eventual goal.

The goal should be to have a fault tolerance HPX in the end. I am all for having it optional, since there's always a trade off between fault tolerance and performance. This is also the first step in that direction so it makes sense to have it defaulted to being turned off for now.

The goal should be to have a fault tolerance HPX in the end. I am all for having it optional, since there's always a trade off between fault tolerance and performance.

Ok. Fair enough.

This is also the first step in that direction so it makes sense to have it defaulted to being turned off for now.

This PR proposes to have it set to ON by default.

I fixed the comment and pushed the changes to the branch Let me know if you need anything else.

@KADichev now inspect is unhappy: https://9559-4455628-gh.circle-artifacts.com/0/tmp/circle-artifacts.TwEixCv/hpx_inspect_report.html


Superuser in DB is not manager in the app

The DB contains 2 different 'role' fields for a User: isSuperUser and isStaff, but both are required to be True for a User to be Manager on the app:
https://github.com/ecrin-github/rms-portal-new/blob/2b29c666224c03c4a4775585351e20ac8e622533/src/app/pages/internal/main-page/internal-main-page.component.ts#L59
https://github.com/ecrin-github/rms-portal-new/blob/2b29c666224c03c4a4775585351e20ac8e622533/src/app/pages/internal/main-page/internal-main-page.component.ts#L76

I don't think there's anything to test, you can probably just close the issue @sergiocontrino

yes, remove any logic in the fe involving the staff is for the moment the right solution, closing.


At RESTORE, if a valid registry tag for a cache has been specified, it fails to use the cached layers

This the warning I'm seeing. Using the pack binary, I know what is the flag I need to use for this. How do I get around this in kpack's context? Is there a way in which I can add -cache-image option to the restore init container using a flag?

This can be done using the .spec.cache fields. You can choose between caching via k8s PVs or the registry


[.Net SDK, Skype for Business]  How to know the status of Skype user on Proactive message sending?

There is a same question from my side. We intergrate the bot into skype for business. When we send the Notification message in a specified time from a bot to a skype user, how could we know the user whether online or offline? Any hits for this ?
With .Net bot.
Similar question with the Node.js. 3126

Thank you for opening an issue against the Bot Framework SDK v3. As part of the Bot Framework v4 release, we’ve moved all v3 work to a new repo located at https://github.com/microsoft/botbuilder-v3. We will continue to support and offer maintenance updates to v3 via this new repo.
From now on, https://github.com/microsoft/botbuilder repo will be used as hub, with pointers to all the different SDK languages, tools and samples repos.
As part of this restructuring, we are closing all tickets in this repo.
For defects or feature requests, please create a new issue in the new Bot Framework v3 repo found here:
https://github.com/microsoft/botbuilder-v3/issues
For Azure Bot Service Channel specific defects or feature requests (e.g. Facebook, Twilio, Teams, Slack, etc.), please create a new issue in the new Bot Framework Channel repo found here:
https://github.com/microsoft/botbuilder-channels/issues
For product behavior, how-to, or general understanding questions, please use Stackoverflow.
https://stackoverflow.com/search?q=bot+framework
Thank you.
The Bot Framework Team


Sweep node tests to use zrequire.

Nice, it's great when we get to delete 250 lines of code.  Merged, thanks @showell!


Allow configurable client signing algorithms

Description
I've filed similar issues under Cosign and Rekor. I realise there's a lot of overlap in maintainers, but wanted to make sure that we discuss each project that we plan to touch. Apologies if this feels a bit spammy.
Hi there! At Trail of Bits, we're looking at potentially implementing part of the Configurable Crypto Algorithms proposal (specifically Phase 1). I wanted to float this idea to each of the relevant Sigstore sub-projects so we can hash out the details in a more concrete way.
Across the Sigstore stack, we default to using ECDSA for signatures and SHA256 for hashing. There's more detail in the linked proposal but there are a number of motivations for wanting to customise the signatures that are generated, including paving the way for post-quantum signatures. The proposed design includes having a "supported algorithm" registry (perhaps this can go in the Protobuf specs) that outlines enumerates the approved signature/hash algorithm combinations. We specifically don't want to allow arbitrary mixing and matching of signature and hash algorithm to avoid some of the security pitfalls listed in the proposal.
I'm not super familiar with the Fulcio side of things but I expect that there's a few places where the assumption that the client signing algorithm is ECDSA-SHA256 is hardcoded, so we'll need to make those more flexible. We can also add a flag like --client-signing-algorithms=alg-1,alg-2 that can be used in case a given Fulcio instance wants to constrain the list of supported algorithms to be more restrictive than the aforementioned registry.

There aren't many places where the signing algorithm is hardcoded. To give a quick overview, there's a few CA "backends" that implement signing logic given a key in some format - KMS, Tink, in-memory, etc. For each backend:

KMS - This uses a cloud KMS to sign certificates. The signing algorithm and hash are provided from the sigstore/sigstore library. Once a KMS provider has added support for PQ signing algorithms, we'll just need to add that to s/s.
Tink - In-memory signing where the signing key is encrypted with a KMS key. Tink will need to be updated to support PQ algorithms (which I assume is on their roadmap)
File-based - Meant as a test CA since keys stored on disk (even password-protected) aren't ideal. This supports any signing key type supported in sigstore/sigstore.
Ephemeral - This is a test CA which hardcodes ECDSA P-256 where the signing key lives in-memory. We can easily make this configurable.

tl;dr is that you'll need to make changes to sigstore/sigstore to support PQ signing keys. If Tink doesn't add support for PQ any time soon, another CA "backend" that loads a KMS-encrypted key into memory might be useful so we don't proliferate the use of the file-based CA (though KMS providers will need to add support for PQ encryption too).
We'll also need to specify the OID of the signature algorithm in the relevant extension, which currently is populated by the Go x509 library rather than self-specific.
You'll also need to update certificate-transparency-go to support PQ, since the SCTs are signed. I'm not sure how this library detects the signing algorithm.

Awesome, thanks for the details @haydentherapper.
Since we're supporting configurable client signing algorithms only as a first pass, I believe that we can kick the can down the road and address the majority of this work later. Most of those points look to be related to configuring the algorithm that Fulcio as a CA uses to sign the issued certificates rather than the type of key embedded in the issued certificates, about SCTs, etc.
To allow configurable algorithms for client signing, I believe we need to add a flag to restrict the allowed subset of algorithms. It looks like we're handling already handling most of the key types we want here but we'd need to add LMS.

I think there might be some problems because cosign still uses a “hand-made” proof of possession instead of using the CSR, so there is no place where the full client algorithm is specified in the message. Fulcio just assumes the hash algorithm is always SHA256, however with recent PRs we also accept SHA384 and SHA512.
That means however that if a client uses cosign with e.g. ecdsa-sha2-384-nistp384, Fulcio will refuse to provide a certificate because it does not understand it should use sha384 for verifying the proof of possession.
What do you think is the best way to handle this? We might just ignore this and accept that cosign will only work with sha256 and at some point it will move to the CSR message.
cc @woodruffw @haydentherapper

See https://github.com/sigstore/fulcio/pull/1517/files#diff-648d47fb9eeb444c1a09095dd41e4012ee5aafcb37b712f7f3bf492d8410017dR146 for how the in-progress PR tries to handle this, but this works only if you assume that a given type of public key only has one type of hash algorithm.

Also, shall we assume ed25519-ph in Fulcio? There is no way to distinguish between ed25519 and ed25519-ph. Are both used by (some) clients, in some cases? This may poses other problems to keep compatibility.

What do you think is the best way to handle this? We might just ignore this and accept that cosign will only work with sha256 and at some point it will move to the CSR message.

This would be my proposal as well. It is straightforward to generate a CSR in Golang, so we can easily switch over now. We'll need to continue to support older Cosign versions or API callers, so assuming sha256 seems reasonable.
A few alternatives:

Add an option to specify the hash and or signature algorithm in the PublicKeyRequest message. We'd still need to fallback to sha256 if no alg is specified. I'm not opposed to this approach, and maybe it's worth doing in addition to using the CSR for clients where CSR generation is not as straightforward.
Change the challenge. I want to do this in the near future anyways, to make the challenge more ACME-like to avoid challenge reuse. We could add algorithm details as part of the challenge response. However this is a more significant change and I was planning to wait until we're ready for a breaking API change.

Also, shall we assume ed25519-ph in Fulcio? There is no way to distinguish between ed25519 and ed25519-ph. Are both used by (some) clients, in some cases? This may poses other problems to keep compatibility.

One thing to note is that the signature algorithm OID is derived from this call, during x509.CreateCertificate. Given our currently supported set of signing algorithms, besides ed25519ph, we should not need to pass SignatureAlgorithm in the template.
ed25519ph is not one of the supported algorithms. My understanding is because it was not specified in https://datatracker.ietf.org/doc/html/rfc8410 (it was in an earlier revision of the rfc).
This is going to get a bit messy - You could sign the challenge with pure ed25519, Fulcio verifies the challenge with ed25519, and certify the key as pure ed25519 [1]. The client will then know to sign the artifact with ed25519ph for hashedrekord or ed25519 for other types. In Rekor, when the verifier is loaded, again it knows which signature algorithm to expect for each Rekor type, so it doesn't matter what the signature algorithm is in the certificate. (italicized because this has yet to be fully verified, but I don't believe any client in sigstore looks at the signature algorithm).
[1] You could also sign the challenge with ed25519ph, and then Fulcio would verify the challenge either as ed25519ph or ed25519. But given it can only certify the key as ed25519, I'd rather just avoid Fulcio being aware of -ph entirely.

ed25519ph is not one of the supported algorithms. My understanding is because it was not specified in https://datatracker.ietf.org/doc/html/rfc8410 (it was in an earlier revision of the rfc).

Ok, this is going to be messy then :) Do you have any idea if there ed25519-ph is ever going to be part of the supported algorithms? Otherwise, what you said might work, but it relies on a lot of assumptions across the whole system.

There was an OID (https://oid-rep.orange-labs.fr/get/<IP_ADDRESS>) allocated in an earlier revision of the RFC, but for whatever reason, it was removed. I assume this OID would be reused if it were to be standardized.
To provide a custom signature algorithm OID, I'm unfortunately not sure if you can. With extensions, you'd need to craft your own extension in ExtraExtensions which should override a default extension (for example, we do this to force the extended key usage to be critical when timestamping code). With this, it's not an extension but a certificate field, and it seems like it only accepts a set of enums code. We might need to file a request for custom signature algorithms on the Golang issue tracker.

Also this is going to be an issue when we want to experiment with PQ-signed certificates.

From asking around, some more info:

Support for ed25519ph has stalled after it was not included in RFC8410. The reason, from the working group summary, is that they could not find a use case for needing the prehash variant, specifically dealing with large data. Large CRLs were brought up, but the preferred solution was segmenting CRLs, rather than supporting large CRLs.
Golang would be unlikely to accept a PR to add support for nonstandard signature algorithms, as the focus in on webPKI
We could patch crypto/x509, specifically x509.go, although I'm reluctant to maintain a fork of this. It would be quite simple to do so, modifying signingParamsForPublicKey and signatureAlgorithmDetails

I would recommend proceeding with certifying ed25519ph keys as ed25519. Since we're adding the feature to Rekor to allow hashedrekord + ed25519ph, any client that has dealt with ed25519 keys would have to be for the byo-key use case. While this would require contextual verification, knowing that ed25519 + hashedrekord = ed25519ph, this should be possible to build into all clients.


Ops Agent does not support traceSampled field mapping

Describe the bug
When trying to set jsonPayload."logging.googleapis.com/trace_sampled" it is not displayed as mapped in Logs Explorer and remains in jsonPayload
To Reproduce
Steps to reproduce the behavior:

Start a GCE VM with image RHEL8
Install Ops Agent version 2.46.1
Customize the agent with configuration

logging:
  receivers:
    messages:
      type: files
      include_paths:
      - /var/log/messages
    secure:
      type: files
      include_paths:
      - /var/log/secure
    applogs_tcp:
      type: tcp
      format: json
      listen_port: 5160
  processors:
    parse:
      type: parse_json
      time_key: jsonPayload.timestamp
      time_format: "%Y-%m-%dT%H:%M:%S.%L%Z"
    adjust_fields_to_stackdriver:
      type: modify_fields
      fields:
        jsonPayload."logging.googleapis.com/trace_sampled":
          default_value: false
  service:
    pipelines:
      default_pipeline:
        receivers: [messages, secure]
      applogs_pipeline:
        receivers: [applogs_tcp]
        processors: [parse, adjust_fields_to_stackdriver]

and restart the agent
6. See log entry does not have traceSampled property and it remains in jsonPayload as jsonPayload."logging.googleapis.com/trace_sampled": "false"
Expected behavior
Value mapped in logs explorer
Environment (please complete the following information):

Project ID - can send via email
VM ID - can send via email
VM distro / OS: RHEL8
Ops Agent version 2.46.1
Ops Agent configuration
Ops Agent log

Additional context
NA

hi @braydonk
No luck with above, I was referring to Google documentation on structured logging:
https://cloud.google.com/logging/docs/structured-logging
We were facing similar issue with logging.googleapis.com/spanId field in same manner but I got resolved I think with some release of ops agent so potentially this needs similar development. Also we are unable to send it as boolean in jsonPayload to map so would it work if we use map_values mapping?
map_values:
  "true" : true
  "false": false

Hi @PawelKGalka, just to learn a bit more about your use case, would it be acceptable if the Ops Agent were to set traceSampled automatically based on whether there's a spanId and trace present?
We've had a few other folks ask for ability to set traceSampled manually as well, and it seems like all use cases so far would benefit from something automatic instead so that you don't need to configure anything.

Hi @jefferbrecht,
Our application is utilizing tracing in 3 modes based on particular criterias - DROP,RECORD,RECORD_AND_SAMPLE (from OTEL lib). This means there are cases where we trace to correlate transaction in the logs but we do not sample it to Cloud Trace. So for our use case it would be best to have this configurable in the config. Or it can take precedence over default setting when spanId/traceId is set


Add final blackduck version

Description
add BlackDuck scan for PRs as well as per cronjob

Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change.
What type of PR is this? (check all applicable)

[ ] 🍕 Feature
[ ] 🐛 Bug Fix
[ ] 📝 Documentation Update
[ ] 🎨 Style
[ ] 🧑💻 Code Refactor
[ ] 🔥 Performance Improvements
[x] ✅ Test
[ ] 🤖 Build
[ ] 🔁 CI
[ ] 📦 Chore (Release)
[ ] ⏩ Revert

Related Tickets & Documents

Related Issue # (issue)
Closes # (issue)
Fixes # (issue)

Remove if not applicable

Screenshots

Added tests?

[ ] 👍 yes
[x] 🙅 no, because they aren't needed
[ ] 🙋 no, because I need help
[ ] Separate ticket for tests # (issue/pr)

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Added to documentation?

[ ] 📜 README.md
[x] 🙅 no documentation needed

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

the issue brought up in the Blackduck check is for the docker/go-metrics project. The Creative Commons license is just for the docs, not for code (which is under APL 2.0), so all good.


Default app locales should derive from "Preferred Languages" on Windows

Currently, Flutter on Windows determines the default locales to assign to the platform configuration based on the value obtained by calling GetThreadPreferredUILanguages. This does not necessarily return the actual languages listed in the "Preferred Languages" setting. We should get the full list of preferred languages to use for application locales.
We can achieve this by querying the registry with
RegGetValueA(HKEY_CURRENT_USER, "Control panel\\International\\User Profile", "Languages", RRF_RT_REG_MULTI_SZ, NULL, str_buffer.data(), &buffer_size)

If this fails, we can fallback to the current implementation.

Resolved by https://github.com/flutter/engine/pull/36536


Feature/media explorer

The media explorer was reworked to the point where this PR is unfortunately no longer relevant


Spaces in Content-Type boundary parameters results in Error: Malformed content type

Sending a boundary containing whitespace (e.g. Content-Type,multipart/form-data;boundary=HyperLog -1663179571660) will cause Busboy to error.
Error: Malformed content type
    at getInstance (/layers/google.nodejs.yarn/yarn_modules/node_modules/busboy/lib/index.js:9:11)
    at module.exports (/layers/google.nodejs.yarn/yarn_modules/node_modules/busboy/lib/index.js:56:10)

I manually confirmed this by calling parseContentType directly.
const samples = [
  'multipart/form-data;boundary=TestCase-1663179571660',
  'multipart/form-data;boundary=TestCase-1663179571660 ',
  'multipart/form-data;boundary=testcase-1663179571660',
  'multipart/form-data;boundary=testcase-1663179571660 ',
  'multipart/form-data;boundary=TestCase<PHONE_NUMBER>660',
  'multipart/form-data;boundary=testcase<PHONE_NUMBER>660'
]

for (let sample of samples) {
  const obj = parseContentType(sample);
  console.log(`${sample} -> ${JSON.stringify(obj)}`)
}

resulting in
multipart/form-data;boundary=TestCase-1663179571660 -> {"type":"multipart","subtype":"form-data","params":{"boundary":"TestCase-1663179571660"}}
multipart/form-data;boundary=TestCase-1663179571660  -> {"type":"multipart","subtype":"form-data","params":{"boundary":"TestCase-1663179571660"}}
multipart/form-data;boundary=testcase-1663179571660 -> {"type":"multipart","subtype":"form-data","params":{"boundary":"testcase-1663179571660"}}
multipart/form-data;boundary=testcase-1663179571660  -> {"type":"multipart","subtype":"form-data","params":{"boundary":"testcase-1663179571660"}}
multipart/form-data;boundary=TestCase<PHONE_NUMBER>660 -> undefined
multipart/form-data;boundary=testcase<PHONE_NUMBER>660 -> undefined

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type says:

boundary
For multipart entities the boundary directive is required. The directive consists of 1 to 70 characters from a set of characters (and not ending with white space) known to be very robust through email gateways. It is used to encapsulate the boundaries of the multiple parts of the message. Often, the header boundary is prepended with two dashes and the final boundary has two dashes appended at the end.

https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html says:

The only mandatory parameter for the multipart Content-Type is the boundary parameter, which consists of 1 to 70 characters from a set of characters known to be very robust through email gateways, and NOT ending with white space. (If a boundary appears to end with white space, the white space must be presumed to have been added by a gateway, and should be deleted.) It is formally specified by the following BNF:

boundary := 0*69<bchars> bcharsnospace

bchars := bcharsnospace / " "

bcharsnospace :=    DIGIT / ALPHA / "'" / "(" / ")" / "+"  / "_"  / "," / "-" / "." / "/" / ":" / "=" / "?"

Thus I feel that supporting spaces in boundaries would be an improvement.

If your boundary contains whitespace, it needs to be properly quoted (like any other parameter), otherwise the parser has no idea whether the next token is part of the previous parameter or a new parameter.

https://github.com/roblperry/busboy/commit/956eb8dd7e005994c56e2b741253ea4d4063a195 seems to work for me, but I am not that terribly familiar with the standard.  So, I recognize that it would be easy for me to be missing important side effects of my changes.
For that matter, I now see that I missed this part: "If a boundary appears to end with white space, the white space must be presumed to have been added by a gateway, and should be deleted." But, I should be able to solve that if this general direction is agreeable.

Whitespace after an unquoted parameter value should be discarded. Whitespace at the beginning or in the middle of an unquoted parameter value would be a syntax error.
The easiest solution to avoid having to worry about any of these problems is to just always use quoted values.

I agree that the client library in question would be improved by modifying it to avoid placing a space in the boundary.  I can't imagine the benefit of having said space. I also don't want to take up too much of your time. However, I would very much like to point the maintainers of the client library at a standard backing my case, otherwise I am just going to be stuck stating that not all server side implementations support spaces inside the boundary, which isn't a terrible argument.  I'd just rather have something stronger.
Clearly, I've looked.  Am I looking at the wrong spec? Do you have anything for me on that front?
Either way, thank you for busboy.  I rather enjoyed crawling around in the performance optimized parser.  I don't think I've played with that style of code since the 90s.

It's an HTTP header, so you would be referencing the ABNF from RFC 7231 (and RFC 7230 for some reused, general types), starting with the Content-Type token.

Excellent.  Thank you, sir.
For anyone else who happens across this RFC 7231 lead me to:

parameter = token "=" ( token / quoted-string )

where

token = <token, see [RFC7230], Section 3.2.6>

which aligns with what mscdex has been trying to tell me.


feat: helper methods on WalOp

This adds a couple of helpers to the WalOp type that were needed in pro, and are generally useful.

I will wait for https://github.com/influxdata/influxdb/pull/25495 to merge this.


[Documentation] Gentoo installation instructions b0rked

Kivy documentation currently lists Kivy as trivially installable under Gentoo Linux via this convenient one-liner:
emerge Kivy

Tragically for us all, that hasn't worked for several years. Kivy was last-rited (i.e., removed from Gentoo's official Portage package repository) over two years ago. Why? Because:

No other Portage packages depended on Kivy.
No one could figure out how to install Kivy 1.10.1 without Cython failures. Not kidding.

This means that running the above one-liner now yields:
$ emerge Kivy

These are the packages that would be merged:

Calculating dependencies ... done!

emerge: there are no ebuilds to satisfy "Kivy".

emerge: searching for similar names...
emerge: Maybe you meant dev-python/kiwisolver?

No, Portage. I did not mean kiwisolver. :face_exhaling:
A New Hope
Thankfully for us all, Gentoo's raiagent overlay recently resurrected support for the full Kivy + Buildozer + python-for-android + KivyMD stack in celebration of the monumental Kivy 2.1.0 release. Gentoo overlays are sorta like Ubuntu Personal Package Archives (PPAs). Full disclosure: I'm the one who did it. :partying_face:
This means that Kivy is yet again installable under Gentoo – albeit not quite as trivially:
$ eselect repository enable raiagent
$ emerge --sync raiagent
$ emerge --autounmask Kivy

So, a three-liner instead of a one-liner. USE flags also substantially differ. garden is no longer a thing, for example.
If you'd like, I'd be happy to submit a PR patching up the docos.

Hi @leycec,
Really appreciate your dedication to get Kivy working on Gentoo!
Having broken docs is certainly something absolutely not-nice and will distract users from really enjoying Kivy.
This means that having a PR that fixes it is something that I will love ❤ .
Unfortunately, considering that 2.1.0 has already been released, it will not be included into 2.1.0/ stable docs, unless we decide to cherry-pick it.
As Ubuntu PPAs, Gentoo support is something that would be nice to be covered during releases. Maybe we should redact a TODO list for Gentoo that we can follow during the next release?

This means that having a PR that fixes it is something that I will love :heart: .

Your wish is Gentoo's demand. :full_moon_with_face:
When I find a free moment, a PR shall be done for the good of the community... and the native Italian red squirrel, of course!

Unfortunately, considering that 2.1.0 has already been released...

No worries whatsoever. The 2.1.0 release cycle was a bit of a gruelling slog, so... we're all just delighted that Kivy's back on top of Python's GUI heap. :mount_fuji:
Thanks so much for the many tireless years of volunteerism, @misl6. You're amazing. Everyone here is.

Maybe we should redact a TODO list for Gentoo that we can follow during the next release?

Yeah! Let's do this, Team Kivy. A preliminary TODO for you:

Shortly before the next release drops, ping @leycec onto a relevant GitHub issue, PR, or discussion thread with a brief list of all significant installation changes to either Kivy itself, Buildozer, or python-for-android – with particular emphasis on changes to:

Mandatory install-time, runtime, test-time, and doc-time (i.e., Sphinx) dependencies.

So, dependencies is the main thing is what I'm saying. Dependencies don't often change – but when they do, it'd be great to get a heads up. :speaking_head:

Yup. We doin' this.


[BUG] CopyStatus is not updated after copy is finished

Describe the bug
I'm making the migration from Microsoft.WindowsAzure.Storage.Blob versio <IP_ADDRESS> to  Azure.Storage.Blobs version <IP_ADDRESS> and trying to replicate the CopyBlobData functionality. In the oldest version, we could check a property CopyStatus to see if the copy is finished. In the new one we still have this property, but it's never updated and so we can never know if the copy has ended. See more bellow:
1. Original code
public virtual void CopyFromBlob(string sourceConnectionString, string sourceContainerName, string sourceBlobName, string targetConnectionString, string targetContainerName, string targetBlobName)
{
    var sourceContainer = GetContainer(sourceConnectionString, sourceContainerName);
    CloudBlockBlob sourceBlockBlob = sourceContainer.GetBlockBlobReference(sourceBlobName);

    var targetContainer = GetContainer(targetConnectionString, targetContainerName);
    CloudBlockBlob targetBlockBlob = targetContainer.GetBlockBlobReference(targetBlobName);

    targetBlockBlob.StartCopy(sourceBlockBlob);

    while (targetBlockBlob.CopyState.Status == Microsoft.WindowsAzure.Storage.Blob.CopyStatus.Pending)
    {
        Thread.Sleep(500);
    }

    if (targetBlockBlob.CopyState.Status != Microsoft.WindowsAzure.Storage.Blob.CopyStatus.Success)
    {
        throw new Exception("Copy failed: " + targetBlockBlob.CopyState.Status);
    }
}

2. New version
public virtual void CopyFromBlobNew(string sourceConnectionString, string sourceContainerName, string sourceBlobName, string targetConnectionString, string targetContainerName, string targetBlobName)
{
    var sourceContainer = GetContainerNew(sourceConnectionString, sourceContainerName);
    var sourceBlob = sourceContainer.GetBlobClient(sourceBlobName);

    var targetContainer = GetContainerNew(targetConnectionString, targetContainerName);
    var targetBlob = targetContainer.GetBlobClient(targetBlobName);

        var copyInformation = targetBlob.SyncCopyFromUri(sourceBlob.Uri);

        while (copyInformation.Value.CopyStatus == Azure.Storage.Blobs.Models.CopyStatus.Pending)
        {
            Thread.Sleep(500);
        }

        if (copyInformation.Value.CopyStatus != Azure.Storage.Blobs.Models.CopyStatus.Success)
        {
            throw new Exception("Copy failed: " + copyInformation.Value.CopyStatus);
        }
}

The problem is: the CopyStatus property inside the BlobCopyInfo object that the SyncCopyFromUri method returns is never updated. I've eve tried to get the blob properties directly, but, as you can see bellow, the CopyStatus property still with the wrong value even if there is no more data to copy:

Expected behavior
The CopyStatus property should be updated as soon as the copy is finished.
Actual behavior (include Exception or Stack Trace)
The CopyStatus is never updated
To Reproduce
Described above
Environment:

Azure.Storage.Blobs version <IP_ADDRESS>
Windows 10, .NET Framework 4.7
IDE and version : Version 16.8.5

Thank you for your feedback.  Tagging and routing to the team best able to assist.

@WillRock19, this is by design.
copyInformation is not a dynamic object, and will not be updated.
I would recommend this approach instead:
public virtual async Task CopyFromBlobNewAsync(
    string sourceConnectionString,
    string sourceContainerName,
    string sourceBlobName,
    string targetConnectionString,
    string targetContainerName,
    string targetBlobName)
{
    BlobContainerClient sourceContainer = GetContainerNew(sourceConnectionString, sourceContainerName);
    BlobClient sourceBlob = sourceContainer.GetBlobClient(sourceBlobName);

    BlobContainerClient targetContainer = GetContainerNew(targetConnectionString, targetContainerName);
    BlobClient targetBlob = targetContainer.GetBlobClient(targetBlobName);

        await targetBlob.SyncCopyFromUriAsync(sourceBlob.Uri);
        Response<BlobProperties> blobPropertiesResponse = await targetBlob.GetPropertiesAsync();
        while (blobPropertiesResponse.Value.CopyStatus == Azure.Storage.Blobs.Models.CopyStatus.Pending)
        {
            Thread.Sleep(500);
            blobPropertiesResponse = await targetBlob.GetPropertiesAsync();
        }

        if (blobPropertiesResponse.Value.CopyStatus != Azure.Storage.Blobs.Models.CopyStatus.Success)
        {
            throw new Exception("Copy failed: " + copyInformation.Value.CopyStatus);
        }
}

-Sean

Hi @seanmcc-msft , how are you?
I think we might need to re-open this issue. The solution that you've proposed does not seems to work, since everytime I'm calling the GetProperties the resulted blobPropertiesResponse still have it's CopyStatus with the value Pending. Here is my code:
        public virtual void CopyFromBlobNew(string sourceConnectionString, string sourceContainerName, string sourceBlobName, string targetConnectionString, string targetContainerName, string targetBlobName)
        {
            var sourceContainer = GetContainerNew(sourceConnectionString, sourceContainerName);
            var sourceBlob = sourceContainer.GetBlobClient(sourceBlobName);

            var targetContainer = GetContainerNew(targetConnectionString, targetContainerName);
            var targetBlob = targetContainer.GetBlobClient(targetBlobName);

            if (!sourceBlob.Exists())
                throw new RequestFailedException($"A copy blob operation was requested, but the source blob {sourceBlobName} does not exist!");

            targetBlob.SyncCopyFromUri(sourceBlob.Uri);
            var blobPropertiesResponse = targetBlob.GetProperties();

            while (blobPropertiesResponse.Value.CopyStatus == CopyStatus.Pending)
            {
                Thread.Sleep(500);
                blobPropertiesResponse = targetBlob.GetProperties();
            }

            if (blobPropertiesResponse.Value.CopyStatus != CopyStatus.Success)
            {
                throw new Exception("Copy failed. The resulted status is: " + blobPropertiesResponse.Value.CopyStatus);
            }
        }

I've post the synchronous version that I need to use, but I've tried the asynchronous version you've post it and it never comes out from the while statement (while debugging, I've noticed the blobPropertiesResponse.Value.CopyStatus never change from Pending, doesn't matter how many times I call the targetBlob.GetProperties()).

Hi @seanmcc-msft, any updates on this?

Hi @seanmcc-msft, any updates on this?


GPIO serial write

Dear Joan / devs,
I respect the work you put in this great library!
Let me ask about gpio (bitbang) serial writing functionality: is it supported by the library? I couldn't find it in the documentation / header, although writing functionality appears in bitbang i2c, spi.
Thanks

For C see http://abyz.me.uk/rpi/pigpio/cif.html#gpioSerialReadOpen etc,  There should be similar calls for Python, pigs, etc.

I closed this by mistake.

Thanks Joan for the quick answer, but I would like to WRITE, however all these gpioSerial* function show only READING functionality. Even the opener is 'gpioSerialReadOpen', and I couldn't find a writer counterpart. Do I miss something? Thanks!

gpioWaveAddSerial

Ahhh great, thank you!

My mistake.  I had forgotten that the write was different to the read (for reasons which are apparent when you write the code, I perhaps should link between the calls in the documentation).

That would be helpful for future users! Anyhow, thanks for the quick help!!!


Namespaces kept on "Terminating" for ever

Hi! Thanks for this useful app, we have been playing around with it a bit, buy my namespaces keep getting into "Terminating" state for a lot of time, sometimes they get deleted, but almost never.
I couldn't' found the reason, may you could throw some light ?
They all get this metadata:
Annotations
remove-empty-ns-operator.kopf.dev/will-remove=True
Finalizers
kopf.zalando.org/KopfFinalizerMarker

I've setup the info logs to the pod and everything seems ok, I get, for example:
[release-testing] namespace release-testing has deletion mark, deleting
but the namespace keeps on terminating status.
UPDATE:
After deleting the operator pod, all namespaces got deleted. I guess the finalizer is expecting operator pod to NOT exister after deleting namespaces ?

I don't think so, we are using kopf operators for remove-empty-ns only.


Keyboard-driven code completion in InternalShell crashes

From<EMAIL_ADDRESS>on 2011-08-16T12:53:42Z
I am using the InternelShell class in a PySide application.  I am using the latest development version of spyderlib from Hg.  Python 2.7 32-bit, Windows 7 Pro 32-bit.
When interacting with the resulting shell, code completion works fine as long as we only use the mouse to select the desired completion result.  But if we use the keyboard cursor keys to move up or down the completion list, the application usually crashes, and the Windows application error dialog appears, stating that python.exe has stopped working.
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=734

From ccordoba12 on 2014-08-23T13:35:56Z
Status: WontFix

From pierre.raybaut on 2011-08-22T02:58:03Z
Unfortunately, PySide is still young and Spyder's behavior is evolving very quickly from a release to another (in the good direction, hopefully). For example, the "Preferences" dialog box was not working with PySide v1.0.4 but is now working with PySide v1.0.5...
I'll do some tests.
Status: Accepted
Labels: Cat-Console Component-PySide


Different page/post types?

Does Bludit have the concept of different post types?
I realize there are categories, but I would like to have differing post types instead. For example, for a portfolio site where I have writing and photos, I would treat those two types of content differently.
Additionally, is it possible to elevate categories to the nav, or allow for non-dynamic links (such as offsite or to the category pages)?

It is possible if you make a plugin for this. The methodology is simple with the Hook: afterPageModify.
Here I leave an approach on how to do it.
#1480


OCPBUGS-36681: Set Azure VM identity if user assigned identity set

What this PR does / why we need it:
Only set the Azure VM identity if a user assigned identity was provided.
Which issue(s) this PR fixes:
Fixes OCPBUGS-36681
Checklist

[x] Subject and description added to both, commit and PR.
[x] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

/cherry-pick release-4.16

/area hypershift-operator

/lgtm


Allow test_isolate to run without install

This updates the commonlib loading logic so that it works without having
corier installed into a system path (the current logic assumes it's
available somewhere, but sometimes when you're developing locally and
you don't have a copy installed into the system)

Does this work if the commonlib.so is dynamically linked? consulting @rdaly525 for any weirdnesses here...

Yes, this was how libraries were designed to be loaded (see https://github.com/rdaly525/coreir/blob/master/doc/LibraryGuide.md#libraries), it's also how the other tests all load the libraries (https://github.com/rdaly525/coreir/blob/master/tests/gtest/test_verilog.cpp#L166-L168)


Bumb TypeScript Version to 2.0

Related to: https://github.com/zalando-incubator/lib-oauth-tooling/issues/2
We should get rid of all external dependencies!


Upgrade to Selenium server 2.3

Originally submitted to Google Code by tsavf... on 27 Jun 2011

New Selenium 1.0.12 brings support to Firefox 5

Originally submitted to Google Code by @jussimalinen on 29 Jun 2011

The final release of Selenium Server should be out in couple of weeks. As the core team is very busy with the Robot 2.6, I think we can wait for the final Selenium release before making a new Selenium Library package.
Remember that you can replace the selenium server from your installation by yourself by downloading the latest jar and replacing the one that came with installation. Or you can start any server version manually [1].
[1] https://code.google.com/p/robotframework-seleniumlibrary/wiki/InstallationInstructions#Starting_Selenium_Server

Originally submitted to Google Code by @yanne on 8 Aug 2011

Selenium 2.3 was released last week, so let's include that in the next release.


Thin rounded rectangle path stroke artifacts

This simple svg just draws a 1px stroke rounded rectangle path using lines and cubics.
<svg width="58" height="19" viewBox="0 0 58 19" fill="none" version="1.1" id="svg7" xmlns="http://www.w3.org/2000/svg">
   <path
      d="m 9.5,0.5 h 39 c 4.9706,0 9,4.02944 9,9 0,4.9706 -4.0294,9 -9,9 h -39 c -4.97056,0 -9,-4.0294 -9,-9 0,-4.97056 4.02944,-9 9,-9 z"
      stroke="#ffffff" id="path5" />
</svg>

This is the rendered image in ThorVG 0.12.9 with no scaling applied.
As can be seen the corners seem quite off

If we inspect this more closely in pixspy,
we can notice that the pixel coverage isn't uniform across the stroke.

This is how the same svg looks renderered by svgviewer running in Chrome:

The problem gradually fades away as the stroke width is increased. As thin strokes are very commonplace in UI design I believe this is quite an important issue.

Somehow, the software renderer compromises quality for performance, this is more of a quality issue than a bug, instead you can observe the svgviewer doesn't seem to render 1-pixel widths accurately.

@hermet I did more tests and found out that the rendering looks quite perfect when using straight alpha tvg::SwCanvas::ABGR8888S.

For comparison this is the same rendered with LunaSVG

Perhaps it's a bug in our rendering engine that doesn't handle pre-multiplied alpha correctly (although it should).
I saw this parameter documentation from Picture::load(uint32_t* data, uint32_t w, uint32_t h, bool copy) noexcept at line 1318 that seems to be left over from a previous version
* @param[in] premultiplied If @c true, the given image data is alpha-premultiplied.
Could this be related in any way ?

@lmdsp it makes sense that the rendering result appears darker than expected when the option is not properly configured. If you're integrating a native system that expects straight (unpremultiplied alpha) images for composition, you'll need to configure the canvas using tvg::SwCanvas::ABGR8888S. This setting is optional because some systems support straight alpha, while others support pre-multiplied alpha. Otherwise, both ways are supported. So I think you just need to use the straight mode. That's all. The load() function you mentioned is not relevant to vector drawing.


It is the problem of shuffle. I did not shuffle my data.

It is the problem of shuffle. I did not shuffle my data.
Originally posted by @hxs91 in https://github.com/google-research/bert/issues/264#issuecomment-449802184
I meet the same problem on chines data set, why it is due to shuffle, could you make clear?

@NNNNNaaaaaa You could refer to https://github.com/Socialbird-AILab/BERT-Classification-Tutorial/issues/11


Individual Sprint 2 GUI

Sprint 2

I encountered some challenges when integrating this code into our application. It works on my practice project but I can not get the JAVA FX Controller to work properly on our app. To ensure its functionality, I created a multi-scene JavaFX graphical user interface within a practice project. Here's a brief overview of the implemented features:
The application starts with a home screen featuring two buttons.
One button leads to a ping scene, where users can input an IP address and initiate a ping operation. This scene also includes a button to return to the home screen.
The other button directs users to a port scanner scene, which allows them to specify port ranges for scanning. Similar to the ping scene, there's a button to return to the home screen.
This structure provides a user-friendly interface for conducting both ping and port scanning operations seamlessly.
I'm also having problems implementing scene builder, I have watched multiple videos and somewhere I'm messing up on setting it up. I like the direction I'm headed so far.


Array inline validation

I need to clarify how to do a validation of array values.
Two examples
one:
{
  currencies: ["USD", "EUR", "GBP"]
}

two:
{
  currencies: [{"code": "USD"}, {"code": "EUR"}, {"code": "GBP"}]
}

Is it any option to check values of array using validator?

Currently, that could be achieved using a custom validator.
const expressValidator = require('express-validator')
const validator = expressValidator.validator
...
app.use(expressValidator({
  errorFormatter: (param, msg, value) =>  ({ [param] : msg }),
  customValidators: {
    isX: value =>
              Array.isArray(value) &&
              Array.length > 0 &&
              value.every(item => validator.isIn(item, ['USD', 'EUR', 'GBP']))
  }
}))

app.post('/', (req, res, next) => {
  req.check('test', 'Invalid').isX()

  req.getValidationResult()
     .then(r => !r.isEmpty() ? r.throw() : res.json({ code: 'OK' }))
     .catch(r => next(r.array()))
})
...

This would be much easier if #234 gets merged.

#234 needs some love in order to be mergeable.
Remember that anyone can take over that branch... and send another PR 😉

Hello folks, v4.0.0 has been released, bringing with it support for validating multiple fields with the same rules!
Please check the README for complete docs, and also check the upgrade guide.


uritemplate version incompatibilities

Noticed by installing a generated SDK in a project with more dependencies I get this error message:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behavior is the source of the following dependency conflicts.
google-api-python-client 1.12.11 requires uritemplate<4dev,>=3.0.0, but you have uritemplate 4.1.1 which is incompatible.
kfp 1.8.19 requires uritemplate<4,>=3.0.1, but you have uritemplate 4.1.1 which is incompatible.

Given the number of issues we are facing with uritemplate libraries across implementations, I'm wondering if it makes sense to roll our own for kiota to be more independent ... cc. @baywet

Thanks for reporting this @andreaTP as always.
I'm fairly novice to Python and currently on my phone.
Are those dependencies that are transiently pulling Uri template dependencies of the kiota packages or of your project?
Is there such a thing as a peer (npm) or internal dependency (implementation with gradle) with pip?

@baywet sorry for the delay:

Are those dependencies that are transiently pulling Uri template dependencies of the kiota packages or of your project?

They are transitive dependencies within a Data Science Jupyter notebook, they come from "my project", or better, from my environment.

Is there such a thing as a peer (npm) or internal dependency (implementation with gradle) with pip?

I quickly looked up this, but I cannot find anything relevant (not a Python deps expert too though).

@samwelkanda could you provide more context on dependency management for Python here please?

Closed by #118


'utf8' codec can't decode byte 0xd5 in position 0: invalid continuation byte

$ disco jobdict  dispipe@54b:37485:6c06
dispipe@54b:37485:6c06
'utf8' codec can't decode byte 0xd5 in position 0: invalid continuation byte

Could you show me your code?I meet this fault
'utf-8' codec can't decode bytes in position 138-139: invalid continuation byte
Thanks.


Better support for images in different bundles

I have some IBDesignable classes that rely on images. Unfortunately using UIImage(named:) doesn't work in Interface Builder, as Interface Builder creates separate bundles for each class which you're supposed to use when creating images. For example, I have the following code:
let bundle = NSBundle(forClass: self.dynamicType)
let image = UIImage(named: "chevronAccessory", inBundle: bundle, compatibleWithTraitCollection: nil)
return UIImageView(image: image)

"chevronAccessory" is an image that exists in my asset catalog, so it would be nice if I could use R.swift to get the name.
Perhaps with the following format:
R.imageName.chevronAccessory
or
R.image.name.chevronAccessory
I'm not sure how typical this use case is, so it might not be worth decreasing the usability of images, but if you wanted to do so you could make images into a nested struct like nib. So then you could have:
R.image.chevronAccessory.instance
R.image.chevronAccessory.insantiateWithBundle(bundle)

Thanks for bringing this to my attention. The bundle "problem" is something I'm aware of, didn't run into it myself yet so that's the reason I didn't fix it.
At least the imageName solution would be something that I could implement easily, but I hope I can create an even nicer solution. It's not at the top of my Todo list, but I'll look into it.
If you're reading this and want this also fixed, please comment with a +1 to this issue.

Alright, took a little peek into this. But I'm wondering what good example use cases are, maybe extensions of designables, but me or someone else should create some good example cases to see what is the best way to implement this.

+1
I have a pretty large project with several targets for different versions of the App. I also have a target CustomGUI that builds as a Framework in which some custom views are implemented that are used by all/some of the other targets. The CustomGUI also needs some images which are shipped in an assets catalog of the CustomGUI target.
In order to use these Images, I need to load them from the correct bundle like this:
UIImage(named: "someImage", inBundle: NSBundle(identifier: "com.myCompany.customGUI"), compatibleWithTraitCollection: nil)

Thanks for the concrete use case! I'll look into it. :)
Any proposal of how you would use R.swift?

Initially, I would have expected to be able to the image name somehow like @nmccann originally suggested.
I agree on this:

I'm not sure how typical this use case is, so it might not be worth decreasing the usability of images

Maybe, one solution would be to have an imageResource
protocol ImageResource {
  var instance: UIImage? { get }
  var name: String { get }
}

struct _R {
    ...
    struct image {
        struct _testImage: ImageResource {
            var instance: UIImage? { return UIImage(named: "image.jpg") }
            var name: String { return "image.jpg" }
        }
    }
    ...

and have both:
struct R {
    ...
    struct imageResource {
        static var testImage: ImageResource { return _R.image._testImage() }
    }

    struct image {
        static var testImage: UIImage? { return _R.image._testImage().instance }
    }
    ...
}

Not sure if this is the best approach though...

Yeah, interesting approach. Could work.
I'm now thinking; If R.swift would know the bundle the image is in we could always load based on the UIImage(named: inBundle: compatibleWithTraitCollection:) constructor. Then it would always work, in depended from the bundle you're calling the R.image.someIcon from.

Yeah that would be nice.
I have been using NSBundle(forClass: self.dynamicType) previously for that which might work here.
But for some reason, this has failed me today (unrelated, don't know why) so I had to use NSBundle(identifier: "com.myCompany.customGUI")

By the way, in my scenario I have several R.generated.swift Files - one for each Target.

One R-file per target is the correct approach when working with a setup like yours.
If I may ask; Am I correct that this problem does occur when you try to access an image that is in the app target from code that is in your library (using the R-struct from the app)? Just to be sure that I create the correct testcase when implementing this.
Maybe you could show me the line of code that fails and a little bit of it's context.

struct R {
    ...
    struct imageResource {
        static var testImage: ImageResource { return _R.imageResource._testImage() }
    }

    struct image {
        static var testImage: UIImage? { return _R.imageResource._testImage().instance }
    }
    ...
}

Current example do not take in account that 2 bundles can have an image with the same name. Maybe as an option we can target bundle first and image resource after:
R.bundle.<Bundle_name>.image.<Image_name>

And as a shortcut target main bundle as a default
R.image.<Image_name>

The same patter can be used for loading xibs, etc from none-main bundle.

Maybe you could show me the line of code that fails and a little bit of it's context?

I can try to create a small sample project representing my use case later today or maybe tomorrow.

Oh sorry, forgot about the sample project. Today I discovered the need to specify the bundle when for UINib.instance as well, so I was reminded of that: https://github.com/iv-mexx/r.swift-example

Tnx! I'll take a look. :)


Retry sending a notification if you're being rate limited by Discord

Hi, all,
First of all great package, thank you!
I'm trying to channel all notifications in my application to our staff discord's channel. Sometimes we send many notifications at once to various text channels we end up being rate limited.
I saw a pull request that returns the underlying GuzzleHttp exception inCouldNotSendNotification. Our notifications are queued and sent "asynchronously", so I was thinking of releasing the job again after the seconds spcified in the response by Discord.
This is my code in theory, because I don't know how to test it without making real requests to Discord (test it in ci/cd pipelines):
public function failed(Exception $exception)
{
    if ($exception instanceof CouldNotSendNotification) {
        if ($exception->getCode() === 429) {
            $json = json_decode($exception->getPrevious()->getResponse());
            $this->release($json->retry_after);
        }
    }
}

Could you please advise on how to proceed?
Thank you,
Ivanka

I have overridden the DiscordChannel class and added the Laravel retry() helper to retry the sending if it fails with a 429 status.
This works for me - am not sure if it's the bullet-proof solution, primary with the getHeader() logic - but in theory it should be correct.
return retry(
    times: 2,
    callback: fn () => $this->discord->send($channel, [
        'content' => $message->body,
        'embed' => $message->embed,
    ]),
    sleepMilliseconds: function (int $attempts, CouldNotSendNotification $exception): int {
        $previous = $exception->getPrevious();

        if ($previous instanceof ClientException && $previous->hasResponse()) {
            return ($previous->getResponse()->getHeader('x-ratelimit-reset-after') ?: 1) * 1000;
        }

        return 1000;
    },
    when: fn (Exception $exception) => $exception instanceof CouldNotSendNotification
        && $exception->getCode() === Response::HTTP_TOO_MANY_REQUESTS
);


tooltip send/exchange/deposit

Pull request template
Checklist

[x] I have read the CONTRIBUTING guide
[x] branch rebased on develop
[ ] styleguide check npx eslint shared/**/*.js
[x] good naming
[x] keep simple
[ ] video or screenshot attached
[x] testing instruction attached
[ ] check your PR once again

Description

Video or screenshot

What and how to test

https://youtu.be/JOwhiPkY6JE

еще не готово

готово

https://drive.google.com/file/d/1MAP_RpFOir-n6sR8Hp6EYcIlu8ZU25cy/view

https://drive.google.com/file/d/1nJkJIOxk-SbZFoZ-VhYHVSZoH5K2gOcd/view


Update README

What was changed

Added note about run and debug view

Why?

Checklist

Closes

How was this tested:

Any docs updates needed?

Closing in favor of https://github.com/temporalio/vscode-debugger-plugin/pull/30


python310Packages.google-cloud-container: 2.17.4 -> 2.19.0

Changelog: https://github.com/googleapis/python-container/blob/v2.19.0/CHANGELOG.md
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 225613 run on x86_64-linux 1

1 package built:

python310Packages.google-cloud-container


openapi3 not working when  running via ssh connected server

I can run in testing api based openapi3 sechme  in local pc.(openapi server is local docker container)
But, not working when running via  ssh connected server.(openapi server is aws container)
bellow err log and runn book

runn book

desc: openapi3 test in jump server
runners:
  sc:
    host:  xx-xx-xx
    keepSession: true
    localForward: '10443:xxxx-xxxx.execute-api.xxxxxx.vpce.amazonaws.com:443'
  req:
    endpoint: https://<IP_ADDRESS>:10443/xxxx
    openapi3: ../../../openapi3.yaml
    skipVerify: true
debug: true
steps:
  openapi_test:
    desc: Create mission input
    req:
      /xxx/xxx/xx/xxxx_xxx/xxxx:
        get:
          headers:
            accept: application/json
            Content-Type: application/json
            Host: xxxxxxxx
            X-API-KEY: xxxxxxx
    test: steps.openapi_test.res.status == 200

err log

  Failure/Error: http request failed on 'openapi3 test in jump server'.steps.openapi_test: no matching operation was found

not working when running via ssh connected server

To identify the cause,
First,
Check if a direct HTTP request is not the problem.
From the error message (no matching operation was found), it seems that the request or response does not conform to the OpenAPI Spec.

Check if a direct HTTP request is not the problem.
Curl commad request check is success.
Also, when I comment out  openapi3: ../../../openapi3.yaml , test success.
On the other hand, I think importance,   when I not comment out  openapi3: ../../../openapi3.yaml , test bellow normal err.

Failure/Error: http request failed on 'openapi test in local '.steps.openapi-test: openapi3 validation error: response body doesn't match schema #/components/schemas/xxxxxxxResponse: value must be an object

Why, only the test via ssh is displayed like bellow.

Failure/Error: http request failed on 'openapi3 test in jump server'.steps.openapi_test: no matching operation was found

This difference is why I think that test via ssh don't read openapi.yaml or not working.
What would you think?

Curl commad request check is success.

To identify the cause,
First,
Check if a direct HTTP request is not the problem.

Try it with runn. cURL does not have the feature to check OpenAPI Spec.

This difference is why I think that test via ssh don't read openapi.yaml or not working.

At this time, we do not know if the problem is via SSH or OpenAPI functionality, as we have little information and cannot reproduce it at hand.

Curl commad request check is success.

To identify the cause,
First,
Check if a direct HTTP request is not the problem.

Try it with runn. cURL does not have the feature to check OpenAPI Spec.

When I execute runn with commet out openapi3: ../../../openapi3.yaml , I have  no check err.
For instance bellow.
desc: openapi3 test in jump server
runners:
  sc:
    host:  xx-xx-xx
    keepSession: true
    localForward: '10443:xxxx-xxxx.execute-api.xxxxxx.vpce.amazonaws.com:443'
  req:
    endpoint: https://<IP_ADDRESS>:10443/xxxx
    # openapi3: ../../../openapi3.yaml
    skipVerify: true
debug: true
steps:
  openapi_test:
    desc: Create mission input
    req:
      /xxx/xxx/xx/xxxx_xxx/xxxx:
        get:
          headers:
            accept: application/json
            Content-Type: application/json
            Host: xxxxxxxx
            X-API-KEY: xxxxxxx
    test: steps.openapi_test.res.status == 200

On the other hand, in local,  (openapi server launch in local)
when comment out openapi3: ../../../openapi3.yaml  , I have no check err.
For instance bellow.

local runn book yaml

desc: openapi3 test in local
runners:
  req:
    endpoint: http://localhost:3000
    # openapi3: ../../../openapi3.yaml
debug: true
steps:
  openapi_test:
    desc: xxxx xxxx xxxx
    req:
      /xxx/xxx/xx/xxxx_xxx/xxxx:
        get:
          headers:
            accept: application/json
    test: steps.openapi_test.res.status == 200

When no comment out openapi3: ../../../openapi3.yaml , I have bellow err.

err log

Failure/Error: http request failed on 'openapi test in local '.steps.openapi-test: openapi3 validation error: response body doesn't match schema #/components/schemas/xxxxxxxResponse: value must be an object

In local, I have openapi3 validation error: response body doesn't match scheme error.
Via jump server, I have  no matching operation was found.
I think local err message normal, but jump server err message illeguluer, becuase of resuponse is imperfect for no my implementing response(for now, open api generator default return value do some magic!)
So, err contents must be scheme varidation err when test via jump server.

Thank you for new release. I grad to appreciate your support.
However, this issue cannot be closed.
Still I encounter this probrem

I'm afraid I can't reproduce the error you're seeing.
https://github.com/k1LoW/runn/blob/65dbb607155014da65b7665316f9e4f384136713/testdata/book/sshd_local_forward_with_openapi3.yml#L1-L21

no matching operation was found

Are the requests and responses as you intended at this time? Is it the same as for local?

I made the error message friendly in v0.75.2. What errors do you get now? Are the HTTP requests ( Method, Path ) as you intended?

Thank you for debug release.
bellow err log.

local errr message(nomarl)

 Failure/Error: include failed on 'xxx xxx xxx'.steps.xxx_xxx_Xxx: failed to run xxx/xxx/xxx/xxx/local/yamls/200/normal.yaml: http request failed on 'xxxx xxx xx xxx'.steps.normal: openapi3 validation error: response body doesn't match schema #/xxx/xxxxx/xxxxxResponse: value must be an object
  Schema:
    {
       ${{ response json }}
    }

  Value:
    "do some magic!"

  -----START HTTP REQUEST-----
  GET /xx/xx/xx/xxx/xxx HTTP/1.1
  Host: localhost:3000
  Accept: application/json

  -----END HTTP REQUEST-----
  -----START HTTP RESPONSE-----
  HTTP/1.1 200 OK
  Connection: close
  Content-Length: 17
  Content-Type: application/json
  Date: Sun, 11 Jun 2023 07:42:03 GMT
  Server: Werkzeug/2.2.3 Python/3.10.5

  "do some magic!"

  -----END HTTP RESPONSE-----

1 scenario, 0 skipped, 1 failure

via jump sever err log  (illegular)

Failure/Error: include failed on 'xxx xxx xx test'.steps.xxx_xxx_xxx: failed to run xxx/xxx/xxx/aws/200/normal.yaml: http request failed on 'xx xx xxxx'.steps.normal: failed to find route: no matching operation was found (GET xxx/xxxx/xxxx)

GET /xx/xx/xx/xxx/xxx HTTP/1.1

1.GET /xx/xx/xx/xxx/xxx

no matching operation was found (GET xxx/xxxx/xxxx)

2.GET xxx/xxxx/xxxx
Can we assume that there is no difference between these two values?

@t-kitamura-axel
Hi!
I felt that the following response seems to me to be an incorrect definition of openAPI.
  -----START HTTP RESPONSE-----
  HTTP/1.1 200 OK
  Connection: close
  Content-Length: 17
  Content-Type: application/json
  Date: Sun, 11 Jun 2023 07:42:03 GMT
  Server: Werkzeug/2.2.3 Python/3.10.5

  "do some magic!"

  -----END HTTP RESPONSE-----

I think the Content-Type is text/plain, not application/json
I think that is why you are getting the following error

openapi3 validation error: response body doesn't match schema #/xxx/xxxxx/xxxxxResponse: value must be an object

I change  application/json  with text/plain.
But, no difference.

GET /xx/xx/xx/xxx/xxx HTTP/1.1

1.GET /xx/xx/xx/xxx/xxx

no matching operation was found (GET xxx/xxxx/xxxx)

2.GET xxx/xxxx/xxxx
Can we assume that there is no difference between these two values?

I see, no difference.
Just a typo.

I change  application/json with text/plain.

Along with the API implementation, the definition of openapi3.yaml itself needs to be modified.
If the openapi3.yaml has been modified, the error messages in the debug output should change.

What do you mean?
One of the things I'm having issues with is that the local output and the output over ssh are different.

Or when ssh port forwarding, Should open api yaml's server section  be replaced?

One of the things I'm having issues with is that the local output and the output over ssh are different.

Are both environments in which you are running the runn command local?

Or when ssh port forwarding, should open api yaml's server section be replaced?

It is assumed that the program is running locally, but instead of having it port forwarding, it changes the endpoint specification.
  req:
    endpoint: ${APP_URL:-https://<IP_ADDRESS>}/xxxx
    openapi3: ../../../openapi3.yaml

It can be switched by env as above.
$ APP_URL= https://xxxxxx.amazonaws.com runn run path/to.yaml

but instead of having it port forwarding, it changes the endpoint specification.

It looks like he wants to execute HTTP requests via SSH port forwarding.

His environment seems to be as follows.
graph LR
    Local[Local] -- can SSH --o xx-xx-xx[xx-xx-xx]
    xx-xx-xx -- can do HTTP requests --o xxxx-xxxx.execute-api.xxxxxx.vpce.amazonaws.com[xxxx-xxxx.execute-api.xxxxxx.vpce.amazonaws.com]
    Local[Local] -- can NOT do HTTP requests  --x xxxx-xxxx.execute-api.xxxxxx.vpce.amazonaws.com[xxxx-xxxx.execute-api.xxxxxx.vpce.amazonaws.com]


Triage Previous Versions styling regression in Theme 1.11

From http://qiskit.org/documentation

I wonder if this is due to config not being set up how we expected.

Neither @javabster nor I can reproduce this now...

It's possible this was bad browser cache state?


X500NameTokenizer does not parse certificates containing DN with escaped quotes

I guess, due to this code https://github.com/bcgit/bc-java/blob/8ca7bbd856c4077af0150070a9191d47c63f964e/core/src/main/java/org/bouncycastle/asn1/x500/style/X500NameTokenizer.java#L61 a certificate with a DN containing escaped quotes can't be parsed correctly.
Example DN (part of it): Wohnungsbaugenossenschaft "Humboldt-Universität" eG
openssl output: Subject: C = DE, L = Berlin, O = Wohnungsbaugenossenschaft "Humboldt-Universit\C3\A4t" eG, CN = transfer.wbg-hub.de
Have a look at the certificate of this website to see an example: https://transfer.wbg-hub.de/intro.xhtml ( transfer-wbg-hub-de.zip )
I can't find a RFC reference which does not allow quotes in the DN.

C=DE,L=Berlin,O=Wohnungsbaugenossenschaft "Humboldt-Universität" eG,CN=transfer.wbg-hub.de
will parse. This is in accordance with https://www.rfc-editor.org/rfc/rfc1779.html

Appears dead.


Support for python 3.10

Please:

[ ] Check for duplicate requests.
[ ] Describe your goal, and if possible provide a code snippet with a motivating example.
Added support for python latest version 3.10

Duplicate of #8097


Possible 0.2.1 regression in replacement strings

This used to work:
assert_eq!(Regex::new("(.)").unwrap().replace_all("a", "$1_"), "a_" );
But now produces "" instead of "a_".
It would appear that underscores somehow cause the replacement to be deleted instead of added.
Named captures don't fix it:
assert_eq!(Regex::new("(?P<char>.)").unwrap().replace_all("a", "$char_"), "a_" ); //actual: ""
But inserting a space between the capture and underscore works!
assert_eq!(Regex::new("(.)").unwrap().replace_all("a", "$1 _"), "a _" );
assert_eq!(Regex::new("(?P<char>.)").unwrap().replace_all("a", "$char _"), "a _" );
As does using curly braces around the identifier (the only apparent workaround)
assert_eq!(Regex::new("(?P<char>.)").unwrap().replace_all("a", "${char}_"), "a_" );

This was an intended but undocumented breaking change. Note the docs for expand say:

name may be an integer corresponding to the index of the capture group (counted by order of opening parenthesis where 0 is the entire match) or it can be a name (consisting of letters, digits or underscores) corresponding to a named capture group.

and also

The longest possible name is used. e.g., $1a looks up the capture group named 1a and not the capture group at index 1. To exert more precise control over the name, use braces, e.g., ${1}a.

So this means that the reason why you're getting the behavior you see is that 1_ is used as the capture group name, and since that doesn't refer to any valid capture, it is replaced with the empty string. Indeed, the fix is to use curly braces.

Ah. My assumption was that the replacement string was parsed in the context of the regex, and that non-existent captures would fail to expand instead.
It'd probably be good to emphasize this, and stick to the curly brace syntax for examples. I'm not aware of other regex implementations with this behavior, thus my confusion :)

@nathanaeljones Go's regexp engine has precisely the same semantics. But yes, better docs here would be great.


UI Indication when data has been truncated > 10 limit

How do you use Sentry?
Sentry Saas (sentry.io)
Version
latest
Steps to Reproduce
in django create a log entry with extra data containing a list with 20 entriies.
Expected Result
I know there is a 10 entry limit.
When I view the UI for the issue and go to extra data I should see the list with 10 entries (due to the limit).
But there should be an indication that this data was truncated. Even better to indicate how much data was truncated. Eg. 50 entries for a list or dict.
Actual Result
Sentry just shows a list with 10 entries. The list looks complete. This is very confusing to the debugger as there is no indication that list entries were truncated.

thanks @pg1671 we already have several issues on the product side tracking this problem, so I will add a link to this there and close this one.
https://github.com/getsentry/sentry/issues/68426