Implement WrapErr for Option

I've noticed, that unlike anyhow's Context trait, miette's WrapErr trait doesn't seem to be implemented for Option. Because of this, I sometimes have to do something like this, whenever I want to get a miette error when an option type is none:
some_option.ok_or_else(|| miette!("Option was none!"))?;

But I think it should be possible to implement WrapErr for Option, in order to allow something like this:
some_option.wrap_err("Option was none!")?;

That would make my code a lot cleaner!

Closing this since it's as intended :)


We have weapons!

Added a debug projectile for the debug weapon
Added a drags variable to FObject so that there isn't a drag on everything (like laser beams)
Weapon automatically fires in character direction (left or right) by pressing space

Todo / bugs:

The weapon never seems to reload when the ammo is gone, or it goes too slowly.
The projectile is supposed to not have any gravity (this is set in the FProjectile constructor) but still moves down a bit
We should destroy projectiles and remove them from the m_projectiles vector when they have collided with something.
We need a good system for creating different projectiles from different weapons, we can't hard-code everything in the character loop in FEngine (line 23 and down) - it will get far too messy. We could do it with a projectilesfolder where we have a lot of classes inherited fromFProjectile`, or some kind of database system. It would be nice to separate parameters for different things away from the engine.

Any thoughts here or anything you want to do @dekespo ?

I like the bug (the picture :D )
I am working on ammo problem, check the new PR
Will check other bugs later on.


OpenAPI spec for the REST API

Context
An organization that I'm working with is considering adopting Backstage as their developer portal. One of the hard(ish) requirements that they have is that all services that we consume provide an API specification. This requirement exists because the infrastructure team needs to automate communication with services. If a specification is available, then they can use code generators to generate API clients for these services.
Feature Suggestion
It would be very helpful if there was a way to ensure that services integrated into Backstage come automatically with specifications. .NET, Rails & FastAPI all provide a mechanism that automatically generates an OpenAPI definition from route definition. I'm not aware of any tools that work like this in Node.js world.
Possible Implementation
express-openapi is a middleware that generates some specification from routes. It doesn't produce complete specifications - I don't even know if this is possible.
This request could also be satisfied by a GraphQL API that wraps services that Backstages consumes. The only consideration is making sure that the design of the APIs covers plugin services as much as possible.

I ran across this project the other day that could be of some use when using a code first approach: https://github.com/bee-travels/openapi-comment-parser.
However, I agree that using annotations or a middleware would be more preferable.

Does it have to be automated from the beginning, or could we start with a manually edited file?

It doesn't have to be automated. Having an OpenAPI spec for services included out of the box could be a good start.

@taras wanna take a stab at it?

I wouldn't mind but we need a way to ensure that they say current. Any ideas?

I think it looks like express-openapi indeed produces a complete specification, although you will have to write the spec for each route manually. The upside to that vs a completely manual spec on the side is that you get the spec closer to the implementation. You may also get validation middlewares for your routes (both for request and response data) amongst others, if desired.. ;)

I tried out https://github.com/bee-travels/openapi-comment-parser recently and it will generate the openapi spec through JSDoc comments. It's still manual, but the spec ends up being right right above the code. I haven't tried it, but I suspect you could use some of the existing openapi tools like validation in conjunction with this approach.

looked a bit more at express-openapi, and I must say that I was stumped to learn that you had to put your implementation along with the spec.. I don't fancy the idea that the spec drives/dictates the implementation, it should be the other way around (the spec reflects the registered routes)

@kaos I agree - spec and implementation should be separate.
We've been experimenting with this inside of our Backstage instance for integrating with our platform. We've been able to decouple the frontend from backend development by using OpenAPI spec in the middle. We generate a client with autorest and use Prism to mock service while in development. We're going to put it through the grind internally and then make an RFC when we're happy with the workflow.

Thanks for the update, looking forward to it :)

Just bumping this to say that there's definite interest in this still! :)

Hey @taras,

We're going to put it through the grind internally and then make an RFC when we're happy with the workflow.

Just curious if there is anything you'd like to share on your experience so far on this setup that you mention!

@OrkoHunter yep, the setup that we have right now is this,

each service that we're consuming in Backstage has an OpenAPI spec
we generate clients for these services with autorest
the generated code is wrapped in a use*API hook which uses the Backstage's ref API
we use the use*API hooks in Backstage
we use Prism to create a mock server in development - this allow us to build UI before backend microservice is implemented

This is pretty spiky at the moment because Autorest is quite heavy. We could create a leaner generator that generates hooks automatically and eliminate glue. Prism is also not exactly what we want to because it allows mocking by matching requests but not a high fidelity simulation. There is an issue to make Prism more realistic.
I would like to see and contribute to is having OpenAPI spec for each service in Backstage. Then generate clients wether at built time or runtime from the OpenAPI spec.

@taras I'm very interested in seeing how you generated the OpenAPI spec for the catalog API, do you mind sharing your implementation somewhere? Can you push up a branch showcasing what you've built so far? Our end goal is to be able to build a catalog-client from the OpenAPI spec for the /api/catalog endpoint. Thanks!

👋 @alexcurtin we haven't integrated with the catalog so far. We only used the clients that we created to integrate services that already have OpenAPI specs into Backstage. I might have an opportunity to do it this week, but I'm trying to figure out how to validate the implementation.

I'm looking at using json-schema-to-openapi-schema to convert at least a part of https://github.com/backstage/backstage/blob/master/packages/catalog-model/src/schema/kinds/API.v1alpha1.schema.json to YAML. From there, I'm going to try to create an OpenAPI spect that  includes the routes and references the generated files.

Nice. Yeah if we could reduce the duplication of defining the entity shape it'd be nice. But why API? I'd have expected Entity instead, since that's the actual return type. Unless you wanna get fancy and do oneOf each of the specific kinds, but the value of that is probably very limited especially since it won't account for custom kinds etc

But why API? I'd have expected Entity instead, since that's the actual return type.

You're right. I referenced the wrong file. It was the last file opened in my tab instead of the proper file :) https://github.com/backstage/backstage/blob/master/packages/catalog-model/src/schema/Entity.schema.json is the right file.

@freben I started creating a @backstage/openapi package. I'm going to put all of the generated YAML files in there. Does that sound alright?

Uh, I am not sure. @Rugvip @benjdlambert @jhaals any input on that?
Gut feeling says that the catalog API, for one, is a thing of its own and should go in the @backstage/catalog-client package with the other client functionality. And that an overall mega-api-definition for a backend, rather than individual plugins, might be of limited use.

Regarding the linked ticket about refs, input is welcome. I have run into annoying situations where ajv and other consumers treat refs in (sometimes subtly) different ways. Some like paths, some like identifiers, ...
We could update the ref usage but it would have to fit at least ajv and openapi at the same time.

We could update the ref usage but it would have to fit at least ajv and openapi at the same time.

I need to customize dereferencing to support identifiers. json-schema-to-openapi-schema also needs to be changed to support draft7. The rest of the conversion mechanism is quite simple so I might end up rewritinging the whole library.

And that an overall mega-api-definition for a backend, rather than individual plugins, might be of limited use.

Yeah, this doesn't seem great. I'm in doing this in part to see if I can use the OpenAPI spec to generate a frontend client. We also want to make sure that anytime that schema changes, we regenerate the OpenAPI spec.

Could it be useful to hand craft the api schema first, and add an integration test that builds a js client from the schema and performs calls against a running backend? :) That way we could at least reject the PR based on forgetting to update definitions - kinda like the api report stuff, one might say.
But yeah if you are far along towards solving schema generation from code, then sure that's even more convenient if it's a stable method.

Could it be useful to hand craft the api schema first, and add an integration test that builds a js client from the schema and performs calls against a running backend?

That was my original plan, but I didn't know how to validate my hand crafted OpenAPI spec, that's why I went down the path of generating from schema file. I started researching how I could generate a client and I came across something that might be interesting.
Have you seen Optic? It monitors test traffic to identify endpoints. It can generate OpenAPI spec from information that's entered in their UI. It can on PRs to show any changes to those contracts.
I'm going to give it a spin over the weekend.

Have you seen Optic? It monitors test traffic to identify endpoints. It can generate OpenAPI spec from information that's entered in their UI. It can on PRs to show any changes to those contracts.

👋 Optic maintainer here -- this seems really useful and I'm happy to help out @taras the community with some docs or guidelines for using Optic to document each plugin.

@acunniffe 👋 I just got optic setup to start the backend and work as a proxy. It did exactly what I need it to - immediately.

@freben I want to make sure that I understand what would be beneficial at this stage. I'm going to describe some user stories that we want to be able to support.

As a developer building a portal with Backstage, I want to see the documentation for REST APIs provided by Backstage core.
As a contributor to a Backstage project, I want to verify that I did not change the implementation of a backend plugin in a way that breaks a documented API contract.
As a developer building a portal with Backstage, I want to be able to easily integrate an existing OpenAPI documented service into Backstage with all of the necessary TypeScript/React Hooks glue.

Did I get these right? Are there other use cases that I'm missing?
For #1, as long as we have an OpenAPI spec that is current and accurate, we can use it to generate documentation.
For #2, we need some automated tests that make calls from the browser to the backend to ensure that the client and backend stay in sync - the contract can be verified by observing these tests
For #3, this needs to be created - right?

The main use case I had in mind was

As an engineer at our company, I want to make an integration with the Backstage catalog from other non-Backstage services written in other languages, and would like to automatically generate clients for it

This is a HUGE use case for us. The catalog powers, or at least informs, so much more than just Backstage itself.
And as a result of that, and to get a good dogfooding going, I was considering exploring whether catalog-client indeed also could make sense to auto generate. As well as exploring generating express stubs for the server side from it - in a schema-first fashion.
But I don't know the viability of all that yet

I don't know either, but I want to figure out how to make it happen.
@freben how attached are you to the JSON Schema implementation that we have today? If there was another way to perform that validation, would you be open to it? I just want to understand what can be changed.

I've been keeping my eye on https://github.com/aws/jsii/ for a while. This could be used to generate clients in different languages.
Ok, I'm thinking that the first step could be to write a few e2e tests that use the catalog-client to interact with the catalog backend. We can use these requests to verify payloads that are sent to the backend API. Then use a proxy to inject a verification mechanism between the client and backend. The proxy will verify that the requests and responses conform to the spec.
Once we have this, then we can figure out how to proceed. How does this sound?

Hi guys. We are currently at the early stage of the Backstage implementation. And any such OpenAPI definitions and/or swagger help page would be really helpfull for us to see, what kind of API backstage and its plugins provide! Any news regarding this?

Sorry, no news. It's just about having the time to soldier through making it - and our priorities have been elsewhere while moving everything toward GA release.
We're more than happy to help out with specifics on discord or elsewhere, and in a pinch you sometimes can glean details by looking at the source for the exposed routers of the plugins.

So we can probably use something like https://dev.to/kabartolo/how-to-document-an-express-api-with-swagger-ui-and-jsdoc-50do to autogenerate a nice documentation site. Just a suggestion.

I wonder if we could get super meta, and use the API Entity and automatically add Backstage components to the backstage namespace so you can see these swagger docs in Backstage.

Any news on this front?

I would love to help out on this as I'm trying to familiarize with the codebase. However, would like to know more on the direction on how to document OpenAPI spec.

@PhakornKiong I, unfortunately, haven't been able to work on this. If you'd like, we can talk in Discord and I can tell you what I know.

https://github.com/backstage/backstage/issues/7162 might be of interest.

Since the Backstage is GA...is this something that can be improved now? O:-)

@jvilimek We're currently doing some work that might touch on this, but not entirely sure where we'll end up or if we'll cover it. I'd say we're looking for ways were it can be more than just a decoupled schema only living in git.

Any update on this?

No. It'll remain a bit down the prio queue for some time, from the maintainer's point of view at least. If somebody comes up with a killer plan and implementation of how to achieve this, of course we and others will be very grateful.

As the person who wrote the original issue, I want to update my view on this request.
I still think it's very valuable, but I haven't done anything to address this issue because I don't want to add yet another file that needs to be maintained in addition to @backstage/catalog-client. The process of extending the @backstage/catalog-client is already quite extensive as you can see by #14354 and #13583. Adding a spec file to requirements would just make it even more difficult.
With that said, I believe we could have best of both worlds: an OpenAPI spec and an up to date client. We can accomplish this by generating the @backstage/catalog-client from an OpenAPI spec. There are a few different tools for doing this, but autorest is one of the most popular options. It's a Node.js based CLI tool that generates clients in many languages.
We could create a template similar to @autorest/typescript that would have Backstage specific client setup. This would make generating clients for Backstage projects very easy.
Autorest is written in Node.js, so we could even include this template as part of our @backstage/cli package.
This is obviously a lot of work. That's part of the reason why I haven't attempted to do it. If there are people who're interested in making it easier to integrate Backstage projects with external APIs, then I'd be happy to work with you to figure out how we can make this happen.

Hey all,
Wanted to chime in, I came across the tsoa package (https://github.com/lukeautry/tsoa) and did an initial PoC, https://github.com/sennyeya/backstage/blob/fb92c797e209aaac0badf4a1165390249d98b754/plugins/catalog-backend/src/swagger.json. Feel free to checkout the branch and poke around, the example-app should be working.
I think this would be a good first step to auto generate the OpenAPI documentation for the catalog. It has express integrations, but doesn't require full buy in. Each plugin can adopt it on their own using the RegisterRoutes method. The compilation is a little weird as it requires decorators and a specific file glob match pattern (https://tsoa-community.github.io/docs/getting-started.html). It also uses a more nest.js/Spring declarative pattern that may not be what we're looking for.
Curious to hear thoughts?

@taras at plugins/bitbucket-cloud-common we already use @openapitools/openapi-generator-cli to generate code from an OpenAPI specification.

@pjungermann I'm not sure I'm following, for #1. How would we capture business logic like pagination and validation with just API specifications? Wouldn't we still need code to capture this? ie something like openapi-comment-parser or express-openapi. In which case, we would be maintaining both anyways.
Once we have an API specification, we can use autorest or @openapitools/openapi-generator-cli to generate the clients. The PoC was more around looking at @taras's user stories 1 and 2,

As a developer building a portal with Backstage, I want to see the documentation for REST APIs provided by Backstage core.
As a contributor to a Backstage project, I want to verify that I did not change the implementation of a backend plugin in a way that breaks a documented API contract.

tsoa may not be a good fit here, but it would give an initial level of verification.

Re: plugins understanding their base paths: Makes sense. the plugin routers might have enough context to pull their subroutes, but it's something I'd have to take another look at.
I definitely agree, tsoa's use of IoC is not great, but I think it would be required to access Backstage service's or the database in tsoa's framework. I'm not advocating full buy in, it's just the best solution I've seen that would auto generate and enforce a spec.
Apologies if I'm missing something, I'm late to this thread and pretty new to the API spec/generated client world.

@Rugvip Hmm, I guess I'm still playing catchup. Here's my understanding at the moment, please correct me if/where I'm wrong,
I took a look at the existing OpenAPI express packages, there seem to be a few camps, schema-first or code-first. The schema-first ones require a full schema with endpoints and operations to do validation, https://www.npmjs.com/package/express-openapi-validator. They generally rely on operationIds being defined in the schema to define implementation. Code-first packages, like tsoa, generate either code and spec or just spec from typescript code or comments. There are also hybrid options, like express-openapi which supports both static file and a NextJS-like file structure.  The only major difference I see (and I could be off base) is that the process to generate the OpenAPI spec output is manual vs automatic. A framework like tsoa adds additional questions around IoC, but there are other options for code-first.
In which case, it seems to me like schema-first would have a higher cost especially if additional plugins don't use AJV (which at quick glance is most of them) or aren't well versed in OpenAPI. Also while both code-first and schema-first would require ongoing validation in PR pipelines, I would think that code-first would have more certainty that what you write is what you get.
Also, wouldn't a code-first approach like tsoa be more similar to schema-first than using an API spec file?

How would we capture business logic like pagination and validation with just API specifications? Wouldn't we still need code to capture this? ie something like openapi-comment-parser or express-openapi. In which case, we would be maintaining both anyways.

You wouldn't do both.
The specification != implementation. As mentioned by @Rugvip you generate code from the spec, usually interfaces/types/... which then are used at the implementation.
Let's say, you use a pagination with limit and offset parameters, then these would be part of your spec incl. what you consider valid values for these. Your implementation then uses the defined parameters to provide pagination.
You don't need anything like openapi-comment-parser or express-openapi here.

The PoC was more around looking at [...]

Sure. Don't take this as critique about your PoC. We need to look at different approaches I guess. Overall, code-first is an option. It just wouldn't be my preferred option however not sure what is the right approach for this project.

The schema-first ones require a full schema with endpoints and operations to do validation, https://www.npmjs.com/package/express-openapi-validator.

Yes, we want a full schema :) Some of its validation requirements might not be necessary without using this specific package though.

They generally rely on operationIds being defined in the schema to define implementation.

This package seems to be very open which comes with a price. They know about the spec and they know the code and need to connect both. Whether the code was manually written or generated with standard or custom code generator templates and whatever generator was used -- they don't rely on anything. Having operationIds is not something crazy though. Code-to-schema generators have them, too, just auto-generated values which might look a little weird sometimes.

Also, wouldn't a code-first approach like tsoa be more similar to schema-first than using an API spec file?

No, this is just code-first, not schema-first. Starting with the schema (file) is what makes it schema-first. The schema file is also code/language agnostic. And you write and align on the contract before you start to implement it. Overall, I also believe that the quality of the schema will improve with schema-first. How good tooling can close the gaps might depend on the language, too.
The general challenge I see still with schema-first here is the fragmentation through plugins participating at the API which are mounted to a certain path prefix. And it should work to get one spec for a backstage deployment incl. custom plugins, etc.
Maybe something like https://www.npmjs.com/package/openapi-merge / https://www.npmjs.com/package/openapi-merge-cli could help.
Maybe https://www.npmjs.com/package/express-openapi works, too. I didn't look too deep into it. It seems like it can combine separately organized parts, however on a first glance it seemed to entangle spec parts and code a bit too much.
Probably, there are more alternatives.
A first try of openapi-merge-cli seems promising. Needs more exploration and would be just one part of the general plumbing.
Again, not sure if this is the right direction. Certainly, it may be one direction.
I think the concern about internal plugins is valid. The setup should support them.

[...] or aren't well versed in OpenAPI

Not sure if this makes a huge difference for all approaches. It may cause issues with code-first as well.
If we consider a spec per plugin and then combine them in some way, it may even be possible to allow code-first at internal plugins by Backstage users / a mixed approach. If the tooling/setup is good enough, it may not be needed though.
Overall, it seems quite big as pointed out before. Maybe we need a few more of these small PoCs to come up with a good plan.

@sennyeya I just pushed my try of openapi-merge to share how this merging could work
https://github.com/backstage/backstage/commit/0b6cb2f8de1f612ab9ca2f1f2a77cc84012c6d81
As the paths are customizable at each Backstage setup, the openapi-merge config file(s) could be part of the create-app command (initial skeleton and upgrade-diff).
This example is still very basic, however shows how this could work, even including customizations and custom/internal plugins, etc.
As $.paths needs to contain the full path, it is otherwise a bit tricky to break this down by sub-paths / path prefixes which is what is used here.

related to generators:
https://npmtrends.com/@openapitools/openapi-generator-cli-vs-autorest-vs-nswag-vs-openapi-generator-vs-openapi-typescript-codegen-vs-openapi3-generator-vs-swagger-client-vs-swagger-codegen-vs-swagger-ts-codegen

I've looked at a bunch of different generators and based on that, I would say that there are at least 3 different considerations:

Tooling ecosystem around the generator - including the CLI command and how it's configured
Quality of the generated code: type correctness and library API
The templating system

Usually, 2 & 3 are connected, because depending on the templating system, it might not able to generate correct types.
My memory is big fuzzy and all of these tools are similarly named, but if I remember correctly, @openapitools ecosystem was very Java(ish). It generated weird types without return types - this might have changed so we need to check. I've tried a bunch of others, but I'd need to go look through my notes.
I mentioned autorest because it's the most mature in all of these areas.

They allow configuring generation from markdown files. So you can describe the configuration of the generators in an MD files as part of documentation.
Their default library is very mature but it might be too much for what we need.
Their templating system is really solid and flexible, but its a bit more complicated to implement.

If we decided to use autorest, we'd need to create our own TypeScript template based on their base template.

@pjungermann Just finished the static analysis piece which might be the missing gap between openapi-merge and dynamic route names, https://github.com/backstage/backstage/compare/master...sennyeya:backstage:open-api-spec-merge#diff-3c3f29b0bd2c69e389f72465709024dcebf6ffdeb02098c35921de9d2a10a54cR91. The example output in the linked comment is the actual return value from the example backend. There is definitely a better/less error prone way to do this :P, but for now it works on the example app.
My thinking was we could use the base (non-CLI) version of the package to merge and then leave it up to plugins to produce correct specs themselves. I'll take a look at connecting the two today.
Re: generators, I think there's also two camps, client and server generators. Client generators would be something like autorest that create the fetch/request tie ins and types. Server generators would be something like express-openapi that use the schema to validate and fill out the requests.

@taras at plugins/bitbucket-cloud-common/templates/ I was also adding some templates on my own. The use case there was a bit limited though.
I have zero experience with autorest. I guess which one to use needs a good look in itself.

I think there's also two camps, client and server generators.

There are generators which support both sides, too. E.g., the one I've mentioned.
Depending on your project, you might just need one though. As the provider of the API, you would use the server part, however you may also offer a set of generated clients/SDKs in different languages. Or consumers generate them on their end.

Alright, quick update on my side. I generated the default output from the @autorest/typescript package inside of the catalog-client folder and cloned the default template into the repo as packages/client-generator. link.
Looking at the generated code, it seems heavily reliant on the azure tool system. For our internal clients it would make sense to template it to match the system we're already set up, but for other language clients, especially corporate ones, that may cause more fragmentation than anything. I'll have to look at openapi-generator more, but an initial look seems promising, they have a wider array of supported languages and frameworks. The Java buy in isn't great, but I think the handlebars templates would make development and updates a lot easier.
Couple of thoughts on the compatibility of the existing client to be generated,

There's some pretty in depth merging of filter values, I'm not we'd want to capture that in a client, especially if we allow generation that we don't control, ie not through the packages/client-generator, from the OpenAPI spec.
requestOptional, where the return value being 404 isn't an error. This shouldn't be an issue if we are generating plugin specs at run time from config, but I'm not sure how we'd denote it otherwise. OpenAPI has no way to denote optional paths. This may be a case for code-first creation of the schema.
requestIgnored, similar to above, how would we denote this at the spec level?

@sennyeya another alternative to @autorest/typescript can be https://github.com/karlvr/openapi-generator-plus. It uses handlebars for templates and no Java required.

you don't need Java for @openapitools/openapi-generator-cli either; it is written in Java though.

Built-in templates are written in Mustache [...]. Beginning with version 4.0.0, we support experimental Handlebars and user-defined template engines via plugins.

you don't need Java for @openapitools/openapi-generator-cli either; it is written in Java though.

I think you do need Java, custom generators are still stored as jar files and the the download process still downloads the main package as a JAR binary. And the run process is a NestJS app  (which I've never seen as a CLI tool, pretty cool) on top of a java command.

Quick progress update, I looked at 3 sets of things today,

OpenAPI SDK generators beyond autorest and @openapitools/openapi-generator-cli.
OpenAPI express wrappers/stub generators like express-openapi and openapi-backend.
JSON schema generators/validators.

@taras  openapi-generator-plus is interesting and definitely the next best contender of the bunch. One thing that I noticed is that its output seems fairly more verbose than the other two, example, but I like the first class typescript support. It does have a little less client language support than either other tool, only Java/Javascript/Swift. If our primary goal with the SDK generation is to auto generate plugin clients, starting with @backstage/catalog-client, it puts us in a good spot from a tooling perspective. Once we have publishable API specs, we could also just host them in either the repo or the backstage.io site to be pulled and let users create their own SDKs from whichever language/library they choose. This would come with the caveat that not all tools support all OpenAPI features and users would need to be educated on what tools would work with which schemas. example.
Couple of thoughts I've had from looking around:

OpenAPI 3.1 is, in a perfect world, what we're looking for because of out of the box support for JSON schema components. Unfortunately, very few of the tools geared towards OpenAPI support it and their timelines are a ways out.
A server solution like openapi-backend that plays well with express would allow for plugins to incrementally adopt an OpenAPI spec and instantly have request validation. I'll have to look and see how incremental this process would be, if it would allow for incremental route swapping or would be at the plugin level.
The latest JSON schema is also pretty cool, but has very little support. https://github.com/hyperjump-io/json-schema-validator is one of the few libraries that actually implement the feature $id AND $ref. I have a couple of concerns about key overwriting with the existing merge system due to it being completely $ref based. Question related to that:

Is it possible to have a plugin running on a different level of @backstage/catalog-client?
Again, this functionality would drop with OpenAPI 3.1, and that won't be for a while so it's not something to concentrate on.

Thoughts for next directions:

We probably want to (atleast initially) peg to OpenAPI 3.0, OpenAPI 3.1 has been out for a while, but still has pretty minimal support across all of the tooling we would need. No generator SDK libraries support OpenAPI 3.1 yet, openapi-generator has a project to track it https://github.com/orgs/OpenAPITools/projects/4/views/1, but this is primarily for tracking the main processing Java project and not the child handlebars templates. From what I can tell, those are going to be a ways out. Additionally, we want to be careful on dependencies we choose as some of the APIDevTools organization projects are at risk of losing their primary maintainer. Packages like swagger-parser and json-schema-ref-parser are soon to probably be unmaintained. There are alternatives, but very few support the newest version of JSON schema. I found https://github.com/hyperjump-io/json-schema-validator which seems very promising, but I haven't been able to fully test it yet.
We probably want to use openapi-backend or express-openapi. openapi-backend supports 3.1, express-openapi doesn't. Both support validation and operationId based routing. express-openapi it looks like doesn't support schema validation, https://github.com/kogosoftwarellc/open-api/tree/master/packages/openapi-request-validator, just request validation.
For this initial work, we'll probably want to:
a. double publish entity specs to JSON schema and then to OpenAPI using json-schema-to-openapi-schema until OpenAPI 3.1 support exists in relevant packages. Currently this would capped at JSON schema v5, due to lack of support from json-schema-transform or
b. just use OpenAPI component schemas. We lost out on some features, I can push for and do some updates in relevant packages, but I'm out of my element for a lot of this.

Wanna chime in and highlight that we feel that the ability to have type definitions generated from the schema is very important, in particular for the server implementation. The DX around building implementations for the endpoints is very important and not having them be typed is both risky and a big gap in the DX.

What do you folks think about getting together to talk through this in a Discord meeting?

👍, definitely would be interested.

I shared this on Discord but posting here as well. I set up https://doodle.com/meeting/organize/id/bD1zz5Kd to track availability for a meeting, hoping to meet next week.

@freben Can we reopen this? Still working on #15667.

@regicsolutions There is already an OpenAPI 3.1 spec of the catalog API. Is that what you're looking for?
I'm also currently working on #15667 which is a solution for the problem of modelling schema-first, using an OpenAPI spec and getting Typescript type restrictions (not runtime) on query/path parameters, request and response bodies and express router paths.

@sennyeya my use case is related to building a Golang software template where i am looking generate go code from an OpenAPI spec as described here

@regicsolutions I think the a part of the goal of this work would be a vended set of clients, but we're not there yet. You're welcome to use the spec for generating clients as needed, but expect the spec to change as it's still missing some catalog endpoints and the rest of the plugins don't have a spec at all. Also, the current catalog spec removes the /catalog or whatever prefix you're using, so you will need to add that back with something like openapi-merge.

@regicsolutions I think the a part of the goal of this work would be a vended set of clients, but we're not there yet. You're welcome to use the spec for generating clients as needed, but expect the spec to change as it's still missing some catalog endpoints and the rest of the plugins don't have a spec at all. Also, the current catalog spec removes the /catalog or whatever prefix you're using, so you will need to add that back with something like openapi-merge.

Thanks! Would I create a action to run the openapi-merge cli command within a software template? Was thinking I would just hit a swagger mock server?

@regicsolutions If your goal is a client Go application for your Backstage instance, I would recommend vending that internally through a Go package and not relying on a template. If you're looking to create a Go application from just any OpenAPI 3 spec, this isn't the right issue. The link you shared should work, but I would be careful to check for OpenAPI 3.1 vs OpenAPI 3.0 compatibility as many tools don't yet work with the new 3.1 spec format.

Thanks @sennyeya my goal is to be able to scaffold a repository/go application based on a OpenAPI 3 spec. The link requires running cli commands wasn't sure how to best handle that in a template.

@freben @taras  Wanted to follow up on this as I have some free time and #15667 was finally merged ❤.
Since we have a spec for the catalog now, I want to take a look at generating the catalog-client. I'm going to focus primarily on openapi-generator with openapi-generator-plus as a backup. We'll definitely want to deep dive into generated code and what not once I have example clients.

@bforbis Stay tuned for #17535, I ended up switching down to OpenAPI 3.0 for that work for the same reasons you're mentioning. If you want to handle that on your side more quickly, you can pull the changes that I made to the spec to translate it from 3.1 to 3.0.

Quick update for those following this,

#17535 has been merged and is part of the 1.20.0-next.0 release. This adds a new catalog provider to show a merged OpenAPI spec for your backend. The set of plugins it pulls from is manually defined and very few plugins have specs so far (limited to just catalog, search and todo rn).
As part of #17535, plugins that use the createValidatedOpenApiRouter method from @backstage/backend-openapi-utils will now have a /api/${pluginId}/openapi.json route that serves the full OpenAPI spec for this plugin, ie all spec paths are correctly mounted to /api/${pluginId}, etc.
#18185 and #19955 have been merged and can be used as part of 1.20.0-next.0. The first PR adds some basic linting of OpenAPI specs to the project using Stoplight and the second PR adds test -> spec validation (and optionally, automatic updating) using Optic.
#17470 is working and building finally! There were some hiccups while trying to find a good path forward, but I think we've overcome those. Will be trying to push on this now that the OpenAPI router work has mostly died down.

Looking forward, there are a couple of interesting areas I'd like to start exploring,

JSON schema ref deduplication. Answering, how can we have a consistent Entity and Error definition across plugins?
Notification of breaking changes when updating the spec. Optic has some pretty nice support for this, should be straightforward to add.
Starting to remove a lot of types from catalog-backend, search-backend, and todo-backend to start and replacing them with their generated counterparts.
API fuzzing. Building off of #17875 and #19955. This one solves two problems,
a. How can we continue to protect against bad input and ensure stability of our APIs? Mainly looking at 500 errors and bad requests.
b. How can we keep our specs up to date without manually iterating through all of the request/responses, either in test cases or in the spec itself?

Additionally, if folks are interested in writing API specs for their own plugins, I've started some documentation at https://github.com/backstage/backstage/blob/master/docs/openapi/test-case-validation.md and will be adding more soon.

Hej @aramissennyeydd - do you think this should stay open?

@tudi2d I think we're good to close this. Tracking the work in #17482.


Option to specify build type

Is there an option to specify the build option i.e Debug vs Release?
Right now i created two folders, debug and release and i manually change the cmake-ide-build-dir variable in .dir-locals.el to point to one of either debug and release, where i manually call
cmake  -DCMAKE_BUILD_TYPE=Release .. or cmake -DCMAKE_BUILD_TYPE=Debug .. when i make the switch from debug to release or vice-verca.
I thought about manually writing an elisp function to change the cmake-ide-cmake-command, but that doesn't sound like a good idea.
Is there a better way to do this which i am missing

Not right now, no. The idea is to run cmake manually first. It's on my TODO list to enable running cmake and selecting options from emacs itself. I'd recommend just having two build dirs, one for release and another for debug and changing cmake-ide-build-dir as needed.

If you don't mind having your cmake-ide-build-dir and cmake-ide-compile-command variables declared globally in your emacs config files, I wrote this little piece of code :
`(defun cmake-ide-debug-shortcut ()
(interactive)
(setq cmake-ide-build-dir "~/PATH_TO_PROJECT/build/debug/")
(setq cmake-ide-compile-command "~/PATH_TO_PROJECT/build.sh -d")
(cmake-ide-compile)
)
(global-set-key (kbd "C-") 'cmake-ide-debug-shortcut)
(defun cmake-ide-release-shortcut ()
(interactive)
(setq cmake-ide-build-dir "~/PATH_TO_PROJECT/build/release/")
(setq cmake-ide-compile-command "~/PATH_TO_PROJECT/build.sh -r")
(cmake-ide-compile)
)
(global-set-key (kbd "C-") 'cmake-ide-release-shortcut)`
And this is the bash script where you specify the cmake/make commands and the different paths for your builds :
`#!/bin/bash
if [[ $1 == "-d" ]]; then
mkdir -p build/debug
cd build/debug

cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ../..

make -j4

elif [[ $1 == "-r"  ]]; then
mkdir -p build/release
cd build/release

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ../..

make -j4

fi`
This a quick and simple solution, but I'm sure there is another way to have the variables defined locally for each project.


Makefile command for AWS deploy

https://github.com/gira-games/front-end/issues/5
https://github.com/gira-games/api/issues/2


Iv 1975 login name param

Login name feature slated for release 06/22/2016

Code Coverage is 82.7%, details at https://s3.amazonaws.com/rs-code-coverage/rsc/2016-06-21_644_Coverage-82.7.txt (sometimes the URL can take a few minutes to be available)

LGTM


Kinesis plugin affects performance of Hive plugin

https://github.com/prestosql/presto/blob/269000132b79e7c9c9056046e4eba45db5b810e9/presto-kinesis/src/main/java/io/prestosql/plugin/kinesis/KinesisConnectorFactory.java#L45
there's also a second place where "com.amazonaws.sdk.disableCbor" is set.

@ankitdixit would you be willing to fix this?
cc @guyco33

Fixed in 76c26e117a


Report CLAuthorizationStatus to Places

A new API is being developed to collect the authorization status of the device WRT access to location.
This issue is to properly update the Places Extension when the Auth Status of the device changes.

fixed in https://github.com/adobe/places-monitor-ios/releases/tag/v2.1.2-ACPPlacesMonitor


chore: consolidate clickHereToEarn translation

Description
Update the click-to-earn translation logic to use a single string, allowing us to support languages such as Turkish where the order of the current strings would be reversed.
I've removed the existing translations so the language team can redo them.
Notice

[x] Have you followed the guidelines in our Contributing guide?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[x] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)
Raised by @firebomb1 on Discord: https://discord.com/channels/554694662431178782/1001570348380200980/1055064315071303700.
Risk
Small and tightly scoped.
Testing
On the "earn" page the banner should show as it did previously. No visual change expected, except that I've lower-cased "earn".

Engineering
☝
Operations
☝
Screenshots (if applicable)
☝

Thanks for fixing this so quickly 😄


Non-admin user cannot join a container to a network

Bug description
A non-admin user joining a public container to a public network results in an error.
Expected behavior
Non-admin user can join a public container to a public network
Portainer Logs
2020/04/07 07:30:13 http error: Unable to proxy the request via the Docker socket (err=Error: No such network: connect) (code=500)
2020/04/07 07:33:26 http error: Unable to proxy the request via the Docker socket (err=Error: No such network: connect) (code=500)
2020/04/07 07:33:29 http error: Unable to proxy the request via the Docker socket (err=Error: No such network: connect) (code=500)
Steps to reproduce the issue:
(with non-administrator user)

Start a new container without access control
Create a new network (bridge) without access control
Open the newly created container in portainer, scroll to the bottom and select the new network
Click Join Network

Technical details:

Portainer version: 1.23.1
Docker version (managed by Portainer): Standalone 19.03.7
Platform (windows/linux): linux
Command used to start Portainer (docker run -p 9000:9000 portainer/portainer): sudo docker run -p 9000:9000 portainer
Browser: Google Chrome 80.0.3987.149

Additional context
Portainer is access behind a reverse proxy, but I've had no issues with anything else - and when an admin user joins the container to the network everything works as expected.

I have confirmed this on Portainer 1.23.2 with the user seeing error access denied to resource

+1

Same with Portainer CE 2.0.0

Hello I have fix available for this one. Just wanted to ask one thing. There is also issue in UI (at the current state of develop branch) - NetworkSelector and networks table uses network name instead of ids so access check will always fail (since it will check against ids) I have fix for that available too so I just want to know if I should add it to the same pull request


Wrong location?

Alexa thinks I am in Seattle with regards to weather, time, etc.  My Pi has the correct time.  How do I get Alexa configured for my current location?

are you sure you aren't in seattle?

:)
You need to run the Alexa app on either IOS or Android.  You can change your location settings there.  Seattle is the default.

Or you can change it in the web app: https://alexa.amazon.com


DataType of ExtensionObject stays null in UaExpert

When I read the server status of the OPCUA reference server using UaExpert, the datatype is correctly resolved. However, when I read the server status of the LibUA, the Datatype is Null (see image).
Do you have any idea why this happens?

I don't serialize extension object types because I couldn't find concrete information in the UA spec. If you capture those reads from the reference server with Wireshark I could take a look at it.


Release Tracking

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.
Releases
<EMAIL_ADDRESS>Patch Changes

#1424 d7b277ef Thanks @camertron! - Allow specifying an unchecked value for check box inputs

#1439 56fae759 Thanks @camertron! - Support system arguments for ActionList item anchor tags

#1438 b0329cda Thanks @langermank! - ActionListItem line-height bug fix

#1435 8312e6ce Thanks @camertron! - Allow NavList items to be selected by the current page

Testing notes: none of the functionality included in this release is being directly exercised by dotcom at this time.


chore(reporter)!: Remove the Excel reporter

The Excel report has not been maintained to reflect ORT's capabilities for a long time. If Excel interoperability is required, the recommendation is to instead use the PlainTextTemplateReporter to create a CSV file that can be imported to Excel.

Note
This is a breaking change for users of the Excel reporter. There is no direct substitute. See above for a recommended alternative solution. Removal was agreed on in today's community meeting.

Merging despite the unrelated test failures.


TST, MAINT: adjust 32-bit xfails for HiGHS

Fixes #16347

ensure that two HiGHS-related tests
that segfault for 32-bit Linux runs
in the wheels repo are not only marked
with xfail, but do not execute at all
(to avoid segfaulting, of course)

for one of the tests, this will be conditional
on 32-bit platform

for the other test, there was already an xfail
mark so I've just bumped it up to not execute for
now

I reproduced the two failures in a 32-bit Linux Docker
image (i386/ubuntu:bionic), and confirmed full suite passing after the
adjustments

Do we have some kind of tracking for xfail? In this case since it's not even running I suppose we should have a reminder somewhere.

We can use pytest flags like -rsx to get a more detailed summary of what was skipped/xfailed at the end, if we need it. I think we recently disabled that in a few places in CI though because it was too verbose and people would rarely use it.
The final pytest accounting isn't too bad, i.e., 49256 passed, 2697 skipped, 607 xfailed, 11 xpassed in 1024.30s (0:17:04) . Core devs can occasionally check on this with -rsx if they want to chip away at xfails for example, though it is probably a dreadful task given that we often add those to avoid annoying stuff in the first place (in this case, an upstream issue I suppose).

I updated the PR to address my own review comment--@mdhaber or @mckib2 let me know if there are any changes you want here.


Fix Round trip Message serialization/deserialization

#336
Fix to "Round-trip Message Serialization/Deserialization"
The test failed before, now it passes. The other tests still pass. It should work on 'image', 'audio', 'tool_call', 'parsed', 'tool_result', but I haven't tested it. Do you think I should still do that?

Sorry, I tried to open the pull request with the issue ...


Java 11 support

Hi,
are there any plans to support java 11 in the near future?
Thank you
Regards

What do you mean by "support Java 11"?

Hi,
"support java 11" means "compile javacc grammar with java11 and execute the generated code with java 11".

Using Java 11, you can compile a Java 8 source code and execute it using the Java 11 JRE. While JavaCC itself is using features from the Java 1.7, it is compiled with the source & target JDK 1.8. Thus you can run the produced parser/lexer with Java 11 without any problem.
Java 9 and Java 10 are not supported means that the Java language features of those version are not handled by JavaCC, i.e., you cannot use Java 9 or Java 10 code in your actions.


JsonWebTokenError: jwt must be provided

While running docker-compose up I follow this steps:

http://<IP_ADDRESS>:3000/
Login with<EMAIL_ADDRESS>foobar
Error 500 in http://<IP_ADDRESS>:3000/api/login
{"error":"Could not verify consent challenge: JsonWebTokenError: jwt must be provided"}

Is there a workaround to make it work?

Please, read the manual. It is explained how the docker-compose tutorial works.
https://ory-am.gitbooks.io/hydra/content/tutorial.html


bundle-config(1) manpage says bundle install doesn't remember options but it does

From https://github.com/bundler/bundler/blob/v2.0.1/man/bundle-config.ronn#remembering-options

Flags passed to bundle install or the Bundler runtime,
such as --path foo or --without production, are not remembered between commands.
If these options must be remembered,they must be set using bundle config
(e.g., bundle config path foo).

IIRC it was true in 2.0.0.dev but had been reverted back to the original behavior. Maybe we should reverse the change to manpages in this commit? https://github.com/bundler/bundler/commit/e120f40f72fe53f55e05db9204f98d28f1065e81

Yes, this should be fixed. We would welcome a PR to help resolve this.

Okay, I'm on it. I found that a few more options are actually remembered -- --frozen, --jobs,  --retry, --shebang, --trust-policy, and --with. I'll describe them too.

Would this be causing an issue when installing from git source?

The git source https://github.com/whoever/whatever.git is not yet checked out. Please run bundle install before trying to start your application

We just started getting this on CI this week, I'm thinking it could be related to the bundler 2.0.2 release.

Confirmed, fixing bundler version to 2.0.1 has solved the error:

The git source https://github.com/whoever/whatever.git is not yet checked out. Please run bundle install before trying to start your application

@TSMMark I believe the issue you have is caused by this change https://github.com/bundler/bundler/pull/6798

Use https in the Github shortcut by default in Bundler 2

That is, gem 'foo', github: 'whoever/whatever' defaults to HTTPS, instead of plain HTTP, since 2.0.2.

Beside that, is there a way to prevent re-writing/touching config file at all? Bundler re-writes config on each bundle install which may lead to another docker layer with only this changed file (in some setups). I'll tried to write-protect it, but this didn't help (as root):

rm .bundle/config && touch .bundle/config && chmod 0440 .bundle/config
bundle -v

Bundler version 2.0.2

bundle install --jobs=4
bundle conf

Settings are listed in order of priority. The top value will be used.
jobs
Set for your local app (/app/.bundle/config): 5
...

Keeping it untouched in this case would be perfect...


415 (Unsupported Media Type) when hitting the Connect API

Hey @johanbrandhorst — great work on this!
I spun it up locally and mostly everything works, including the REST API.
But I get a 415 error when I hit the Connect API (as shown in the docs):
curl -X POST -v \
    --header 'Content-Type: application/json' \
    localhost:8080/users.v1.UserService/ListUsers

returns
HTTP/1.1 415 Unsupported Media Type
Accept-Post: application/connect+json, application/connect+proto, application/grpc, application/grpc+json, application/grpc+proto, application/grpc-web, application/grpc-web+json, application/grpc-web+proto
Content-Length: 0

🤷🏻 I'm not sure why this is happening, is there any reason to believe that it is because of the setup? Have you asked the Connect maintainers? Does it only happen if the grpc-gateway is in front of it?

The AddUser endpoint is working
http -v localhost:8080/users.v1.UserService/AddUser<EMAIL_ADDRESS>
But not the ListUsers endpoint:
# fails
http -v POST localhost:8080/users.v1.UserService/ListUsers "Content-Type":application/json Accept:"application/json, */*;q=0.5"

# succeeds
curl localhost:8080/api/v1/users

No change when removing the OpenAPI handler and the Gateway handler.


Keep the current topic when switching between Jmix versions 1 and 2

Use case description:

Open the documentation. It automatically opens Jmix 2 version
Find some specific article like https://docs.jmix.io/1.x/jmix/1.5/data-access/fetching.html
Switch to Jmix 1 version in the header navigation bar and the current topic will be dropped to the Intro

This use case is important for those, who are still working with Jmix 1.X and forget to switch the Jmix version after opening the documentation page. It is also useful when developers see that some information in Jmix 2.X is missing and they would like to check the same in Jmix 1.X version

The problem is mainly resolved in #897 and #915


[Coverity CID :195871]Integer handling issues in /drivers/counter/counter_ll_stm32_rtc.c

Static code scan issues seen in File: /drivers/counter/counter_ll_stm32_rtc.c
Category: Integer handling issues
Function: rtc_stm32_read
Component: Drivers
CID: 195871
Please fix or provide comments to square it off in coverity in the link: https://scan9.coverity.com/reports.htm#v32951/p12996

Cf analysis in #14415.
Will be fixed when upgrading SDK to 0.10.0.

@erwango actually coverity spot this problem even with latest sdk. Question, why don't you cast this way:
ticks = counter_us_to_ticks(dev, (u64_t)ts * USEC_PER_SEC);

Here the we ensure that the multiplication won't overflow

actually you don't need the cast at all, ts is time_t

https://github.com/zephyrproject-rtos/zephyr/pull/14799

@ceolin , cf my comment in #14799, none of the proposal works actually.

Can we close this now?

Fixed by #14799


BLE Host - Fix peer OTA addr and type on pairing.

I may be missing something, so please don't hesitate to yell at me if this change breaks some expected use case.
This change address the following scenario:

Both devices using public addresses; no privacy.

A connects to B.
A initiates pairing.
Pairing succeeds; B indicates that its ID address is the same public
or static random address it was advertising with (i.e., no change in
address).

After this sequence completes, the connection object should be populated
as follows (from A's perspective):
* bhc_peer_addr.type: BLE_ADDR_PUBLIC

However, I see the following instead:
* bhc_peer_addr.type: BLE_ADDR_PUBLIC_ID

This is problematic because BLE_ADDR_PUBLIC_ID indicates that the peer
is using an RPA ("Public Identity Address (Corresponds to Resolved
Private Address)").  The ble_hs_addrs() function uses the address type
to determine the peer's OTA (over-the-air) address.  If the peer's
address type is an "_ID" type, the rpa field is used for the OTA
address.  In this case, the peer is not using an RPA, so its OTA address
is identical to its identity address.
The change is to only set the peer's address type to "_ID" if it is
using an RPA.

This indeed looks suspicious. Lets do some more testing (including PTS) just to be sure.
I'll merge it once we confirm no regressions.

For me it looks good.

Talked to sjanc and he don't have objections - merging.


How to implement vip load balance？

3 control plane：
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
vip：
<IP_ADDRESS>
kube-vip daemonset run ok in my k8s cluster.
so,in k8s cluster, i run three of  "whoami" images pods in worker nodes ,and create "whoami" service with loadbalance type,and loadbalanceIP is <IP_ADDRESS>, expose 80 port。
but i use chrome visit http://<IP_ADDRESS>，only response one node data，kube-vip no loadbalance for this service。
Is there a problem with my usage? and How to configure？

can you paste your service yaml definition below.

Closing this as it's been two months.


Redirect 404

I have a project made with next.js with a structure like this:
pages

folder

[mypage].ts

/folder/mypage works but /folder return 404
I want to be able to redirect /folder to /folder/mypage
I tried using now.json like this:
{     "routes": [         {"src": "/folder", "dist": "/folder/mypage"}     ] }
and
{     "routes": [         {"src": "/folder", "status": 404, "headers": {"Location": "/folder/mypage"}}     ] }
but it did not work
thanks for help

Hi there @abraaobuenotype. Could you please create a repository demonstrating what you are trying to achieve so that we can understand better?
Also, this is probably the kind of question that could be better answered at https://zeit.co/chat

hi @TooTallNate thanks for answering and your help
basically i have this structure:

where index is the main page (route: "/")
and [tech] another page (route: "/for-you/tech")
I call this page (tech) using the next/link component

this is working, even when I reload the page, the "/for-you/tech" url works
but when I type in the browser url "/for-you" the answer is 404
so what I need is:
if the user enters the "/for-you" route in the browser, the server redirects the route to "/for-you/tech" instead of returning 404
I tried to do it like in the other message, editing the file now.json, sure I'm doing something wrong because it didn't work

The first example you showed should be working:
{ "routes": [ {"src": "/folder", "dest": "/folder/mypage"} ] }

Note that it's dest, not dist. Not sure if that was how your code actually was or just a typo in this issue.

no ... this really isn't working...

here is the project url:
https://pismoio.abraaoas3.now.sh/for-you/tech

hey @TooTallNate
I managed to make it work this way:

thanks for your time and your help


Add TLS via TCP example

Shows how to use the TLS stage to implement an Echo stage.
The connection completes once the server gets the "z" which we expect to be the last element.

Merging - travis has trouble due to the S3 outage it seems


feat: add /similarity route

Required for compatibility.

@OlivierDehaene Would it make sense to make it compatible with huggingface-hub client: https://github.com/huggingface/huggingface_hub/blob/ca111fb51193968faed548a4b865c79736822aed/src/huggingface_hub/inference/_client.py#L1416 ? It looks like they expect a bit different names for the attributes of SimilarityInput (https://github.com/huggingface/text-embeddings-inference/blob/4cc38bda85b2ecdbf34082796ca9738466483b85/router/src/http/types.rs#L364).

What do you mean? It is compatible.

The payload is: https://github.com/huggingface/huggingface_hub/blob/ca111fb51193968faed548a4b865c79736822aed/src/huggingface_hub/inference/_client.py#L1457
which is what we expect here.

Oh, my apologies. I must've looked at a different revision or something. Thanks for the quick reply!


move contributing to root and add contributors

#112

@Lioscro I have updated the contributor page. Could you please review it?


Add Benchmarking for planner benchmark

This PR adds various utils and packages that are being used for benchmarking the planner

Voxblox local planner benchmark

Hilbertmap integrated into benchmarks!

Also planning

Merging for now


QSO logging tabindex

Because there were some changes in UI since the last tabindex code it needed a fix -
Was reported by DO1EDK
Sequence is:
1 - Mode
2 - Band
3 - Frequency
4 - Date         (only in POST QSO)
5 - Time On   (only in POST QSO)
6 - Time Off   (only in POST QSO and if seperate times is enabled)
7 - Callsign    (after save a QSO you start again here)
8 - RSTs
9 - RSTr
10 - Name
11 - IOTA (if enabled in user settings)
12 - SOTA (if enabled in user settings)
13 - WWFF (if enabled in user settings)
14 - POTA (if enabled in user settings)
15 - SIG (if enabled in user settings)
16 - SIG Info (if enabled in user settings)
17 - DOK (if enabled in user settings)
18 - Location
19 - Gridsquare
20 - Comment
21 - SAVE QSO
22 - Clear Input

What was the old sequence?


Create create-vault.json

new vault and backup policies

@snoviking Could you take a look at the pull request? Thanks.

New templates can't be merged in this repository. We'll close the pull request.


docs(book): recommend running with Docker Compose

Good point here https://x.com/odysseas_eth/status/1779535662116430035

I think there should be a comment about using mounted volumes, as I assume that for most users that run this on a server, the default docker volume directory is on the smaller main disk. In contrast, the user will probably want to use the mounted disk in a different directory.
I think the only way for this to be easily customizable is to have a bash script that accepts as an argument a directory (e.g /data/reth) and:

mkdir the directory recursively if it doesn't exist
adds the directory as a mounted volume in the docker compose

If we really want to hand hold users, the same bash script could check that the directory will have the required storage for the default reth configuration (archive node)


Grammar mistake? --> "Opportunities are sheer endless"

"Opportunities are sheer endless" on the home page (website/static/index.html) reads strange.
Perhaps revise to “There is a sheer endless number of ways to put Legend to use ... "
What do you think @nicoalba @beekemarie?

This has been fixed.

cool thanks


Install nco on jupyterhub?

Would it be possible to install nco (https://nco.sourceforge.net/) on the jupyterhub?
Thanks!
David

Hi David, we've added nco to the docker image already, but we'll need to update the CryoCloud JupyterHub to use that docker image, which may take a day or two. cc @tsnow03


feat: 🎸 add press events to Tab

✅ Closes: #6810
✅ Pull Request Checklist:

[x] Included link to corresponding React Spectrum GitHub Issue.
[x] Added/updated unit tests and storybook for this change (for new code or code which already has tests).
[x] Filled out test instructions.
[x] Updated documentation (if it already exists for this component).
[x] Looked at the Accessibility Practices for this feature - Aria Practices

📝 Test Instructions:

yarn start
Navigate to Tabs Example in React Aria Components in Storybook
Notice that clicking on a Tab fire press event actions
Notice no breaking changes to existing tab behavior

🧢 Your Project:

I just signed the CLA, not sure why it's showing up as not signed still...

I cannot seem to get the CircleCI Pipeline check to pass despite following these instructions: https://support.circleci.com/hc/en-us/articles/12504745606171-Why-am-I-seeing-the-Could-not-find-a-usable-config-yml-you-may-have-revoked-the-CircleCI-OAuth-app-message
Do I really need to submit a support ticket?


JSONView has been removed/disabled from the Chrome Web Store

The extension is also automatically disabled with the notice that "this extension contains a serious security vulnerability."
Does anyone know more or how to fix it?
Further input

https://github.com/bhollis/jsonview/issues/135
https://news.ycombinator.com/item?id=12925197#12925360

Update
I emailed @gildas-lormeau but did not hear back from him.
I switched to "JSON Viewer" now like @dan-blanchard suggested. JSON Viewer has a cleaner Issue and PR List than JSON-formatter, so I go with JSON Viewer and hope the maintainer will stay with us :).

Does this PR from 2 years ago address the issue? https://github.com/gildas-lormeau/JSONView-for-Chrome/pull/49

Yep, can confirm it. Use the following page as a reproducible test case
The gist file being served can be found here: https://gist.github.com/MattRyder/f356b402f696f147943907eb8a3859e5

I've been getting some questions on my (unrelated & unpublished) jsonview-chrome repository, https://github.com/jamiew/jsonview-chrome
Has anyone stepped up to fix things in this repo + republish yet?

There are alternative extensions that seem to be just as nice (if not nicer):

JSON Formatter
JSON Viewer

Yep, there are actually several different UXSS issues in JSONView-for-Chrome's master.

One @joevennix reported in the autolinking code https://github.com/gildas-lormeau/JSONView-for-Chrome/pull/49
One found by a Chinese security researcher in the JSONP handling http://blog.knownsec.com/2016/04/明枪易躲暗箭难防-jsonview-0day/
Another UXSS I privately reported to the maintainer / Google along with a patchset. This issue allowed more-or-less complete Same-Origin policy bypass because it abused an eval() in the content script. The maintainer didn't respond so Google removed the extension.

If someone wants to take ownership of a fork, this patchset should fix all three issues.

This issue was moved to sloanlance/JSONView-for-Chrome#1

@dan-blanchard They don't work the way JSON View did.

Which feature(s) are those 2 that @dan-blanchard listed missing?

Yeah, I've found JSON Viewer to be a lot nicer than JSONView.

Main thing I miss from old JSONView is the ability to see the path in the status bar, and easily copy the path or value via right-click.
I now switch between using JSON Formatter and JSON Viewer, but far as I can tell neither offer that old beloved feature.

@alahosky I think you should post this as a feature request on one or both of the other plugins github repos, so they can think about adding it.
Btw, ATM https://chrome.google.com/webstore/detail/jsonview/chklaanhfefbnpoihckbnefhakgolnmc ist live again – don't know why.

@tordans The maintainer applied the patchset from #76 and re-listed the extension in January, not sure why the repo isn't up to date.

@JordanMilne Thanks. This could all go so much more smoothly, but well… :-)

Yup @tordans ... I've also mentioned this on the existing JSON Viewer feature request here: https://github.com/tulios/json-viewer/issues/67#issuecomment-278881487
Feel free to vote for it over there :)


nlmeans error when sr = 0

returns vapoursynth.Error: "a" must be positive

What we can do for the time being is change the param names, accept kwargs, and pop the current param names (that were changed) and throw a deprecation warning if any exist?


Allow reviewers to be added when creating a Pull Request, the same way as assignees can be added

Gitea version (or commit ref): 1.30.0
Git version: N/A
Operating system: CentOS

Database (use [x]):

[ ] PostgreSQL
[x] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[x] Yes (provide example URL): https://try.gitea.io/api/swagger#/repository/repoCreatePullRequest
[ ] No

Log gist:

Description
The gitea API allows assignee/assignees to be added when creating a Pull Request using the API.
It would be really useful if the API could also allow reviewer/reviewers to be added in the same way.
Screenshots
N/A

This is currently not possible through the UI either at pull request creation time.
The reviewers can only be added after the PR has been created:

This would be useful

Thank you @delvh,
Already tested your suggestion if configure with regex reviewers successfully assigned. Problem  when assign reviwers to specific folder.

Below example I have tried:

     /docs/  @org-1/Team2
     /frontend/src/   @org-1/Team1
    /folder/sub-folder/  @user1

All three folders(docs, frontend, folder) are located at the root of repository.

CODEOWNERS seems related, but not exactly what was requested in this issue.
But there is a PR that resolves the requested feature: #26596


netlify-cms-app fails to install with react 17 as version 6 of react-aria-menubutton in incompatible with react 17

Describe the bug

When installing netlify-cms-app with react 17 the install fails with the following error:
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/react
npm ERR!   peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   node_modules/react-dom
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!     peer<EMAIL_ADDRESS>|| ^17.0.0" from<EMAIL_ADDRESS>npm ERR!     node_modules/netlify-cms-app
npm ERR!       netlify-cms-app@"*" from the root project
npm ERR!     8 more (netlify-cms-widget-date, ...)
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!   38 more (netlify-cms-app, @emotion/core, @emotion/styled, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer<EMAIL_ADDRESS>|| ^15.0.0 || ^16.0.0" from<EMAIL_ADDRESS>npm ERR! node_modules/netlify-cms-ui-default/node_modules/react-aria-menubutton
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   node_modules/netlify-cms-ui-default
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!     node_modules/netlify-cms-app
npm ERR!       netlify-cms-app@"*" from the root project
npm ERR!     23 more (netlify-cms-backend-azure, ...)
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!

It seems like react-aria-menubutton in netlify-cms-ui-default needs to be upgraded to version 7.0.1 see: https://github.com/davidtheclark/react-aria-menubutton/issues/140
To Reproduce

Run npm init -y && npm i react react-dom &&  npm i netlify-cms-app in an empty directory.
See Error
Expected behavior

netlify-cms-app installs without having to use --legacy-peer-deps
Screenshots

NA
Applicable Versions:

Netlify CMS version: 2.15.6
Git provider: NA
OS: Arch Linux
Browser version NA
NPM version: 7.12.1
Node.JS version: 16.1.0

CMS configuration

NA
Additional context

Seems like a dependency that didn't get updated while fixing
https://github.com/netlify/netlify-cms/issues/5111

Thanks @something-something-something, this is also an issue with react-split-pane:
https://github.com/tomkp/react-split-pane/pull/681

Blocked by https://github.com/davidtheclark/react-aria-menubutton/pull/133

Blocked by davidtheclark/react-aria-menubutton#133

This is now complete

Woo @shirshendubhowmick, this is amazing. Thank you for the quick follow up ❤

I seems the react-codemirror nested dependency could also be blocking this?
This is the error I get when trying to update to React 17:
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/react
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!   peer<EMAIL_ADDRESS>|| ^17.0.0" from<EMAIL_ADDRESS>npm ERR!   node_modules/framer-motion
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!   45 more (react-awesome-styled-grid, react-dnd, react-dom, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer<EMAIL_ADDRESS><=16.x" from<EMAIL_ADDRESS>npm ERR! node_modules/netlify-cms-widget-code/node_modules/react-codemirror2
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   node_modules/netlify-cms-widget-code
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!     node_modules/netlify-cms-app
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!

Thanks @paulobarcelos, that's helpful. Looks like there's a PR for it https://github.com/scniro/react-codemirror2/pull/224, but no comment from the maintainer.

Thanks @paulobarcelos, that's helpful. Looks like there's a PR for it scniro/react-codemirror2#224, but no comment from the maintainer.

By the looks of it, seems that project may be dead. Wonder if it would be a good idea for Netlifly CMS to change to the proposed updated fork while the original one doesn't change (if it ever will). Feels like halting support for React 17 is a bigger issue than dealing with the annoyance of having to migrate to an unofficial fork of the dependecy.

That fork doesn't seem to be updating frequently too and I don't think we want to install directly from GitHub.
For React 17 + npm@7 support there's a workaround by passing the --legacy-peer-deps flag, which is clearly stated in the error.
Not sure if we should find a completely different solution to our Code widget.
Since this seems to be an issue with more that one dependency, we might have to think of a solution that doesn't involve chasing down maintainers.
I wonder if there's a way we can update dependencies package.json files on the CMS side to allow users to install it.

I get a similar error when trying to install the package netlify-cms-app:
npm ERR! Could not resolve dependency:
npm ERR! peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR! node_modules/netlify-cms-app/node_modules/react-split-pane
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   node_modules/netlify-cms-app/node_modules/netlify-cms-core
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!     node_modules/netlify-cms-app
npm ERR!       netlify-cms-app@"*" from the root project
npm ERR!

It seems that react-split-plane may be dead or at least poorly maintained as the last commit was in May. There's a highly commented issue for the react 17 upgrade - tomkp/react-split-pane#713 and a PR for it - tomkp/react-split-pane#681. With the maintainer having left no comments on either.

After waiting for months, I realized the whole project was only a couple of MIT-licensed files, so I just copied them into my project and was done with it.

@JeremyGrieshop How did you integrate the copied files into the package?

@JeremyGrieshop How did you integrate the copied files into the package?

I don't have a public repository to share at this moment, but I just copied these 4 files into a local directory, called SplitPane in my project and imported from it instead of react-split-pane:
https://github.com/tomkp/react-split-pane/tree/master/src
It does, at this time, require a polyfil, so you'll need to add react-lifecycles-compat as a dependency for now.  If these are ever converted over to class-less components/hooks, then it won't be necessary.  It also required react-style-proptype, but I just removed that code from the appropriate classes, since I didn't care about the prop types.  Otherwise, you'd need to add that dependency as well.

For the mantainers of Netlify CMS would it be possible to follow what this user had commented on the PR in react-split-pane - https://github.com/tomkp/react-split-pane/pull/681#issuecomment-1000543520 and simply copy the needed components into the internals of Netlify CMS with the right license?

Here's an example I quickly pushed to my personal repository that someone could copy into their own SplitPane folder.  I converted over some of the older class dependencies and made them functional components that use hooks instead:
https://github.com/JeremyGrieshop/react-split-pane

Hi everyone 👋 At this point I think we will accept a contribution that inlines react-split-pane as it seems the won't be a any progress with https://github.com/tomkp/react-split-pane/pull/681

@erezrokah Sounds great! I'm working on that right now in this fork https://github.com/LuisOsta/netlify-cms based on the version created by @JeremyGrieshop.
Btw Jeremy let me know if the attribution I included in the commit looks good to you (I included the README and a link to your repository)

@erezrokah If you could take a look when you have the time that'd be much appreciated

Aside from react-split-pane are there any remaining packages that are causing React 17 compatibility issue? What's left for us to be able to definitely close this issue?

Btw Jeremy let me know if the attribution I included in the commit looks good to you (I included the README and a link to your repository)

Looks great!  I hope you find it useful and didn't leave anything out of the original work.

This seems to be fixed


[DATA] add Global Human Settlement layer to geohub

Issue Description
need to add Global Human Settlement Layer to GeoHub.
References
https://human-settlement.emergency.copernicus.eu/download.php?ds=builtH
Dataset ID
No response
Screenshots
No response

ghsl population layers were added by @iferencik


Fix read_svm_format when no label is associated with an example

What does this PR do?
Fix a bug introduced in commit 72c6458.
Because labels is None when no label is associated with an example, we should check whether it is None before calling split().
Test CLI & API (bash tests/autotest.sh)
Test APIs used by main.py.

[ ] Test Pass

(Copy and paste the last outputted line here.)

[x] Not Applicable (i.e., the PR does not include API changes.)

Check API Document
If any new APIs are added, please check if the description of the APIs is added to API document.

[ ] API document is updated (linear, nn)
[x] Not Applicable (i.e., the PR does not include API changes.)

Test quickstart & API (bash tests/docs/test_changed_document.sh)
If any APIs in quickstarts or tutorials are modified, please run this test to check if the current examples can run correctly after the modified APIs are released.

I think we discussed this previously, and the regex guards against bad inputs.
Is there an sample data where the original function fails?


Enabling PCA transform algorithm support for GPU

Enabling PCA transform algorithm support for GPU

make test

make test

@maria-Petrova , I did not find any MR for pca-transform tests. Please create it


infoDict.has is not a function // undefined is not a function

PDF files generated via a certain book scanner can not be displayed in owncloud 7.0.4 (stable).
A sample file will be prepared and sent to me soon.
Product: 	book2net
Component: 	tocosa
Version: 	5.84 (tocosa)
All 3 browsers are affected but with different error messages:
Mozilla FireFox 36.0.1
Internet Explorer 11.0.9600.17631
Internet Explorer Updateversion 11.0.16
Chrome Version 40.0.2214.115 m
after clicking on that file start  all 3 error messages start with:
///////////////////////////////////////////////////////
[Could not load PDF file]
PDF.js v0.8.1136 (build: ca612e5)
///////////////////////////////////////////////////////
followed by:
[1] Mozilla Firefox - Message: infoDict.has is not a function
[2] Internet Explorer - Message: Object doesn't support property or method "has"
[3] Google Chrome - Message: undefined is not a function
PDF document properties are: PDF Version 1.6 (Acrobat 7.x)
Adobe Acrobat xmp output for the PDF file:

xmlns:xmp="http://ns.adobe.com/xap/1.0/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:pdf="http://ns.adobe.com/pdf/1.3/">
xmp:CreateDate2015-03-05T22:01:26+01:00/xmp:CreateDate
xmp:ModifyDate2015-03-05T22:01:26+01:00/xmp:ModifyDate
xmp:CreatorToolBGD-PDF Library Ver. 3.4/xmp:CreatorTool
dc:formatapplication/pdf/dc:format
pdf:ProducerBusiness Graphics Datentechnik GmbH/pdf:Producer
/rdf:Description
/rdf:RDF
/x:xmpmeta

Could you provide a link to a sample PDF that generates this kind of error?

Closing as duplicate of #5825


Pattern design kit: centered lead space pattern

Wonil-Suh1 created the following on Oct 01:
User story
As a designer working on an innovation project, I need to be able to use the centered lead space pattern without having to recreate it every time I use it so that my designs are to the specs and consistency is maintain across different touch points in the user's journey.
Notes
This is a discovery story to build centered lead space symbols to understand the mechanisms and limitations of Sketch symbols to inform the larger strategy for the final design kit(s).
Acceptance criteria

[ ] 5 pattern symbols for the 5 break points have been built
[ ] Different line numbers can be handled
[ ] Different numbers of buttons can be handled
[ ] Different background image can be added
[ ] Background gradient can be turned on / off

Original issue: https://github.ibm.com/webstandards/digital-design/issues/1793

@wonilsuhibm @larahanlon2 @oliviaflory do we have this captured already or we still need this issue?

We might need this issue, but it depends a little on what we decide to do with the pattern design kit.


Documentation/ARM Templates with Application Gateway

Do you have any to share please?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 77a03366-6bf3-c687-d9e4-39eb8774d3a2
Version Independent ID: ac420896-0070-59b1-6a76-376c3ed22875
Content: Networking patterns for Azure Service Fabric
Content Source: articles/service-fabric/service-fabric-patterns-networking.md
Service: service-fabric
GitHub Login: @rwike77
Microsoft Alias: ryanwi

@ohwellwhatever Thanks for the feedback. We are actively investigating and will get back to you.

@ohwellwhatever  Hi, take a look at this article which describes how to integrate Azure API Management with Service Fabric:  https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-tutorial-deploy-api-management .  There's a link to the sample ARM template as well.

Thanks.  I have seen that, but am looking specifically for documentation regarding integration with Application Gateway, similar to this reference architecture: https://blog.nilayparikh.com/azure/services/cloud-architecture-pattern-azure-service-fabric-and-microservices-part-3-security-architecture/

@ohwellwhatever  No, we don't have any docs or samples right now specifically for integrating with Application Gateway.  We do recommend API Management because it handles routing to specific services and static public IP addresses, unlike Application Gateway.  I've opened a work item in our backlog to investigate adding some guidance for Application Gateway.  Thanks for your questions.  #please-close


🐛 Make modifier keys work in react-shortcuts

Fixes #133
We weren't checking for modifier keys properly using getModifierState, but just checking event.key instead. This doesn't always work, as the user would need to press the modifier keys in that particular order. (which shouldn't be the case, example: Cmd + Option + c equals Option + Cmd + c)
Doing what we did also meant that the user couldn't perform an onMatch by holding the modifier keys and then repeatedly pressing a normal key. (example: Cmd (hold) + v + v + v pastes thrice, but it would work only once if we use the current version of react-shortcuts)
Considering all these problems, I think that separating the keys prop into modifierKeys prop and keys prop made most sense.
Also, I'll push this and #171 together, and then release.

@tzvipm there's two tests for the new behaviour now. I don't think we need more, but if you think of a particular case that should be in here, lmk and I'll add it. 👍


Integrate Socket API into existing CloudFront Solution

Hi,
thanks for the sample, which helped us a lot to understand how to use WebSockets with Amazon API Gateway. We were able to create a new Socket API for our solution based on the samples. We wanted to integrate it created via Amazon CloudFront into our existing solution (according to the AWS Documentation WebSockets should work on CloudFront: https://goo.gl/4AMRPb).
Currently we do have a CloudFront distribution (incl. CNAME with e.g. mydomain,com) with multiple origins

an S3 Origin where we do have the frontend  (frontendOrigin)
a custom origin for the API  (apiOrigin)
an additional S3 Origin, where we also store frontend pictures (picturesOrigin)

For these origins we created several Cache Behaviours:

The Default (*) behaviour is pointing to frontendOrigin
A Behaviour with Path Pattern /api/* which is pointing to apiOrigin
A Behaviour with Path Pattern /pictures/* which is pointing to picturesOrigin

Now we wanted to integrate our new Socket API created via API Gateway to CloudFront and created a new Origin called socketOrigin with a Behaviour with Pattern /api/socket which is pointing to socketOrigin (with the correct precedence).
After redeploying the distribution we were not able to open a connection to the socket, because of a ** HTTP 403** Status code. I found a way to get more details by using a curl:
curl --include \
     --no-buffer \
     --header "Connection: Upgrade" \
     --header "Upgrade: websocket" \
     --header "Host: <cloudfrontid>.cloudfront.net" \
     --header "Origin: https://<cloudfrontid>.cloudfront.net/api/socket" \
     --header "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" \
     --header "Sec-WebSocket-Version: 13" \
     https://<cloudfrontid>.cloudfront.net/api/socket

We got the following response back by CloudFront:
HTTP/1.1 403 Forbidden
Content-Type: application/json; charset=UTF-8
Content-Length: 75
Connection: close
Date: Thu, 10 Jan 2019 09:57:07 GMT
X-Cache: Error from cloudfront
Via: 1.1 blahblahblah.cloudfront.net (CloudFront)
X-Amz-Cf-Id: YxowUpxLm6t3VEV1Foue36kmdrOilhVlxqHN0S59dy-7cca681LiHw==

We thought, that it might have something to do with the path pattern, as we don't specify any wildcard, so we switched the path pattern to /api/socket/* , but this didn't help. We checked the Viewer Protocol Policy (HTTP & HTTPS) and Origin Protocol Policy (Match Viewer), but this also didn't help to solve the 403.
So, here is the question, if Amazon API Gateway & Amazon CloudFront is working with WebSocket APIs and if yes, how it must be configured to get to work. Can you provide any samples? If it is not working yet, we would like to open a feature request to support the above mentioned configuration.
Kind Regards,
Marcell

auto: bump from week 5 to  week 10

@jt0 do you have any spare time to take a look onto this one? I've bumped twice and I'll probably be locked until week 10/2019.

@kirnberger were ever able to get this to work? Or is it basically not working?

I believe problem here is path mapping between CloudFront and API Gateway.
Let's say you have WebSocket API Gateway with id mce8m6z8s8 and stage v1. The WebSocket URL is then: wss://mce8m6z8s8.execute-api.eu-central-1.amazonaws.com/v1
When you want to connect to the WS, you must connect to exactly this URL. If you try to connect to, let's say, wss://mce8m6z8s8.execute-api.eu-central-1.amazonaws.com/v1/whatever, you will get 403 forbidden error.
Now you attach this API Gateway to the CloudFormation though the behavior with path pattern /api/socket. When you call xxxx.cloudfront.net/api/socket CF sends a request to the API Gateway URL: wss://mce8m6z8s8.execute-api.eu-central-1.amazonaws.com/v1/api/socket instead of root path.
CloudFront does not cut the path from the request when sending it to the origin. It is not possible to configure either.
See possible path mappings here: https://advancedweb.hu/how-to-use-api-gateway-with-cloudfront/#origin-request-url - at the end always full path is passed to the origin.
The only workaround I see to configure WS on path is to set API Gateway stage name, for example "socket", and then configure path pattern for it. So you can have it working on url xxxx.cloudfront.net/socket

As @m-radzikowski mentioned, this isn't the default behavior. I'll pass this as a feature request, but closing for now.

@dgomesbr Please do. I already mentioned it to Katrina Bakas. This is really annoying, because it requires us to use the WebSocket stagename in our front-end application code, which is not ideal.

Dear all,
I am writing in this thread because i really don't know where to look anymore. The issue experienced by the user ghost is very similar to mine, even though i didn't use any of the code of this repo.
I am in dire need of having an unsecured websocket connection which i can't realize with a straight ApiGateway on AWS, as we all know.
My idea was to use CloudFront to allow HTTP/HTTPs access to the websocket endpoint.
I have therefore set up a CloudFront distribution. I am fairly sure that everything is okay there. I chose not to cache anything, and to accept everything from the viewer (headers, etc).
My api gateway endpoint for the websocket is somethng like wss://xxxxxxx.execute-api.us-east-1.amazonaws.com/stage
What is did was to indicate as origin in the cloud front distribution wss://xxxxxxx.execute-api.us-east-1.amazonaws.com and path /stage and as behavior i have indicated both HTTP or HTTPs connection plus the caching behavior and header forwarding i have just mentioned.
It should all be pretty standard.
when i try to connect via ws://yyyyy.cloudfront.net or even wss://yyyyy.cloudfront.net I receive error 403.
A portion of my response header report is here:
x-amz-apigw-id: TA01EHd5IAMF84w=
X-Amz-Cf-Id: f_6Ka5sqZsyiXXIRqstsBXEKmJLDPCiE24nVBrwnoR9wweOPzVGRGQ==
X-Amz-Cf-Pop: PMO50
X-Cache: Error from cloudfront

Of course i have also tried to indicate as origin name wss://xxxxxxx.execute-api.us-east-1.amazonaws.com and path / in this case my request would be made to ws://yyyyy.cloudfront.net/stage but the result would be identical.
I really don't know what else to try. I have activated logs for cloudfront, towards a logs S3 bucket, but there is nothing there apart for a generic "error" message. I have also activate cloudwatch logs for the API Gateway and whenever I make a request directly to  wss://xxxxxxx.execute-api.us-east-1.amazonaws.com/stage i can see that it is successful (as expected) but NONE of the requests that were made via cloudfront distribution at either ws://yyyyy.cloudfront.net/stage or ws://yyyyy.cloudfront.net are reported at all. At this point i start thinking cloudfront doesn't even talk with the api gateway, but whenever i try to contact the host with a simple wget ws://yyyyy.cloudfront.net i receive an Upgrade error which means, somwhere, somehow something that relates to the websocket is being actually triggered.
Please let me know if you know a way, any way, to get rid of SSL in the api gateway. I don't need to use cloud front. i just cannot use SSL on my IoT device because of memory constraints.

@faustomilletari It took me a while to get CloudFront to work with WebSocket. You can see a sample of the integration here which uses LambdaSharp notation, but it's close enough to CloudFormation YAML to follow: https://github.com/LambdaSharp/Chat/blob/main/Module.yml#L164
However, I will tell you that there is no way to connect over HTTP/WS. You have to use HTTPS/WSS. CloudFront will redirect any HTTP connection to HTTPS. It does not serve a HTTP connection. So if SSL is not supported by your device, you are out of luck with API Gateway. Instead, you will need to run something like an EC2 instance to act as a WS relay or server.

Thank you Bjorg. Your insight is helpful. I was indeed hoping to just do the same I have already done with a rest endpoint I had. In that case cloud front helped getting http without ssl layer. That said... I will look at the file you pointed and I will post here if I get it to work with either https or even http (which at this point I’m a bit discouraged about)
At this point I am almost about running EC2 instances with fastapi websocket which would probably be even better.
The thing is... lambda allows to scale in a way thy even eks struggles to do.
Thank you very much for your help. I appreciate it

Hey there,
I spent perhaps more than a day, but found the catch. You need to configure the Origin Request Policy properly. Here is an example for WebSockets : https://nickzamosenchuk.medium.com/configure-amazon-cloudfront-cdn-for-websocket-connection-43c44b3f877c
Cheers!

Would make send to create a new repo or add this example/functionality to the readme?

Hey there,
I spent perhaps more than a day, but found the catch. You need to configure the Origin Request Policy properly. Here is an example for WebSockets : https://nickzamosenchuk.medium.com/configure-amazon-cloudfront-cdn-for-websocket-connection-43c44b3f877c
Cheers!

Dear @nzamosenchuk
thank you so much for your guide! Unfortunately despite this guide my API gateway websocket continues to be unreacheable through CloudFront. Could you please tell me if are there other configurations to do?
Thanks!

Hey there,
I spent perhaps more than a day, but found the catch. You need to configure the Origin Request Policy properly. Here is an example for WebSockets : https://nickzamosenchuk.medium.com/configure-amazon-cloudfront-cdn-for-websocket-connection-43c44b3f877c
Cheers!

Just for sharing some general knowledge: this solution works because API Gateway does not want an Host header different from the one assigned to each endpoint (eg: xxxxxx.execute-api.eu-central-1.amazonaws.com ). Whitelisting websocket headers prevents the Host header (ie: your ClouFront distribution URL) to be forwarded, so CloudFront will use the API Gateway endpoint as value for the Host header.

@httpiga I confirm that solution works! The errors was due to a miss configuration on my side. 😅


Test PR for branch beta/v0.12.2007.1527 (will not be merged)

Using this PR to test generated IQ# API docs for beta build v0.12.2007.1527.

Docs Build status updates of commit faff909:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

api/iqsharp-magic/azure.connect.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/azure.execute.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/azure.jobs.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/azure.output.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/azure.status.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/azure.submit.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/azure.target.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/check_kata.md
:white_check_mark:Succeeded

api/iqsharp-magic/chemistry.broombridge.md
:white_check_mark:Succeeded

api/iqsharp-magic/chemistry.encode.md
:white_check_mark:Succeeded

api/iqsharp-magic/chemistry.fh.add_terms.md
:white_check_mark:Succeeded

api/iqsharp-magic/chemistry.fh.load.md
:white_check_mark:Succeeded

api/iqsharp-magic/chemistry.inputstate.load.md
:white_check_mark:Succeeded

api/iqsharp-magic/config.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/estimate.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/index.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/kata.md
:white_check_mark:Succeeded

api/iqsharp-magic/lsmagic.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/lsopen.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/package.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/performance.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/simulate.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/toc.yml
:white_check_mark:Succeeded

api/iqsharp-magic/toffoli.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/who.md
:white_check_mark:Succeeded
View

api/iqsharp-magic/workspace.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

@geduardo here are the generated IQ# magic command docs with your latest fix. Looks good to me, could you give it a sanity check when you have a chance?

Looks good to me as well @rmshaffer! Thanks!


Make the repository public

We should make it public roughly at the same time of uploading it to PSGallery (#2) , so everyone can use it and its source code together.

Done!


It always show Your Microfrontend is not here page

I have already follow the guide and override the path in https://react.microfrontends.app/ and refresh the page. it always redirect to our Microfrontend is not here page. Is there anyone know how to fix it?

Looks like root-config need to upgrade single-spa version to 6.0.2. Currently, it is  ^5.9.3

Fix released in webpack-config-single-spa@7 and upgraded in https://github.com/react-microfrontends/people/pull/44


Aliasing/saving credhub targets

As a platform operator, I would like to context switch between multiple credhub environments.
I often find myself needing to switch between credhub servers. Instead of having to log in and out of them, I would like a mechanism similar to how the bosh-cli has bosh -e <env> to allow switching between environments. Or how Concourse's fly cli has different targets to choose from
e.g.,
credhub -t ci get -n /concourse/blah/...
credhub -t sandbox-bosh-oss get -n /blah/.../
credhub -t dev-opsman get -n /blah/.../

Thanks for your time and consideration

That sounds like a good idea to me. ^ @crawsible

This does seem like a good idea. We would love to do this, but at this time it is not a priority. We are going to leave it open because we do this it would be a good thing to add if someone got the time.


improve wallet balance call behavior

Wallet balance should not be called again if there's a pending request. On web, we can also consider making this every 10 seconds or so to improve performance there.

This is pushed up on a branch that's running on lbry.tv


fix videohandler problem

fixes VIDEOHANDLER problem by swapping videohandler with vlc.MP4Handler bc hxCodec replaced videohandler with vlc.MP4Handler NOOooo

btw the main.yml is bugged idk how to fix it (this one and the original one)

a