GEP: Add Name to HTTPRouteRule and HTTPRouteMatch

What would you like to be added:
Add a name field to HTTPRouteRule and HTTPRouteMatch structs to allow users to identify different routes
Why this is needed:

Policy Attachment

Route Attachment/Delegation: Ability to have more fine grained matches for policy references IF we add sectionName to the policy object

Improved Debugging: Ability to add debugging information, such as adding the route matched to some access log

/assign @hzxuzhonghu

If I'm understanding correctly based on the closed PR, we've decided not to move forward with this at this time. If that's not the case, please feel free to drop a comment and we can re-open.

I think we decided this would be reasonable if we had a bigger use case associated with it. This will likely end up being included as part of route delegation in #1058.

Ok. The project state also says "abandoned" so we should probably update this too so this issue doesn't look as though it is in fact abandoned.

Ok, then let's consider this closed in favor of the implementation in https://github.com/kubernetes-sigs/gateway-api/issues/1058 :+1:

I think it makes sense in general, but will need careful writing of the spec - we can't add the new field as required, but what value should  be used in contexts where it's expected (like log messages or telemetry) if it's not set?

"", "-", "unknown", nil - not sure. It may be usage specific, some systems have different ways to describe "not present" I would think, which makes it hard to prescribe against them

@youngnick @howardjohn We would really appreciate having this in the next version of the HTTPRoute. We have a similar requirement to apply some policies per rule as mentioned in the description.
cc: @apalsule

Within the Kuadrant project, we also see value in adding this to support policy. Would really like to see it in the next version also

One of the most common arguments against this feature is that we really shouldn't be encouraging users to have large Routes. Routes with lots of rules are remarkably hard to deal with in cases like partially invalid config where many Route rules are valid, but one is not. We still have not solved this problem so the API only provides guidance for when a Route is invalid.
It seems like any request for attaching policies and other resources to a specific Route Rule name has to answer this key question: Why not just use more than one Route? Any Route that has 10 rules could also be equally represented by 10 Routes with 1 rule each, or 5 Routes with 2 rules each, or 2 Routes with 5 rules each, etc. I'm concerned that adding name to Route rules would encourage even larger Routes, which would only exacerbate other problems that we have not solved yet like partially valid Routes.

@robscott I think everything you said is true, but also if you have >1 route having a name is useful. So even for a small 2 route service its helpful.
If its such a concern, why do we even allow multiple routes at all?
Its very likely too late to change that so mostly a theoretical argument. But it does seem compelling that if you have multiple things, you ought to have a way to refer to them. Listener names, service port names, container names, etc all follow this pattern.

We currently have a maximum items set of 16 in the validation for rules across all the rule types, so I think we're managing the larger routes concern to some extent. I think I should be clear that we will be very reluctant to allow more than that (I actually think 16 is probably too large as it is, but we can't make it smaller without an API version bump).
That said, I think I'm in agreement with @howardjohn here - it is useful for things to have a name field if there's a list.
However, versioning means that we have the following things to address in this proposal:

The new name field must be optional, or it's a breaking change.
Because the field must be optional, we need to define cases like:

What implementations should do with empty name fields for logging. Leave the field out, leave it empty? Doesn't really matter what we choose, we just need to choose something.
What happens when something (like a Policy or whatever) refers to a name field that doesn't exist? To me, this is a ResolvedRefs error, the same as if the object as a whole did not exist. But again, we need an answer.

As long as we carefully handle the implications of this field being optional, I think it's okay to proceed.

My 2 cents:

The new name field must be optional, or it's a breaking change.

I think this is unfortunate but understandable.

What implementations should do with empty name fields for logging. Leave the field out, leave it empty? Doesn't really matter what we choose, we just need to choose something.

Leave it empty would be my preference.

What implementations should do with empty name fields for logging. Leave the field out, leave it empty? Doesn't really matter what we choose, we just need to choose something.

What happens when something (like a Policy or whatever) refers to a name field that doesn't exist? To me, this is a ResolvedRefs error, the same as if the object as a whole did not exist. But again, we need an answer.

Agree again. As I have a particular interest in policy, for me the name should likely become part of the target ref as called out in the future work section of the policy attachment GEP. For me the all of the target ref in a policy has to be validated before the policy is applied. So it would be invalid to have a policy with a target ref that matched the resource but had an invalid sectionName ref.
My 2 cents

@swathinsankaran

We have a similar requirement to apply some policies per rule as mentioned in the description.

Could this potentially be done by using a filter referenced from a routing rule?

@pleshakov Yes, what we are looking for is this GEP-713 to attach the policy per HTTPRouteRule and may be policy per backendRef (like policy to reencrypt traffic).
I have observed that GEP is not progressing, and this issue being a subset of the GEP, will help us going forward with our implementation.

@pleshakov A filter currently cannot be provided by a 3rd party. With kuadrant we are building out a set of policy APIs that hope to provide to many gateway providers. Policy Attachment allows this type of set up, whereas a filter has to be provided by the gateway provider AFAIK

Right now, there's #2128 to add some clarify to Policy Attachment's discoverability, but I agree that this would be nice to address soon.
#2128 is also going to have a bunch of other followup actions, so this issue will be in good company.
Also, I don't think we explicitly disallow third party filters right now, we just had never considered them - I suspect that they will be heavily dependent on the underlying dataplane though, as the Gateway implementation will need to be able to direct its underlying dataplane to send traffic to a third-party filter somehow. That would need configuration somehow, which we also haven't though about yet.
Hmm, reading over that last paragraph, I agree Policy Attachment is probably easier. 😄
All of that said, @swathinsankaran, I'm going to remove the status from this, and we can figure out if it needs to be in 1.0 or not.
/remove-lifecycle rotten
/lifecycle frozen

@youngnick Wondering if this has been discussed further for 1.0? If there is anything I can do to help progress it let me know

We triaged this for GA and we think it's indeed not something that needs to block GA, but also we don't think it's actually ready for someone to pick up:
/label-remove help
We expect more conversation and considerations are needed before we can start taking action on this one:
/triage needs-information

It turns out not having the name field available in the HTTPRouteRule (and perhaps also in the HTTPRouteMatch) has been biting us in more ways, due to the CEL validations in the HTTPRoute (v0.8+).
Kuadrant policies (based on Policy Attachment, GEP-713) support associating parts of a policy spec ("policy rules") to specific sections of the main targeted network resource – particularly to HTTPRouteRules, when a HTTPRoute is the target.
Because HTTPRouteRules cannot be referred by name, our solution was to define matches []HTTPRouteMatch fields in our policy APIs, which we use to lookup for HTTPRouteRules that declare identical matches, to associate the policy rule portion with. We call these structures "route selectors" (which also include hostnames.) A typical Kuadrant policy can contain multiple rules, with multiple route selectors each. See example in a Kuadrant RateLimitPolicy and in a Kuadrant AuthPolicy.
This would work fine until v0.7, but since validation rules based on Common Expression Language (CEL) have been added to the HTTPRoute (https://github.com/kubernetes-sigs/gateway-api/pull/2253), the estimated cost limit exceeds the thresholds, and we have been struggling with having to set some hard constraints on the number of policy rules a user can declare.
In particular, these HTTPRoute validations have been flagged as the ones adding most to the estimated cost of our APIs: https://github.com/kubernetes-sigs/gateway-api/blame/14197584076cbc6853caf4a6cef29d6d5810595c/apis/v1beta1/httproute_types.go#L193
A much better alternative – that would not only fix the issue of the estimated complexity of the imported validation rules, but also improve assertiveness in how those associations between policy rules and HTTPRouteRules ("HTTPRouteRule refs") are declared – would be having name fields available in the HTTPRouteRule type.

For default values: "1", "2"...
Or match-1 if we want to avoid confusion with array syntax.
Since most routes have few matchers - just allowing attachment with an array syntax or the default names would work very well with no change to the route.

This is starting to feel more useful with steps towards standardizing support for partial validity of routes in https://github.com/kubernetes-sigs/gateway-api/pull/2429 and could greatly help clarify corresponding error messages.
Would strongly prefer @youngnick and @maleck13's suggestions that unset optional names be empty/nil and a mismatched sectionName in policy targetRef should be a ResolvedRefs error - array-based indexing would be too easy to accidentally switch targets of a policy when adding or removing a rule from a route.

@guicassolato you recently discussed your strong need and use case for this, are you interested in being one of the assigned people and to try and move this forward by starting a GEP?
We would ask the first pass at this be focused entirely on the WHAT and WHY and try to stay away from the how (e.g. no API design in the first PR) so we can build consensus on what it is and why we want to do it, but not get lost (yet) in the how we do it.

@shaneutt, absolutely yes. I think @maleck13 may have brought the topic to the GAMMA meeting today. The two of us will sync up tomorrow and proceed with the next steps to put the GEP together asap. Thank you!

Awesome! Thank you! Let us know if you get stuck or need help.
/assign @guicassolato

Opened the draft PR with TLDR, a few goal/non-goals and brief summary of alternatives considered for now. #2593
cc @shaneutt @maleck13

/remove-lifecycle stale

/remove-lifecycle-stale

/remove-lifecycle rotten

/remove-lifecycle stale

Can this be closed now that #2985 has been released in v1.2, or is there any more to do here?


usage.adoc in asciidoctor format: first version

Hi Andres,
This is from the hackergarden last night: added the contents to usage.adoc
Cheers,
Oleksiy

thank you 😃


Update agent spec with current semconv

There are a few notes in here: https://github.com/elastic/apm/tree/main/specs/agents/mobile#additional-span-attributes which are out of date: specifically session.id.

https://github.com/elastic/apm/pull/844


Improvement: User-friendliness

ability to change bg size in data
ability to choose filepath for converting vigilance to json

Added .setPos
Added .setSize
Added .apply
Changed example.js based on new additions


Enum Request: SalesDocumentNoIsVisible(DocType: Option Quote,"Order",Invoice,"Credit Memo","Blanket Order","Return Order",Reminder,FinChMemo) Codeunit 1400 "DocumentNoVisibility"

Hi,
We need the possibility to extend the parameter "DocType" in the "SalesDocumentNoIsVisible" function of the Codeunit 1400 "DocumentNoVisibility".
Thank you.

Hello! Do you plan to implement additional sales document type? Should we consider using enum "Sales Document Type" here in this case?

Hi,
We would like that the parameter "DocType" in the "SalesDocumentNoIsVisible" function to be an Enum and not an option, to avoid compatibility errors, passing a wrong enum, since option is marked as deprecated
thank you

No, this is not the case. Option data type is not deprecated. Both option and enum will continue. If this is the only purpose, no need to make this change


Space (or any non-encoded character) in route segment causes a hydration mismatch in Form action

What version of Remix are you using?
1.13.0
Are all your remix dependencies & dev-dependencies using the same version?

[X] Yes

Steps to Reproduce

Use space in a route segment e.g. /post/hello world
Use a Form in that route

Expected Behavior
The encoding or url should match on the client and the server and there should be no warning.
Actual Behavior
I get a hydration warning Warning: Prop action did not match. Server: "/post/hello world" Client: "/hello%20world"

This is resolved by https://github.com/remix-run/react-router/pull/10769 and should be available in Remix v2


Please ignore, wrong target.

2 things changed essentially:

make the global indexing part synchronized instead of asynced. This will add a few seconds to the total time but frees us from the need to wait for definition-added event to complete.
avoid re-opening/re-loading a same document's content in definition queries. We could do better with memory usage, but I think at this moment it works for the purpose.

Sorry, wrong target ...

Makes me wonder, what was the target?

this: https://github.com/lambdalab/php-language-server/pull/1


Feature/1915/update cwl tool version

For ga4gh/dockstore#1915

Updates the CWL dependencies.
Partly fixes the issue with the tests calling the Dockstore production webservice.  The issue is that when the test has no config file or a config file does not have a server-url specified, it will by default set it to production.  To fix this, some tests now have a correct config file with server-url or it has --script flag which skips checking the dependencies

Coverage increased (+7.6%) to 74.727% when pulling 83da74137f0515da67f88bf13977a12a33fe340a on feature/1915/updateCWLToolVersion into db7890a5b46234612fbf6c8ff40e06a7c1f3e397 on develop.

Coverage increased (+7.6%) to 74.727% when pulling 83da74137f0515da67f88bf13977a12a33fe340a on feature/1915/updateCWLToolVersion into db7890a5b46234612fbf6c8ff40e06a7c1f3e397 on develop.


[Epic] Improved Deployments

Objective
Simplify the following modes of deployment

Docker
K8s
Heroku
AWS

Success Metrics
List of all metrics you are tracking and the desired goal.

Goal
Metric

Improved Installation Funnel
95%

Better Telemetry Capture Rate
50%

Requirements

Requirement
User Story
Importance
Notes

Docker
The user should be able to deploy appsmith with a single docker command on any OS. The container is rebuilt whenever a change to the frontend or backend occurs
High

Docker
The user should be able to bring up appsmith with a docker-compose file/run command
High

K8s
The user should be able to deploy on Kubernetes with a helm chart. We should have a public chart repo for appsmith with the default release
High

AWS
The user should be able to deploy on AWS with a single click using the AWS stack
High

Heroku
The user should be able to deploy on Heroku with a single click after setting up mongo with no additional configuration required
High

UI Config
The user can configure all environment variables via a UI.
High

SSL
The user should be able to install SSL via a single CLI command for all modes of deployment
Medium

Logs
The user should be able to view the logs of the container with a simple docker logs command
Medium

Export
The user should be able to easily take a dump of the instance and restore it
Medium

Inspiration
Metabase
User Story
Users self-hosting appsmith should be welcomed first instead of being thrown onto a login screen. They should proceed to fill in the details of the administrator of the instance.

Name
Email
Password
The Intended Use of appsmith
Personal / Company
Company Name
They can choose whether they are deploying appsmith locally or on a cloud server.
Agree to anonymous share usage statistics of their instance.
Subscribe to our newsletter for updates about appsmith
The user is instantly logged in

The user can now update the instance settings from the administrative section.
This section allows users to configure their appsmith instance

General

Instance Name
Admin Email
Generate Docker Compose File
Option to enable / disable anonymous tracking
Option to connect a custom domain & SSL

Email

SMTP Host
Port
SMTP Security - None | SSL | TLS | START TLS
Username
Password
From Address
Links to guides on integrating with Sendgrid / SES

Version

The current version of appsmith
The release notes

Authentication

Form Signup

Enable / Disable (default)

Google Signup

Client ID
Allowed Domains
Link to guide to configure

Github Signup

Client ID
Link to guide to configure

Google Maps

Google Maps API Key
Link to guide to generate API Key

Developer Handoff Document in Figma
Link to the developer Handoff Document:

Can we add monitoring to this? The self-hosted setup could come with an initial monitoring system as well (optionally), that will monitor the server, frontend, mongo etc. in one single place.

WIP Figma File
https://www.figma.com/file/lqtUfzJO50RpEhxKttYeOg/Local-Dev?node-id=1%3A70

New Comment by a user
embedded mongo db password in docker-compose.yml poor choice.  Should go in an environment file that can be given stricter permissions

Opened #6345 for issue from above comment.

Admin Configuration Fields

Display Name
Type
Internal Name
Detail

MongoDB URI
Text
APPSMITH_MONGODB_URI
The URI to a MongoDB server, that will be used as the database.

Redis URI
Text
APPSMITH_REDIS_URL
The URI to a Redis server, that will be used for storing sessions and other ephemeral information.

Enable Email
Boolean
APPSMITH_MAIL_ENABLED
If true, email is enabled, and needs to be configured, using the other fields here. If false, Appsmith server won't be able to send any emails.

Email: From Address
Email Address
APPSMITH_MAIL_FROM
The email address to use as the from address when sending emails.

Email: Reply To
Email Address
APPSMITH_REPLY_TO
The email address to use for the Reply-To header for emails sent by Appsmith.

Email: SMTP Host
Text
APPSMITH_MAIL_HOST
The host address of the SMTP server to use.

Email: SMTP Port
Number
APPSMITH_MAIL_PORT
The port of the SMTP server to use.

Email: SMTP Authentication
Boolean
APPSMITH_MAIL_SMTP_AUTH
Enable authentication, when talking to the SMTP server.

Email: SMTP Username
Text
APPSMITH_MAIL_USERNAME
Username to use for authenticating with the SMTP server, if authentication is enabled.

Email: SMTP Password
Text
APPSMITH_MAIL_PASSWORD
Password to use for authenticating with SMTP server.

Email: Enable TLS for SMTP
Boolean
APPSMITH_MAIL_SMTP_TLS_ENABLED
Whether to use TLS when authenticating with SMTP server, if authentication is enabled.

Signup: Disable
Boolean
APPSMITH_SIGNUP_DISABLED
If true, signing up on this Appsmith instance will not be allowed.

Signup: Allowed Email Domains
Boolean
APPSMITH_SIGNUP_ALLOWED_DOMAINS
When set to a comma separated list of domains, only emails with these domains will be allowed to signup.

Google ReCaptcha Site Key
Text
APPSMITH_RECAPTCHA_SITE_KEY
Provide ReCaptcha Site key here, if you wish to use it for the signup page.

Google ReCaptcha Secret
Text
APPSMITH_RECAPTCHA_SECRET_KEY
Provide ReCaptcha Secret key here, if you wish to use it for the signup page.

@Nikhil-Nandagopal, regarding the item Generate Docker Compose File, instead of generating just a docker-compose.yml file and having that download, should we give a zip with the docker-compose.yml file and the docker.env file, with the current env variables? The docker-compose.yml file would have a reference to this docker.env file, so I'm guessing just the yml file wouldn't be of much use.
Additionally, if the idea of providing this is to be able to export the whole environment in one click, to reproduce elsewhere, shouldn't it include the database dumps as well?

Currently when deploying using k8s, there is no health probe. Sometimes connection to mongo or redis may end, but since no health route exists, there is no way for the pod to be restarted.

Hey @heroic, you mean a health endpoint on the backend server, right? One option is to use /api/v1/users/me, and confirm it responds with 200 status code, and perhaps that the JSON body contains anonymousUser.
Let me know if I misunderstood your question.

@sharat87 the zip sounds good. The idea for this was quick replication for the DB admin. I think with git sync the need for a DB dump will go away. Only the users will need to be re-added which should be fine once we create domain wide sign ups & invite links

Currently when deploying using k8s, there is no health probe. Sometimes connection to mongo or redis may end, but since no health route exists, there is no way for the pod to be restarted.

Hey @heroic, you mean a health endpoint on the backend server, right? One option is to use /api/v1/users/me, and confirm it responds with 200 status code, and perhaps that the JSON body contains anonymousUser.
Let me know if I misunderstood your question.

Yes. I ended up using /applications

@sbalaji1192, here's the variable names for some of the configs:

Authentication

Form Signup

Enable / Disable (default) (APPSMITH_SIGNUP_DISABLED)

Google Signup

Client ID (APPSMITH_OAUTH2_GOOGLE_CLIENT_ID)
Allowed Domains (APPSMITH_SIGNUP_ALLOWED_DOMAINS)
Client Secret (APPSMITH_OAUTH2_GOOGLE_CLIENT_SECRET) 🆕
Link to guide to configure

Github Signup

Client ID (APPSMITH_OAUTH2_GITHUB_CLIENT_ID)
Link to guide to configure (APPSMITH_OAUTH2_GITHUB_CLIENT_SECRET)

Google Maps

Google Maps API Key (APPSMITH_GOOGLE_MAPS_API_KEY)
Link to guide to generate API Key

Currently when deploying using k8s, there is no health probe. Sometimes connection to mongo or redis may end, but since no health route exists, there is no way for the pod to be restarted.

Hey @heroic, you mean a health endpoint on the backend server, right? One option is to use /api/v1/users/me, and confirm it responds with 200 status code, and perhaps that the JSON body contains anonymousUser.
Let me know if I misunderstood your question.

Can I add this liveness probe in the backend deployment file. My plan is to setup a http request probe to the /api/v1/users/me path with a period of 5 seconds.

Currently when deploying using k8s, there is no health probe. Sometimes connection to mongo or redis may end, but since no health route exists, there is no way for the pod to be restarted.

Hey @heroic, you mean a health endpoint on the backend server, right? One option is to use /api/v1/users/me, and confirm it responds with 200 status code, and perhaps that the JSON body contains anonymousUser.
Let me know if I misunderstood your question.

Hey ,can I send a PR to add this liveness probe in the backend deployment file. My plan is to setup a http request probe to the /api/v1/users/me path with a period of 5 seconds.

@DiptoChakrabarty Sure. Please go ahead.

@sharat87 are the same events being fired for when telemetry is opted in / out?

Hi, in the latest blog post Helm support has been announced, but I cannot find the docs for it

Hey @jalberto, you can checkout https://github.com/appsmithorg/appsmith/tree/release/deploy/helm where we have some documentation for this. We are working on including this in our official docs.appsmith.com soon. Thanks!

Action Items:
@sharat87 -- Review and merge helm charts, infra compliance changes.
@sub1983 -- Work with GeekUP for IaC changes.

Closing this.

@hiteshjoshi I still see 10 open issues in the epic. What are we doing about those?

@Nikhil-Nandagopal They will still be open, the problem is DevOps is being tracked on Notion + EE repo due to a lot of restrictions. We will close them but the priority have changed a bit for some of the open issues.


Why not add a library for JSON transformation?  Like Jackson ,fastjson ,Gson

If I want to send json，I need to use  "javax.json.JsonObject"  !!!,We usually don't take this approach。As for me,I often use some librarys like Jackson,fastjson,Because we usually take a java bean convert to a json string !!!

We currently only support JSONP (aka JSON Parsing API) in Helidon SE.
Support for a JSON Binding framework is a new feature.
If you want to implement this yourself, you can start with this file: https://github.com/oracle/helidon/blob/master/webserver/json/src/main/java/io/helidon/webserver/json/JsonSupport.java
I did a few experiments, see this gist: https://gist.github.com/romain-grecourt/d2b04178140f7eab3b69f8ea2481fc10

We currently only support JSONP (aka JSON Parsing API) in Helidon SE.
Support for a JSON Binding framework is a new feature.
If you want to implement this yourself, you can start with this file: https://github.com/oracle/helidon/blob/master/webserver/json/src/main/java/io/helidon/webserver/json/JsonSupport.java
I did a few experiments, see this gist: https://gist.github.com/romain-grecourt/d2b04178140f7eab3b69f8ea2481fc10

Thanks,I ‘m very much looking forward to it !!!

Maybe I misunderstand the question. But if you take the quickstart-mp example, you can add something like this:
@Path("/dogs")
@ApplicationScoped
public class DogResource {

  @GET
  @Produces(MediaType.APPLICATION_JSON)
  public Set<Dog> dogs() {
    final Dog dog1 = new Dog();
    dog1.name = "Falco";
    dog1.age = 4;
    dog1.bitable = false;
    final Dog dog2 = new Dog();
    dog2.name = "Wastl";
    dog2.age = 2;
    dog2.bitable = true;
    return Stream.of(dog1, dog2).collect(Collectors.toSet());
  }

  @POST
  @Consumes(MediaType.APPLICATION_JSON)
  public Response bite(final Dog dog) {
    return Response.ok().build();
  }

}

$ curl 'http://localhost:8080/dogs' -H 'Content-Type: application/json' -H 'Accept: *'  -d '{"age": 4,"bitable": false,"name": "Falco"}'
$ curl 'http://localhost:8080/dogs'
[{"age":4,"bitable":false,"name":"Falco"},{"age":2,"bitable":true,"name":"Wastl"}]

So from my point of view, there is no need to add more binding libraries.

This is about support in the webserver directly. I.e this is for helidon-se, not helidon-mp.
In the case of helidon-mp, jsonb is can be used as-is inside a jaxrs resource.

Hi there,
I have read thru the document.
Is there any elegant way we can do the object transformation using helidon-se? Perhaps using helper or sth?
Thanks!!

I have a use case where the json I receive has a uses a complex key as a key in a map. We need the ability to register a deserialiser for it

@gcameo This is something you can do today by registering your own Reader in the request.
See how the JsonSupport is implemented:  https://github.com/oracle/helidon/blob/6ddb6c479bf61d8d857e2304b31a5ce79666fe57/webserver/json/src/main/java/io/helidon/webserver/json/JsonSupport.java#L148

With #351 and #387 now implemented, this issue is resolved.


hasSession does not appear in docs despite being public

The method hasSession is pretty important for login, but doesn't appear in the docs despite being public.

Workout why typedoc isn't including it
Add docs explaining what it is for

This is documented as a module, not on the object (because it isn't on the object!)
https://developers.verida.io/docs/api/verida-js/modules/verida_account_web_vault#hassession


What is the timeline for code release?

Can you tell me if you're planning on releasing the code?

The inference code is present in demo.py and in the Colab notebook.
To train your own ControlNet for Sketch-a-Sketch 1) run HED edge detection on a text-image dataset (I used 50000 images from https://huggingface.co/datasets/laion/laion-art), 2) vectorize using https://github.com/MarkMoHR/virtual_sketching, 3) delete random subsets of strokes from the vectorized image, and 4) use the standard diffusers ControlNet training script (https://github.com/huggingface/diffusers/blob/main/examples/controlnet/train_controlnet.py).
I'll put the dataset itself on HuggingFace soon. Not sure yet when I'm releasing the actual dataset-creation script.


AKS Node patching Conflicts in Docs

As per current document it states security patches get automatically applied to nodes by default each evening, in similar document its mentioned that doesn't automatically patch them by default. Because AKS isn't a platform-as-a-service (PaaS), components like agent nodes have shared responsibility, and users must help maintain the AKS cluster.
Reference link :https://learn.microsoft.com/en-us/azure/architecture/operator-guides/aks/aks-upgrade-practices#node-image-upgrades
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: e0fcc83c-c911-4d82-98af-ed8880ca6c4a
Version Independent ID: 00462d96-9a30-8cef-5932-c99bab237706
Content: Concepts - Security in Azure Kubernetes Services (AKS) - Azure Kubernetes Service
Content Source: articles/aks/concepts-security.md
Service: container-service
GitHub Login: @miwithro
Microsoft Alias: miwithro

@Osshaikh
Thanks for your feedback! We will investigate and update as appropriate.

@miwithro
Can you please check and add your comments on this request as applicable.

@Osshaikh, Thanks for the feedback! I don't believe these statements are in conflict. You're correct that maintaining agent nodes is a shared responsibility, but the responsibility from the user is to reboot nodes after the patches are automatically applied by AKS. So, it is true that patches do get automatically applied, but because there is a user requirement to reboot nodes to complete the process, it is still considered a shared responsibility.
I hope that clears things up, if you still have further questions feel free to re-open and tag me.
#please-close


required should be configurable rather than attribute only

required attribute should be configurable as a bool rather than as a single attribute declaration.
<md-pickers required="config.required" ...

for example.

Can you please elaborate?

@gigablox Did you try using ng-required directive?


Copilot REST API blog posts

@asafashirov  we need your SEO input on this.

@arturl For the launch blog post, you should probably direct them to the docs first (https://www.pulumi.com/docs/pulumi-cloud/copilot/api/), samples second (https://github.com/pulumi/copilot-api-samples/tree/main/samples), then submitting an issue or asking a question on Slack.
@adamgordonbell On the engineering blog post, the picture of Simon isn't showing up. You may need to add that.


Update Unity part 2

Sequel to #194

Changed binary to lfs by default, as discussed
Added back in the eol=lf attribute for YAML files that I replaced with text in the last PR by mistake
Excluded the third-party plugins folder from GitHub stats (Assets/Plugins). There's one issue with this however. Users need to actually put their plugins into this folder. For example, TextMesh Pro gets imported directly under Assets by default, and some people just leave it there. Storing these third-party assets under the Plugins directory is a popular convention that some assets in the Asset Store do by default (e.g. DOTween), so I only included this folder so we don't have to specify popular plugins in the gitattributes one by one. Also, since TextMesh Pro's default folder name has a space in it, it would have to be defined like this: Assets/TextMesh[[:space:]]Pro/** linguist-vendored, which just looks bad imo.

Thank you!


HTTP: resources Session updates are lost but the last one

Hi, in v.3.0 I encountered the following behavior:
If after making a request which is defined in .resources block some data is extracted, it's impossible to use it in the next request, outside .resources  block, because gatling can't find it in a session.
The structure is like that:
	val req1 = http("req1")
		.get("/req1")

	val req2 = http("req2")
		.get("/req2")
		.check(jsonPath("$.foo[0].bar[0]).find.saveAs("param"))

	val req3 = http("req3")
		.get("/req3")
		.resources(
		    req1,
		    req2)

	val req4 = http("req4")
		.get("/req4?${param}")

	val tst = scenario ("tst")
             .exec(req3)
             .exec(req4)

When running a test, when .exec(req4) is performed I get the message: Failed to build request: No attribute named 'param' is defined. However, the "param" variable was extracted successfully, I can see it in session data of req2.
I'm not quite sure, but after serios of experiments I can make an assumption that this error occures only if req1 duration is more then duration of req2. In this case, when gatling trying to execute req4 it searches the variable in session data of req1. If left only req2 in .resources section everything is ok.
In v.2.3 I can't reproduce this behaviour.

Thanks for reporting!


Update SQLite sample app

Use the latest version of SQLite related packages
Remove the dependencies of Tizen.Wearable.CircularUI package

LGTM

LGTM


Refactor with rope does not adhere to files.exclude

Steps to Reproduce:
I have a project that is used on many Python versions, with the CPython test suites in the workspace.
Renaming a symbol with Python extension is giving a rope error. Excluding the Python2 only files from test suites, didn't help at all.
I am conviced that files.exclude is not respected when repo refactors, but it should be. Alsoit would be nice to specify a list of files that are never subject to refactoring. I e.g. also have syntax error test cases that will never work, but should be in the workspace.
I can provide a link to a problematic Python workspace if that helps, but I don't want to spam you with my project link.
Yours,
Kay

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on then we will be happy to re-open this issue to pick up where we left off.


Create additional info blocks

Make your resume less official by adding more entertaining info, like your hobbies and interest, or what do you like, some interactive content or joke, easter egg, etc.

Hobbies and interest:
Cooking and baking
Solving puzzle games
Reading books and articles
Skills
HTML
CSS
JavaScript
React.js
Redux
TypeScript
Nest.js
MySQL
MongoDB


Support for dbbackup of database in Docker container

@bhandariroshan
I often think about the following thing:
To have a standalone Django project allowing to backup (or restore) without change my settings.py
I don't think it is really in DBBackup's domain, but more a way to design your architecture.
If you have a great Docker image for DBBackup, do not hesitate to submit us, we'll add it to the organisation and the integration documentation

This one is up to you, I did it myself.
You can use the file system storage and mount your backup folder into your container.
docker-compose run --rm web python manage.py [backup command here]

Could you give us an URL?

I think I used this one when I was getting started with Docker and Docker-compose
https://docs.docker.com/compose/django/

OK, let me explain exactly that I have in head:

I think developers will create their own Docker image of their apps
We must provides them a Docker image acting like manage.py, with project settings or specific to the backup action
The DBBackup image could contain a wrapper to easily dump/restore database/media

I don't think docker-compose will be helpful: We need to launch a thread for our action and let it die. No daemon here. We should create a single container linked to the database one(s).
Maybe you don't see this like that at all ?

We I run docker-compose run web --rm python manage.py dbbackup I will have the backup in my local folder.
This is what I wanted to know ...

I think it doesn't make sense to have a docker image just for the backups, if you are using docker with your project, you should be able to access to manage.py

While I develop this app, I always think about huge companies with separated team, They could have a backup team which mustn't (or can't) have access to the project code. Instead they could have a separated light project with other configuration and specific tools, for example: Have different database credentials, dump/restore specific tools or else.
Developers may don't want admin to have every manage.py actions.
So it makes sense to have a container for backup only

Thanks everyone, I did my own implementation. Here's what I did:
a.  Created a pgdump inside the container
backup_command_name = sudo docker exec -i <container_name> + bash -c  'pg_dump -U <db_user>  <db_password> > <file_name> '
os.system(backup_command_name)

b. Moved it outside the container to local file system using docker cp
 copy_command_name = docker cp <container_name>:<file_name> <path_to_local_folder>
os.system(copy_command_name)

c.  Finally used dropbox ibrary to sync it to my dropbox.

@bhandariroshan : Thank you for this example.
I'm looking something more "Djangonic", using entirely DBBackup mechanism.

@ZuluPro look forward to it .. it would be awesome..

I don´t see how this is possible unless pg_dump is available in the application image.

@busla I think the best way is the following:

We create a base image for Django-DBBackup
Maybe create sub images (MySQL, Postgres, etc) from base
Users use one these images with its settings file or env vars

What do you think about ?

I think the problem at hand is not related to django-dbbackup at all. When isolating services into images (app, db, nginx, redis, et.c) and keeping the images as lean as possible (separation of concerns) the compromise is the range of tools we normally have access to.
If django-dbbackup will use pg_dump the tool needs to be available in the image, like you suggest, as a merged child image.
Everyone has their preference on how their application images is built so I think offering a django-dbbackup image might not get any attention. But who knows.
Perhaps we could look at it from the database side. Extend the DB images (postgres, mysql, etc) and link to the application container from there. Somehow relay commands from the application container to the database container?

I don't think we should use database images. They are here for server, not client.
In the case of AWS RDS this image doesn't exist.
I think about a real standalone image with nothing more than the requirements to connect to database and dbbackup. No need to inherit from user's project, just need an appropriate settings.py.

And such a beautiful solution almost turned out...
pg_dump: server version: 11.4; pg_dump version: 9.6.8

pg_dump: aborting because of server version mismatch

until I realized that both locally and in production I can't make it work using Docker.
any ideas?
it seems that you can share a single pg_dump file inside the container with the application, but I'm not sure that this is enough

Finally got it work!!!!
Here's my Dockerfile
`FROM python:3.8.2-alpine3.11
RUN apk add --no-cache
postgresql-client
postgresql-dev
pkgconfig
git
gcc
openldap
libcurl
python3-dev
gpgme-dev
libc-dev
set work directory
WORKDIR /usr/src/app
set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
copy project
COPY . /usr/src/app/`

@bhanduroshan, It looks like you've resolved your issue. It's unclear if there is anything that we should be doing / add to this project to make things easier. If you feel like something concrete can / should be done we can re-open this issue.

Finally got it work!!!!
Here's my Dockerfile
FROM python:3.8.2-alpine3.11
RUN apk add --no-cache \
    postgresql-client \
    postgresql-dev \
    pkgconfig \
    git \
    gcc \
    libcurl \
    python3-dev \
    gpgme-dev \
    libc-dev

# set work directory
WORKDIR /usr/src/app

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt

# copy project
COPY . /usr/src/app/

Finally got it work!!!!


Add Connexion for Python

Add the Connexion framework for Python

@mfornos Could you check this pull request?


How has Legend been used to standardize data models in the industry?

As Legend Studio was being open sourced, delivering value for the industry was important and therefore connecting with ISDA and the CDM was a natural expansion of already established engagement. This led to two working groups being set up as part of the pilot – both committed code to the golden source of the CDM:
• FX Options: the Averaging Model developed by the FX Options workstream was proposed back to ISDA, and has been approved and released in the CDM (see 2.82.8).
• Commodity Reference Data: especially in the context of a Fixed vs Float swap, including defining a payout model which has been approved and released in the CDM.
Legend is also being used in the FINOS Reference Currency workstream building a standard model for currencies and the related data.
There are a few other projects that are being explored and will be proposed through FINOS in the coming months – please look out for proposals here: https://github.com/finos/community/issues/new/choose and please do provide other challenges that may be tackled through collaborative modelling in the same place.


Migrate MultisigTransaction to zod

Summary
This migrates the validation of MultisigTransactions to zod.
Changes

Replace confirmationSchema with ConfirmationSchema
Replace multisigTransactionSchema with MultisigTransactionSchema
Infer types from the schemas and propegate them.
Add appropriate test coverage for the above

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

21 of 21 (100.0%)  changed or added relevant lines in 6 files are covered.
2 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.06%) to 92.798%

Files with Coverage Reduction
New Missed Lines
%

src/routes/transactions/entities/tests/human-description.builder.ts
1
90.0%

src/routes/transactions/mappers/common/transaction-info.mapper.ts
1
73.98%

Totals

Change from base Build<PHONE_NUMBER>:
-0.06%

Covered Lines:
6691

Relevant Lines:
6947

💛  - Coveralls

If count is nullable why did we have to change the count from null to 1 here? 🤔

It won't let me respond to this. It was a poor invalid value. It won't validate a string. I changed it to "invalid" in d8857aa928a976a49f366a3391f7ba5de4a985e8 for clarity.

If count is nullable why did we have to change the count from null to 1 here? 🤔

It won't let me respond to this. It was a poor invalid value. It won't validate a string. I changed it to "invalid" in d8857aa for clarity.

Oh I see. Thanks for clarifying! Now I understand that it was the difference between a valid number vs a numeric string 👍 Thanks!


Problem: broken link on crates.io

Solution: point docs.rs/emerald-rs to docs.rs/emerald-core

Can you also add create.io bunge to README.adoc? A link to https://crates.io/crates/emerald-cli.


Does Not Run In Wine/Proton

Proton 5.0 running through steam on Ubuntu Budgie

Fixed the Mono problem. However, still gives a FileNotFound error.
Steps to reproduce:

Create a wine64 prefix in the default directory.
Open a new terminal and run the commands in the code block in this comment.
Run winetricks dotnet472 It might give an error or two. Follow the install buttons and wait until the terminal gets to a new prompt.
Download VRCX and copy to ~.wine/drive_c/users/[YOUR_NAME]/Downloads/. Choose a different name for the destination file.
Run winetricks winecfg and ensure that the version is set to "Windows 10", add version to the overrides and confirm.
Run wine64 wineconsole You are now in CMD.
Type into CMD C:/windows/system32/explorer.exe and press Enter.
In the window that pops up, click on "My Computer" and navigate to your Downloads directory in the C drive. Double-click on the VRCX exe file that you have chosen the name for.
Follow the installation prompts.
Still in Explorer, navigate to C:/Program Files/VRCX/ Run VRCX.exe The error below should pop up.

Give this guide a try if you haven't already. https://web.archive.org/web/20220611004115/https://github.com/RinLovesYou/VRChat-Linux/wiki/VRCX

Had to fix some configuration issues on my end, and the provided solution worked.
Here's what I had to do:

Install laptop vendor drivers for Ubuntu
Set my graphics device in nvidia-settings to only the NVIDIA GPU.
Reboot

Note that the issues I had with creating Wine Windows in 7.0 should appear only if you have a laptop that uses hardware muxing with an Intel iGPU and the iGPU is set to be at least partially used.

@Flimsy-Fox
/ Off-side comment, but your explanation on how to fix the bug made me think of this https://twitter.com/Iinux/status/1570564762865651713


Logs sent to collector, but not seen in X-Ray (open-telemetry 0.0.17)

Hi,
First, let me say I'm super-excited about the potential of this project, and thanks to the contributors for their efforts to date. As a PHP developer running on AWS Lambda, I'm seeing this as a fantastic opportunity to fill an important gap.
I'm currently trying to get open-telemetry running on PHP on Lambda without success.  I am hoping someone here can help me get to the bottom of it.
Environment
I'm running Bref on AWS Lambda with php 7.4 (yeah, I know) and open-telemetry 0.0.17, as follows:
    "open-telemetry/contrib-aws": "0.0.17",
    "open-telemetry/opentelemetry": "0.0.17",
    "open-telemetry/opentelemetry-php-contrib": "0.0.17",

Changes from the example
I notice that the latest 0.0.17 of open-telemetry has some changes since 0.0.14 that make the examples break, so I had to change a few things.
The line:
        $spanProcessor = new SimpleSpanProcessor(new OTLPExporter());

no longer works because OpenTelemetry\Contrib\OtlpGrpc\Exporter has been removed.
I replaced it with this, which I think this is equivalent.
        $transport = (new GrpcTransportFactory())->create(
            'http://<IP_ADDRESS>:4317' . OtlpUtil::method(Signals::TRACE)
        );
        $exporter = new OpenTelemetry\Contrib\Otlp\SpanExporter($transport);
        $spanProcessor = new SimpleSpanProcessor($exporter);

Collector layer
I've built a custom layer for the collector as described here.
My collector config is as follows:
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: <IP_ADDRESS>:4317

exporters:
  awsxray:
    region: ap-southeast-2
  awsemf:
    region: ap-southeast-2
  logging:
    loglevel: debug

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [logging, awsxray]
    metrics:
      receivers: [otlp]
      exporters: [awsemf]
  telemetry:
    logs:
      level: debug

Glueing OpenTelemetry + Xray together
I've also built a class in PHP to enclose all the openTelemetry+AWS code together for easy viewing here that puts all the OpenTelemetry code in one place.
IAM
And I added a IAM Policy to allow my Lambda execution environment access to write traces:
    Type: 'AWS::IAM::Policy'
    Properties:
      PolicyName: "${self:service}-${sls:stage}-open-telemetry"
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          Effect: Allow
          Action:
            - "logs:PutLogEvents"
            - "logs:CreateLogGroup"
            - "logs:CreateLogStream"
            - "logs:DescribeLogStreams"
            - "logs:DescribeLogGroups"
            - "xray:PutTraceSegments"
            - "xray:PutTelemetryRecords"
            - "xray:GetSamplingRules"
            - "xray:GetSamplingTargets"
            - "xray:GetSamplingStatisticSummaries"
            - "ssm:GetParameters"
          Resource:
            - "*"
      Roles:
        - !Ref IAMRoleLambda

Result
With all this in place, I get some logs but nothing shown visually in XRay.
I can see logs such as this, which I presume are generated from the collector.
2023-01-08T11:43:42 {"level":"info","ts":1673178222.7031276,"msg":"TracesExporter","kind":"exporter","data_type":"traces","name":"logging","#spans":1}
2023-01-08T11:43:42 {"level":"info","ts":1673178222.7031856,"msg":"ResourceSpans #0\nResource SchemaURL: \nResource attributes:\n     -> faas.name: Str(XXXXX)\n     -> faas.version: Str($LATEST)\n     -> cloud.region: Str(ap-southeast-2)\n     -> cloud.provider: Str(aws)\nScopeSpans #0\nScopeSpans SchemaURL: \nInstrumentationScope io.opentelemetry.contrib.php \nSpan #0\n    Trace ID       : 63baac6e38c7d4ad3a7b755834919a73\n    Parent ID      : 84bfa37214deb991\n    ID             : d810f21b345baa63\n    Name           : XXXX::YYYY::ZZZZ\n    Kind           : Internal\n    Start time     : 2023-01-08 11:43:42.607256547 +0000 UTC\n    End time       : 2023-01-08 11:43:42.695558894 +0000 UTC\n    Status code    : Unset\n    Status message : \nAttributes:","kind":"exporter","data_type":"traces","name":"logging"}

I can confirm the "Trace ID" in that payload matches the Lambda invocation trace ID (different format but same number).  And if I misconfigure the open-telemetry transport then I get a connection error, so I know it's connecting :-)
When I look in AWS X-Ray's UI, I see my Lambda invocation but no subsequent PHP trace.
I'm not sure how to debug this further to work out whether it's open-telemetry or the collector or something-something AWS.
Thanks in advance, Scott

Hi Scott!
Thanks for reaching out and leaving a very detailed explanation of the issue you are running into. As mentioned in the similar issue in the bref repo, there have been many changes that were made in opentelemetry-php ahead of the 1.0 beta (to give some context, the AwsSdkInstrumentation changes were part of the 0.0.15 release). I will be working on updating our docs to reflect these changes :)
As @brettmc mentioned in the issue on the opentelemetry-php repo, your collector config doesn't seem to have any issues. I have been looking closely into the gist you posted and have a few questions, and some suggestions of where the problem could be coming from.

From what I learned about PHP and Lambda, PHP is not one of the Lambda supported runtimes, so I am assuming you have set up a custom runtime. You also mentioned that you can see Lambda traces in X-Ray, do you mind posting a service map or a screenshot of how those Lambda traces look like in X-Ray? Things are a little different for custom runtimes as we don't support active tracing in Lambda out of the box, so I am curious what those Lambda traces look like and where they are coming from (My guess right now is that they are coming from the resource detectors).
The AwsSdkInstrumentation that was added will only instrument calls to AWS SDK clients (like S3, DynamoDB, etc.) which I do not see being used in the gist. That's probably why you are not seeing subsegments in X-Ray. The sample app on this repo has an example which includes an S3 call, and that results in the creation of an S3 subsegment that is visible in X-Ray. However, this does not include Lambda.
I noticed in the gist that there is a rootSpan and rootScope being initialized and destroyed, but I don't see any other spans being created. I am not sure if there are other spans being created elsewhere, but that might be a reason why traces are not showing up in X-Ray so I thought I would point that out :)

Hi @carolabadeer,
Thanks for jumping on.  I really appreciate you helping out. I'm hoping I'm not the only person in the world trying to get PHP + Lambda + Xray working :-)
To answer your questions...
Supported runtime in use
I'm not using a custom runtime — I am using the Lambda supported runtime provided.al2.
On top of that I drop in:

some layers for Bref (which provides the PHP binary),
my codebase, and
a layer for otel-collector — which was built following the recommended instructions on the opentelemetry-lambda project.

I've uploaded my service map in XRay, and a visualisation of a trace. In it, you'll see I get as far as the Lambda function but not further.

AwsSdkInstrumentation
Yeah, I get that it only affects internal calls to AWS services, but I'm grasping at straws getting it all glued together and figured what the hell, as the available docs are out of date and I was struggling to get things working :-)
Anyway, I do have some php-aws-sdk use within my code that I hope will benefit from this, once things start working.
Spans
In my gist you'll see methods for startSpan() and endSpan().  In addition to the root span you see in the gist, my actual code has start and ends spans throughout key parts, as follows:
OpenTelemetry::startSpan(__METHOD__, __CLASS__);
try {
    // do stuff
} catch (Throwable $e) {
    OpenTelemetry::recordException($e->getMessage, $e);
} finally {
    OpenTelemetry::endSpan();
}

Support?
For what it's worth, I've also got a slow moving support ticket with AWS on this, so if you have the ability to see that, I can send you a support ticket ID and you could use that to take a deeper peek.
I'm sharing this ticket with that agent, but as I said, it's a slow moving ticket at present and I'm getting much appreciated help through GitHub right now.
Thanks again for your help. It's my hope to get this working and then share what we've learned to help others.

I'm going to throw out a bunch of wild questions here and hope they help progress the issues further.

Are there any logs that seem to related to the awsxray exporter? Maybe go down to info level in service.telemetry.logs.level. Note more logs == more cost. I would suggest only leaving this on for troubleshooting.
Have you tried using an official AWS Lambda Layer. I think the generic layer should work for you here. The idea for go/dotnet should be very similar to what you are trying to do. See docs here. That layer just contains the collector.
Did you enable active tracing?
The traceid looks to be the correct format. Ignorant question here: Is it supposed to have the 1- in front of the traceid? 63baac6e38c7d4ad3a7b755834919a73 -> 1-63baac6e38c7d4ad3a7b755834919a73? Maybe the xray exporter handles that but I would like to see those logs.

Hi @bryan-aguilar, thanks for contributing.
I'm not sure how to find/access any additional awsxray exporter logs. I went to debug level already in exporters.logging.loglevel and sevice.telemetry.logs.level in my yaml and provided a snippet of logs in my original post. There was nothing there that gave me any insight, just start up and shut down messages. Is there additional config I can set?
Re the official lambda layer, I'll try the official go/dotnet one and report back. If they're all just housing the same collector, I wonder why all the layers exist?
To your other questions, yes, I enabled Active Tracing in Lambda, and I saw that prefix too. My memory is a bit foggy on where that's generated, but I found a method somewhere to generate it to the spec. I'll run some experiments around it and report back.

Can you post the startup logs for the collector? There should be a set of logs where the collector says that it's initializing components.

I'm following @bryan-aguilar's advice on using a supported layer to try that out.
I'm switching over to the Go supported Lambda layer by following these instructions, which tells me to use the layer arn:aws:lambda:ap-southeast-2:901920570463:layer:aws-otel-collector-x86_64-ver-0-68-0:1 (I've substituted the region and architecture). I presume<PHONE_NUMBER>63 is an AWS account used for hosting managed layers.
However when I deploy, I get a permission denied error for the resource:
UPDATE_FAILED: PhpApiLambdaFunction (AWS::Lambda::Function)
Resource handler returned message: "User: XXXXXX is not authorized to perform: lambda:GetLayerVersion on resource: arn:aws:lambda:ap-southeast-2:901920570463:layer:aws-otel-collector-x86_64-ver-0-68-0:1 because no resource-based policy allows the lambda:GetLayerVersion action

Anyone have any ideas on this? Is there something I need to do at my side, is that a permission on the resource itself, or are the docs out of date and I got the layer wrong as a reuslt?
Thanks again.

arn:aws:lambda:ap-southeast-2:901920570463:layer:aws-otel-collector-amd64-ver-0-68-0:1

@sc0ttdav3y, sorry about the confusion. I'll make a note of that and see how we can improve the clarity of those website docs.

Good news 🎉
By using the layer arn:aws:lambda:ap-southeast-2:901920570463:layer:aws-otel-collector-amd64-ver-0-68-0:1, I am now seeing span data from my PHP app. Many thanks @bryan-aguilar and @carolabadeer for your help so far.

I'll now be able to go and get some experience with the stack over the few days, and I'll write up my experiences into and drop them back into this ticket. From there, I'd be happy to help write up some PHP instructions for the docs, if you like.
From my point of view, only one minor issue remains... X-Ray shows the caller as being the first AWS service, and not the Lambda Function node.

I suspect this is to do with the trace ID I'm using from the request header, which would have been set at the first service called. I'm using Aws\Xray\Propagator::AWSXRAY_TRACE_ID_HEADER (which is x-amzn-trace-id), and passing it to the Aws\XRay\Propagator like this (full code can be seen in the gist at the top of this ticket):
...
$context = $xrayPropagator->extract($headers); // <-- $headers is an array containing 'x-amzn-trace-id'
$xrayPropagator->inject($carrier, null, $context);
$awsSdkInstrumentation->setPropagator($xrayPropagator);
$tracer = $awsSdkInstrumentation->getTracer();
...

Is there another environment variable or header provided in the Lambda event that tells us the correct trace ID to use?

Is the parent span being set correctly? Could you share the raw trace information? If it's not being set correctly it could point to an issue in the PHP X-ray propagator.

It may be hard to sanitize all that information, so at least an insight into the traceid/parent span information for each span would be helpful.

@sc0ttdav3y Is it necessary to extract the headers and then re-inject the context? If I remember correctly, the context should be propagated without you needing to do those extra steps. I think that might also be changing the context and causing the disconnect in the service map.

I've looked at the raw trace, and I see the segment containing my PHP trace has a parent_id set to a sub-segment that matches a segment from AppSync. @bryan-aguilar, the trace is available here
To @carolabadeer's suggestion, if I remove the code that supplies that header into the Propagator, then the the PHP traces are no longer connected to the trace, but instead appear as separate traces in X-Ray.

Hi @carolabadeer, thanks for the examples.
I don't seem to have a _X_AMZN_TRACE_ID environment variable.
I've looked in both PHP's $_ENV and $_SERVER superglobals. $_SERVER is where all the other AWS environment vars live (e.g. AWS_SESSION_TOKEN, AWS_LAMBDA_FUNCTION_NAME, AWS_XRAY_DAEMON_ADDRESS). I do see an environment variable called AWS_XRAY_CONTEXT_MISSING set to LOG_ERROR, if that helps.
Do I need to do anything to make that appear in Lambda? I have active tracing enabled already.

Are you retrieving the value of the _X_AMZN_TRACE_ID environment variable inside the Lambda function handler?

Yes, I'm in the function handler, and there's no trace of _X_AMZN_TRACE_ID (pun intended).
This is a dump of the environment variables I'm seeing...
    {
        "AWS_LAMBDA_FUNCTION_VERSION": "$LATEST",
        "AWS_SESSION_TOKEN": "XXX",
        "LD_LIBRARY_PATH": "\/lib64:\/usr\/lib64:\/var\/runtime:\/var\/runtime\/lib:\/var\/task:\/var\/task\/lib:\/opt\/lib",
        "AWS_LAMBDA_LOG_GROUP_NAME": "\/aws\/lambda\/XXX",
        "LAMBDA_TASK_ROOT": "\/var\/task",
        "AWS_LAMBDA_RUNTIME_API": "<IP_ADDRESS>:9001",
        "AWS_LAMBDA_LOG_STREAM_NAME": "2023\/01\/28\/[$LATEST]XXX",
        "OPENTELEMETRY_COLLECTOR_CONFIG_FILE": "\/opt\/php-common\/conf\/opentel\/config.yaml",
        "AWS_LAMBDA_FUNCTION_NAME": "XXXX",
        "AWS_XRAY_DAEMON_ADDRESS": "<IP_ADDRESS>:2000",
        "PATH": "\/usr\/local\/bin:\/usr\/bin\/:\/bin:\/opt\/bin",
        "AWS_DEFAULT_REGION": "ap-southeast-2",
        "PWD": "\/var\/task",
        "SERVICE": "XXX",
        "LANG": "en_US.UTF-8",
        "LAMBDA_RUNTIME_DIR": "\/var\/runtime",
        "AWS_LAMBDA_INITIALIZATION_TYPE": "on-demand",
        "TZ": ":UTC",
        "AWS_REGION": "ap-southeast-2",
        "STAGE": "dev",
        "ODBCSYSINI": "\/opt\/microsoft\/conf\/",
        "SHLVL": "1",
        "_AWS_XRAY_DAEMON_ADDRESS": "<IP_ADDRESS>",
        "_AWS_XRAY_DAEMON_PORT": "2000",
        "BREF_AUTOLOAD_PATH": "\/opt\/php-common\/vendor\/autoload.php",
        "AWS_XRAY_CONTEXT_MISSING": "LOG_ERROR",
        "_HANDLER": "functions\/php-admin\/handler.php",
        "AWS_LAMBDA_FUNCTION_MEMORY_SIZE": "4096",
        "_": "\/opt\/bin\/php"
    }

Of possible interest are these:
{
        "_AWS_XRAY_DAEMON_ADDRESS": "<IP_ADDRESS>",
        "_AWS_XRAY_DAEMON_PORT": "2000",
        "AWS_XRAY_CONTEXT_MISSING": "LOG_ERROR"
}

Here's a snapshot of the Lambda function monitoring configuration page in the console. It has active tracing enabled (I read somewhere this is required) but code profiling and enhanced monitoring disabled.

Hey Scott!
Had to do some research to get to the reason why this environment variable isn't showing up. Here's what I found:
Since you're using a custom Lambda runtime, the _X_AMZN_TRACE_ID environment variable is actually not automatically set. You would have to manually set it in order to propagate the Lambda context correctly. The Processing tasks section of the  AWS documentation on building a custom runtime explain how to do this. Hope that helps!

I ended up finding the trace ID in Bref's Context object (it pulls it from lambda-runtime-trace-id in the next invocation response).
I am now seeing linked traces :tada:

Thanks so much for all your help getting this going @carolabadeer and @bryan-aguilar .
For anyone following, check out my gist on a working implementation.
https://gist.github.com/sc0ttdav3y/0d320d08a726dd0ed204a47bd8ebb78b


Signer

Right now I'm only playing with dev accounts which don't need passwords.
How should we handle signers

popup like on apps? Could we reuse the exact same component?
Another page after the tx form, like on Fether?

@yjkimjunior I'll start working on this soon. Do you have a mockup in mind? Should I start the functionality part first?

@amaurymartiny i don't have a mockup in mind for this yet, but I don't think it'd be a bad idea to reuse the signer component from apps.

Mockups:
Closed

Open


Save default weight unit with the list (not everyone uses lbs)

My friends are forced to change the weight unit every single time they receive my list since the default unit is lbs.

I've made metric the primary unit system. I'll also add a feature that saves whichever option the user chooses.


Cannot build lean on linux armv7l (ubuntu 16) due to segfault compiling library

Prerequisites

[X ] Put an X between the brackets on this line if you have done all of the following:

Checked that your issue isn't already filed.

Specifially, check out the wishlist, open RFCs,
or feature requests.

Reduced the issue to a self-contained, reproducible test case.

Description
The build of lean fails due to a segfault.
Steps to Reproduce

git clone https://github.com/leanprover/lean [I cloned at 8849df]; cd lean
mkdir -p build/debug; cd build/debug; cmake -D CMAKE_BUILD_TYPE=Debug ../../src
make

Expected behavior: [What you expect to happen]
make should succeed
Actual behavior: [What actually happens]
Make segfaults, apparently during a compile of the lean core?
70%] Built target smt2_frontend
[ 70%] Built target leanstatic
[ 71%] Built target lean
/home/abliss/proj/lean/library/init/core.lean: subtype.has_sizeofSegmentation fault
CMakeFiles/standard_lib.dir/build.make:57: recipe for target 'CMakeFiles/standard_lib' failed
make[2]: *** [CMakeFiles/standard_lib] Error 139
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/standard_lib.dir/all' failed
make[1]: *** [CMakeFiles/standard_lib.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

(The exact target printed in the last line varies slightly.)
Reproduces how often: [What percentage of the time does it reproduce?]
5/5
Versions
$ bin/lean --version
Lean (version 3.1.1, commit 8849df0d3ee1, Debug)
$ uname -a
Linux localhost 3.4.0-gad29d11 #1 SMP PREEMPT Sat Jul 2 00:00:53 UTC 2016 armv7l armv7l armv7l GNU/Linux
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"

Additional Information
I'll try attaching a core dump, but it's 46M and I don't know if github will allow it. It seems to be some problem during the handling of an exception:
at /home/abliss/proj/lean/src/library/module_mgr.cpp:368
#16 0x0073d04c in main (argc=2, argv=0xbebcd074) at /home/abliss/proj/lean/src/shell/lean.cpp:636
(gdb)
#0  0x00bc8b10 in lean::scanner::next (this=0xbebcae80) at /home/abliss/proj/lean/src/frontends/lean/scanner.cpp:99
#1  0x00bc8c02 in lean::scanner::throw_exception (this=0xbebcae80, msg=0xcd3a48 "unexpected token")
    at /home/abliss/proj/lean/src/frontends/lean/scanner.cpp:119
#2  0x00bca304 in lean::scanner::read_key_cmd_id (this=0xbebcae80) at /home/abliss/proj/lean/src/frontends/lean/scanner.cpp:590
#3  0x00bca6fa in lean::scanner::scan (this=0xbebcae80, env=...) at /home/abliss/proj/lean/src/frontends/lean/scanner.cpp:659
#4  0x0095a6c6 in lean::parser::scan (this=0xbebcae10) at /home/abliss/proj/lean/src/frontends/lean/parser.cpp:228
#5  0x0096a6c8 in lean::parser::next (this=0xbebcae10) at /home/abliss/proj/lean/src/frontends/lean/parser.h:282
#6  0x00966630 in lean::parser::parse_imports (this=0xbebcae10, fingerprint=@0xbebcacd0:<PHONE_NUMBER>, imports=std::vector of length 6, capacity 8 = {...})
    at /home/abliss/proj/lean/src/frontends/lean/parser.cpp:2401
#7  0x0096711e in lean::parser::get_imports (this=0xbebcae10, imports=std::vector of length 6, capacity 8 = {...})
    at /home/abliss/proj/lean/src/frontends/lean/parser.cpp:2477
#8  0x008b2ab8 in lean::module_mgr::get_direct_imports (this=0xbebcc694, id="/home/abliss/proj/lean/library/init/category/default.lean",
    contents="/-\nCopyright (c) 2016 Microsoft Corporation. All rights reserved.\nReleased under Apache 2.0 license as described in the file LICENSE.\nAuthors: Leonardo de Moura\n-/\nprelude\nimport init.category.applica"...) at /home/abliss/proj/lean/src/library/module_mgr.cpp:415
#9  0x008b179e in lean::module_mgr::build_lean (this=0xbebcc694, id="/home/abliss/proj/lean/library/init/category/default.lean",
    contents="/-\nCopyright (c) 2016 Microsoft Corporation. All rights reserved.\nReleased under Apache 2.0 license as described in the file LICENSE.\nAuthors: Leonardo de Moura\n-/\nprelude\nimport init.category.applica"..., mtime=1497112814, module_stack=lean::rb_tree = {...})
    at /home/abliss/proj/lean/src/library/module_mgr.cpp:227
#10 0x008b1054 in lean::module_mgr::build_module (this=0xbebcc694, id="/home/abliss/proj/lean/library/init/category/default.lean", can_use_olean=true,
    module_stack=lean::rb_tree = {...}) at /home/abliss/proj/lean/src/library/module_mgr.cpp:208
#11 0x008b1904 in lean::module_mgr::build_lean (this=0xbebcc694, id="/home/abliss/proj/lean/library/init/default.lean",
    contents="/-\nCopyright (c) 2014 Microsoft Corporation. All rights reserved.\nReleased under Apache 2.0 license as described in the file LICENSE.\nAuthors: Leonardo de Moura\n-/\nprelude\nimport init.core init.logic "..., mtime=1497112814, module_stack=lean::rb_tree = {...})
    at /home/abliss/proj/lean/src/library/module_mgr.cpp:240
#12 0x008b1054 in lean::module_mgr::build_module (this=0xbebcc694, id="/home/abliss/proj/lean/library/init/default.lean", can_use_olean=true,
    module_stack=lean::rb_tree = {...}) at /home/abliss/proj/lean/src/library/module_mgr.cpp:208
#13 0x008b1904 in lean::module_mgr::build_lean (this=0xbebcc694, id="/home/abliss/proj/lean/library/data/bitvec.lean",
    contents="/-\nCopyright (c) 2015 Joe Hendrix. All rights reserved.\nReleased under Apache 2.0 license as described in the file LICENSE.\nAuthors: Joe Hendrix, Sebastian Ullrich\n\nBasic operations on bitvectors.\n\nTh"..., mtime=1497112814, module_stack=lean::rb_tree = {...})
    at /home/abliss/proj/lean/src/library/module_mgr.cpp:240
#14 0x008b1054 in lean::module_mgr::build_module (this=0xbebcc694, id="/home/abliss/proj/lean/library/data/bitvec.lean", can_use_olean=true,
    module_stack=lean::rb_tree = {...}) at /home/abliss/proj/lean/src/library/module_mgr.cpp:208
#15 0x008b2684 in lean::module_mgr::get_module (this=0xbebcc694, id="/home/abliss/proj/lean/library/data/bitvec.lean")
    at /home/abliss/proj/lean/src/library/module_mgr.cpp:368
#16 0x0073d04c in main (argc=2, argv=0xbebcd074) at /home/abliss/proj/lean/src/shell/lean.cpp:636

Here's a zipfile containing the core dump.
core.zip

Anyway, it seems like the least invasive fix for this is to add -fsigned-char to the CXX_FLAGS in CMakeLists.txt. But I would suggest also enabling at least -Werror=type-limits . And the fact that this manifested as an overflow and segfault might point to safer ways to code the scanner's exception handling.

@abliss Thanks for reporting the problem.
I checked the C++ documentation, and the meaning of char is platform dependent. I didn't know, I thought it was always signed :(
http://en.cppreference.com/w/cpp/language/types
Here is the relevant part:

signed char - type for signed character representation.
unsigned char - type for unsigned character representation. Also used to inspect object representations (raw memory).
char - type for character representation which can be most efficiently processed on the target system (has the same representation and alignment as either signed char or unsigned char, but is always a distinct type). Multibyte characters strings use this type to represent code units. The character types are large enough to represent 256 different values (in order to be suitable for storing UTF-8 encoded data) (since C++14). The signedness of char depends on the compiler and the target platform: the defaults for ARM and PowerPC are typically unsigned, the defaults for x86 and x64 are typically signed.

BTW, according to the standard EOF is a macro definition of type int that expands into a negative integral constant expression (generally, -1).
@gebner @kha @jroesch I think we should go with @abliss' suggestion (i.e., add -fsigned-char) at least in the near future. What do you think?

BTW, note that according to the standard there is no guarantee that EOF fits in a signed char.

@abliss I don't have an ARM machine to make experiments.
Could you please try the following experiment?

Replace all occurrences of EOF with std::char_traits<char>::eof() at src/frontends/lean/scanner.cpp, and check whether it fixes the build problem.

Thanks @leodemoura , I'll give that a try and report back.

So it seems that there may be another arch-specific assumption buried in there somewhere.

I think it is the same assumption. The whole code base is assuming char is signed.

Sorry, my previous comment doesn't make sense. The -fsigned-char should make sure char is signed in the whole system.

@kha @gebner @jroesch Do you have access to an ARM machine?

Yeah, I figured that the -fsigned-char should've fixed it too, but it seems that the serializer code is unable to re-read the coe.olean file that it wrote. I'm stepping through it now trying to figure out exactly where it went wrong.
BTW, if you want to experiment locally, you might have some luck installing the Android NDK on your x86 machine. I believe it comes with a cross-compiling toolkit and a QEMU emulator. (I'm running out of an ubuntu chroot on my quad-core Nexus 7; the qemu performance will likely be much slower.)

BTW is there documentation on the .olean format somewhere? I'm not sure whether it's writing the .oleans badly or just reading them badly. The .olean files don't generated on ARM don't match byte-for-byte the ones from x86, but I'm not sure whether they're supposed to be platform-independent (or even reproducible).

@abliss .olean is based on serialization. Here is a potential red flag:
https://github.com/leanprover/lean/blob/master/src/util/serializer.cpp#L66-L77
The C++ standard does not guarantee that EOF fits in a char.
When you compile with -fsigned-char, does the following program return 1 on your 32-bin armhf machine?
#include <iostream>
int main() {
  char c = -1;
  return (c == EOF);
}

@leodemoura The EOF value in the scanner is completely arbitrary (it just shouldn't be 0 or a character that we actually want to read).  I'll change it to 0xFF.
I have now rebooted my Raspberry Pi for the first time in 2 years and will try to fix this issue:
Last login: Fri May 29 08:34:25 2015 from <IP_ADDRESS>

@gebner Thanks @gebner!

The problem already happens when reading init/core.olean.  Bisecting shows that it happens at the psigma.has_sizeof instance.  Weirdly enough, switching the sigma and psigma instances makes the error go away.
I'm now compiling a debug build with the latest changes.  Each build takes about 8 hours on my rpi, so I'm afraid this will take a bit of time.

I'm now compiling a debug build with the latest changes. Each build takes about 8 hours on my rpi, so I'm afraid this will take a bit of time.

@gebner Thanks again for your heroic efforts!

I tried reducing the library to just init/core.lean. If I run lean --make, it seems to occasionally produce a valid core.olean. But most of the time it is corrupt.
Is there expected variation in a produced .olean file (timestamps, random numbers, threading?) or should it be deterministic based on the inputs?
I do notice that if I run it under valgrind, it reports a handful of operations on undefined data.

Thanks for figuring out that it works with the olean file generated on amd64.  I have tried to add a few checks in the code to narrow down the problem.
From a top-down view, an olean file consists of a bit of metadata (lean version, module dependencies), followed by the actual content (a byte array) plus a hash code (which we don't check for performance reasons).  For some reason, when reading the olean file, already this actual content part is corrupted and the hash code is wrong.

Is there expected variation in a produced .olean file (timestamps, random numbers, threading?)

Yes, mainly due to multi-threading.  AFAICT we don't use random numbers or store timestamps.  The variation comes mainly from the mk_fresh_name function, which generates globally unique names.

I wonder if there might be a thread synch issue (even with -j1), which for some reason only affects ARM

You can run lean with the -j0 option, which disables multi-threading (almost) completely.  We use this option for deterministic tests.  However the bug is there even with -j0.
Typically these non-deterministic threading bugs arise due to timing differences: the ARM machine is simply slower and hence hits some corner cases.  We've had this situation with slow debug builds in the past as well.

Thanks for those details. I can confirm that the bug still happens for me with -j 0, and it happens in exactly the same place as you: psigma.sizeof (core.olean line 462), even though my quad-core ARM Nexus 7 sounds much faster than your RPi (debug build is well under an hour). Pretty strange!

This is terribly, terribly weird.  We seem to lose 3 bytes when writing the blob section of the olean file.  All of these bytes have the value 0xFF (there are other 0xFF bytes that make it through, though..).
Stepping through with the debugger, when writing, the blob has size 152240--this is consistent between amd64 and arm and also the size we write into the file.  However 3 out of those 152240 bytes don't make it to the file.  So when reading, we are 3 bytes short....
Here is the full diff:
[gebner:~/lean/library/init] master+* ± diff <(od -w1 -vAn -tx1z core.olean) <(od -w1 -vAn -tx1z core.olean.klikia)
15,18c15,18
<  0b  >.<
<  c6  >.<
<  7b  >{<
<  3e  >><
---
>  78  >x<
>  46  >F<
>  6f  >o<
>  fe  >.<
40960d40959
<  ff  >.<
57344d57342
<  ff  >.<
113032c113030
<  13  >.<
---
>  a6  >.<
136846,136847c136844,136845
<  15  >.<
<  85  >.<
---
>  44  >D<
>  92  >.<
136861,136862c136859,136860
<  15  >.<
<  82  >.<
---
>  44  >D<
>  8f  >.<
138994c138992
<  13  >.<
---
>  10  >.<
138998,138999c138996,138997
<  0f  >.<
<  8d  >.<
---
>  47  >G<
>  80  >.<
139009c139007
<  13  >.<
---
>  10  >.<
139013,139014c139011,139012
<  0f  >.<
<  8a  >.<
---
>  47  >G<
>  7d  >}<
139264d139261
<  ff  >.<
145170c145167
<  14  >.<
---
>  10  >.<
145174,145175c145171,145172
<  15  >.<
<  16  >.<
---
>  4b  >K<
>  37  >7<
145185c145182
<  14  >.<
---
>  10  >.<
145189,145190c145186,145187
<  15  >.<
<  15  >.<
---
>  4b  >K<
>  36  >6<

Are these 3 bytes the only ff bytes?

Are these 3 bytes the only ff bytes?

That was my first thought as well, but no, other 0xFF bytes survive just fine.

Even worse: they get lost at 0xXXX000 (4096) page boundaries!

Good catch!  Now the issue becomes interesting!  Even better, it only happens at some 4096*n page boundaries, namely n=10, n=14, and n=34 ?!?

The 4096 byte boundaries align perfectly with the 8192 byte buffer that libstdc++ uses:
write(4, "oleanfile\0\3\2\1\377xFo\376\0\0\377\0\2R\260TK\0Prop"..., 8192) = 8192
write(4, "\2\377\0\0\4\1PInfo\0\0\347\200\nATTR\0\0i\377\0\0\3\350\0\347\0\0"..., 8192) = 8192
write(4, "\205\10\2x\342\202\201\0\4\0\1\10\2y\342\202\201\0\4\0\1\10\2x\342\202\202\0\4\0P\10"..., 8192) = 8192
write(4, "\377\0\0\7\262\7\0\205\0\0\272\0\377\0\0\7\271\7\0\205\0\0\222\7\2_\0\0\0\377\0\0"..., 8192) = 8192
write(4, "\0\26\0\377\0\0\nz\0\333\7\0\377\0\0\2z\4\0\377\0\0\nw\7\0\377\0\0\2{\0"..., 8191) = 8191
write(4, "\0\0\r\1\10\0\377\0\0\2\366\0\6\0\1\6\0\377\0\0\f\362\0\26\10\0\377\0\0\2\370\0"..., 8192) = 8192
write(4, "\377\0\0\3n\0\6\6\0\377\0\0\17\336\0\2\0\2\0\275\6\0\2\6\6\3\0\367\1\1\1\0"..., 8191) = 8191
write(4, "\0\0\3\277\4\0\377\0\0\21\356\10\0\377\0\0\3\275\4\0\377\0\0\21\353\10\0\377\0\0\3\277"..., 8192) = 8192
write(4, "\377\0\0\24N\0\272\0\272\2\377\0\0\4\1PInfo\0\0\377\0\0\4M\377\0\0\1\30"..., 8192) = 8192
write(4, "\6\3\4\0\377\0\0\4\275rec\0\2\1\1\0\1\1\4\0e\0\1\7\0\377\0\0\4\307\1"..., 8192) = 8192
write(4, "\0\377\0\0\31k\0\1\0\377\0\0\31l\0\377\0\0\30\234\7\0\377\0\0\5\37\4\0>\7\0"..., 8192) = 8192
write(4, "\4\0\377\0\0\5\237mk\0\1\0\377\0\0\1\332\0\26\0\2\10\0\377\0\0\5\243\0\6\0\377"..., 8192) = 8192
write(4, "\2\0\377\0\0\2\5\0\377\0\0\2\6\0\272\7\0\377\0\0\6\21\4\0\377\0\0\36\226\6\6\0"..., 8192) = 8192
write(4, "\3\350\0\377\0\0\6\200\0class\0\0\0\377\0\0\6\200gind\0\0\0\2\1\0"..., 8192) = 8192
write(4, "C\0\0\377\0\0\0075\0\377\0\0#m\2\0\4\0\377\0\0\0079\0\0\377\0\0\0078\0\0"..., 8192) = 8192
write(4, "\1\0e\1\10\0\377\0\0\7\326\0\0\4\0\377\0\0\n\376\1\0\377\0\0\7\333\10\0\377\0\0"..., 8192) = 8192
write(4, "\0\377\0\0\10<\0\0\t\0\t\0\1\0\2VMC\0\0\377\0\0\10;\0\377\0\0'h\6"..., 8191) = 8191
write(4, "\0\0\10\261\2\1\4\0e\1\4\0\326\0P\0\26\0\1\0\2\0\377\0\0)e\7\0\377\0\0"..., 8192) = 8192

Well, except for the times where it flushes a buffer with 8191 bytes instead of 8192?!?!

And I have a reproducible minimal test:
#include <iostream>
#include <fstream>

int main() {
  constexpr auto fn = "buffer_0xff.out";
  constexpr unsigned size = 1000000;

  { // write a million times 0xFF
    std::ofstream out(fn, std::ios_base::binary);
    for (unsigned i = 0; i < size; i++) {
      out.put(0xFF);
    }
  }

  auto actual = std::ifstream(fn, std::ios_base::ate | std::ios_base::binary).tellg();
  std::cout << "The file " << fn << " has size " << actual
    << " (expected: " << size << ")" << std::endl;
}

The output depends on -fsigned-char:
$ c++ -fsigned-char buffer_0xff.cc -o buffer_0xff && ./buffer_0xff
The file buffer_0xff.out has size 999877 (expected: 1000000)
$ c++ buffer_0xff.cc -o buffer_0xff && ./buffer_0xff
The file buffer_0xff.out has size 1000000 (expected: 1000000)

Upstream knows about this breakage: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=47275

By changing the signedness of char you change the ABI and everything must use the same ABI.

Indeed, all the bets are off here unless the library is both built (installed) and then used with the same signedness for char. Thus, closing as invalid.

So, this means we must not use -fsigned-char.  I've removed the flag in https://github.com/leanprover/lean/commit/7342741263651641ebdacfc11ca81c49f500bc09
We'll have to fix all places where we assume that char is signed.

Amazing. I get the exact same result on my device (999877 with -fsigned-char, 1000000 without). Nice detective work.

@abliss The latest git version can now compile the standard library (well, half of it, my rpi is slow).  Can you confirm that it compiles on your device, and could you please run the tests to make sure everything works?

I think curr() function should return int, because EOF is not char but int.
https://github.com/leanprover/lean/pull/1701

Thanks for the fix @gebner . Here's what I've got on my device at commit ce5ca7:
46/1146 Test #1146: lp_tst ......................................................   Passed    0.05 sec

99% tests passed, 3 tests failed out of 1146

Total Test time (real) = 37334.13 sec

The following tests FAILED:
          2 - leanchecker (Timeout)
        359 - leanruntest_all (Timeout)
        1071 - leanittest_complete_trailing_period.lean (Failed)
Errors while running CTest
Makefile:127: recipe for target 'test' failed

(btw it failed at first because make clean didn't delete the corrupt .olean files; is that expected?)

Here's what I've got on my device at commit ce5ca7:

Thanks so much for running the tests!  I'll take a look at leanittest_complete_trailing_period.lean.

(btw it failed at first because make clean didn't delete the corrupt .olean files; is that expected?)

Yes, unfortunately there is no way to add additional commands to the clean target that is generated by CMake.  You can call make clean-olean to clean the olean files.


update prefetch section to remove todo and mention risk of "stale" data

During the Argonaut security review, Dixie Baker identified a potential risk of pre-fetch'd data becoming stale, and pointed out that Calling a CDS Service -> HTTP Request -> prefetch contains an outstanding todo.
We should:

remove the todo
Note that pre-fetch is an optional performance enhancement and is a negotiation between the EHR and the cds service. Pre-fetch data naturally includes a potential risk of clinical data becoming outdated. The longer the running time of the CDS service, the greater this risk. The EHR/provider should weigh this potential risk against the benefits of using pre-fetch'd data.

Note that the new-ish Providing FHIR resources to a CDS service section of the specification includes this guidance:

Regardless of the method used to provide FHIR resources to a CDS Service, the EHR must assure that clinical data provided to the CDS Service are the most current data available to the EHR user. Decisions based on stale clinical data pose a safety threat to the patient and must be avoided.


#440 fix for back button click

This might be a bit simplistic fix for the issue #440, but it works.
Ideally signinRedirect() call should be performed by oidc-client JS library whenever callback state is missing in the storage (e.g. this is how it is done in by the Keycloak JS adapter), but this will still require a small change to react-oidc package.

The back button problem is fixed in v4.
<EMAIL_ADDRESS>

I Close the PullRequest,
This is fixed in V4 and V5.
Thank you very much for your help. The new version is easier to make evolved.


Spike Accessibility plan

Miguel and Kacper with Tobias were working on the Accessibility plan.
Let's conclude and get some tickets created in the new year.
Original spike: SPA-1037 (has lots of great detail)
Also see this Accessibility Post-Summit Inventory in SPA-1364

Proposed functionality should follow a SDLC as specified here: https://wiki.wdf.sap.corp/wiki/display/PSSEC/Secure+Software+Development+Lifecycle+Overview
And must comply with security standards as detailed here:
https://wiki.wdf.sap.corp/wiki/display/PSSEC/SAP+Product+Standard+Security+-+Version+8.0


when no notifiers are configured, prometheus can't find alert manager and constantly logs

Rancher/rancher:v2.2.2
No notifiers are configured in the cluster, and I assume that no alert manager instances are deployed. If that is the case, then we should not set the alert manager config in prometheus. It tries to find it ever 6 minutes or so.  Did you rename your cluster_domain DNS name. It is set to cluster.local by default.
It looks like the DNS lookup always use cluster.local as cluster_domain even if you configured it differently. As a consequence, it cannot resolve your alertmanager-operated.cattle-prometheus service.
To solve this issue, I put the cluster_domain configuration back to default (i.e cluster.local) and it did resolve.

@reymonlu Where could I check the cluster domain? I'm sure we did not change the domain , but get the same error.

Somewhere in the rke config file

but be careful, it will change all your DNS records.

Ok, I looked that up. We did not change the domain. How could I debug from here?

can you go inside a Pod and check the /etc/resolv.conf and paste it here ?

the content is
$ cat /etc/resolv.conf
nameserver <IP_ADDRESS>
search cattle-prometheus.svc.cluster.local svc.cluster.local cluster.local ourdomain.com
options ndots:5

But maybe I fixed our issue with increasing the resources for prometheus. I will observe that.

I shouldn't have said that, it crashed right now ...

I would like to see another resolv.conf from a totally different Pods (not linked to monitoring) just to see if search domain ate different.

in on of our application containers it' pretty the same
nameserver <IP_ADDRESS>
search <ourappname>.svc.cluster.local svc.cluster.local cluster.local ourdomain.com
options ndots:5

Well, it looks good about that. Sorry i don't think I have a solution for you then :\

same problem

Experiencing the same.  /etc/resolv.conf in pods is fine. Any update on this issue ?

I got similar error in rancher v2.2.4 as below
level=info ts=2019-07-12T09:59:54.702873033Z caller=main.go:140 msg="Starting Alertmanager" version="(version=0.17.0, branch=HEAD, revision=c7551cd75c414dc81df027f691e2eb21d4fd85b2)"
2019/7/12 下午5:59:54 level=info ts=2019-07-12T09:59:54.702986361Z caller=main.go:141 build_context="(go=go1.12.4, user=root@932a86a52b76, date=20190503-09:10:07)"
2019/7/12 下午5:59:54 level=warn ts=2019-07-12T09:59:54.714340122Z caller=cluster.go:226 component=cluster msg="failed to join cluster" err="1 error occurred:\n\t* Failed to resolve alertmanager-cluster-alerting-0.alertmanager-operated.cattle-prometheus.svc:6783: lookup alertmanager-cluster-alerting-0.alertmanager-operated.cattle-prometheus.svc on <IP_ADDRESS>:53: no such host\n\n"
2019/7/12 下午5:59:54 level=info ts=2019-07-12T09:59:54.714383293Z caller=cluster.go:228 component=cluster msg="will retry joining cluster every 10s"
2019/7/12 下午5:59:54 level=warn ts=2019-07-12T09:59:54.714404723Z caller=main.go:230 msg="unable to join gossip mesh" err="1 error occurred:\n\t* Failed to resolve alertmanager-cluster-alerting-0.alertmanager-operated.cattle-prometheus.svc:6783: lookup alertmanager-cluster-alerting-0.alertmanager-operated.cattle-prometheus.svc on <IP_ADDRESS>:53: no such host\n\n"
2019/7/12 下午5:59:54 level=info ts=2019-07-12T09:59:54.714518706Z caller=cluster.go:613 component=cluster msg="Waiting for gossip to settle..." interval=2s
2019/7/12 下午5:59:54 level=info ts=2019-07-12T09:59:54.740466019Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/etc/alertmanager/config/alertmanager.yaml
2019/7/12 下午5:59:54 level=info ts=2019-07-12T09:59:54.740925851Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/config/alertmanager.yaml
2019/7/12 下午5:59:54 level=info ts=2019-07-12T09:59:54.745652367Z caller=main.go:365 msg=Listening address=:9093
2019/7/12 下午5:59:56 level=info ts=2019-07-12T09:59:56.714766379Z caller=cluster.go:638 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000147426s
2019/7/12 下午6:00:04 level=info ts=2019-07-12T10:00:04.715475078Z caller=cluster.go:630 component=cluster msg="gossip settled; proceeding" elapsed=10.000859305s
2019/7/12 下午6:01:16 level=info ts=2019-07-12T10:01:16.749989001Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/etc/alertmanager/config/alertmanager.yaml
2019/7/12 下午6:01:16 level=info ts=2019-07-12T10:01:16.750562018Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/config/alertmanager.yaml

I think the domain name alertmanager-cluster-alerting-0.alertmanager-operated.cattle-prometheus.svc should be alertmanager-operated.cattle-prometheus.svc

Rancher/rancher:v2.2.8
I had the same issue. Solved in v2.3 ?

Same here.
Cluster-Monitoring runs fine for a couple of days, then "Monitoring API is not ready" and the prometheus container logs :

level=warn ts=2019-09-25T12:39:49.038781441Z caller=main.go:295 deprecation_notice="\"storage.tsdb.retention\" flag is deprecated use \"storage.tsdb.retention.time\" instead."
level=info ts=2019-09-25T12:39:49.03889983Z caller=main.go:302 msg="Starting Prometheus" version="(version=2.7.1, branch=HEAD, revision=62e591f928ddf6b3468308b7ac1de1c63aa7fcf3)"
level=info ts=2019-09-25T12:39:49.038925503Z caller=main.go:303 build_context="(go=go1.11.5, user=root@f9f82868fc43, date=20190131-11:16:59)"
level=info ts=2019-09-25T12:39:49.03894996Z caller=main.go:304 host_details="(Linux 3.10.0-862.14.4.el7.x86_64 #1 SMP Fri Sep 21 09:07:21 UTC 2018 x86_64 prometheus-cluster-monitoring-0 (none))"
level=info ts=2019-09-25T12:39:49.038974301Z caller=main.go:305 fd_limits="(soft=65536, hard=65536)"
level=info ts=2019-09-25T12:39:49.038996007Z caller=main.go:306 vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2019-09-25T12:39:49.039956656Z caller=main.go:620 msg="Starting TSDB ..."
level=info ts=2019-09-25T12:39:49.040070764Z caller=web.go:416 component=web msg="Start listening for connections" address=<IP_ADDRESS>:9090
level=info ts=2019-09-25T12:39:49.043700213Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1569355200000 maxt=1569362400000 ulid=01DNJR1HE89GCQXK0XPS7QGN9Q
level=info ts=2019-09-25T12:39:49.045527787Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1569362400000 maxt=1569369600000 ulid=01DNJYYW3J6VGNB2NHQVTT919B

level=info ts=2019-09-25T12:39:49.047415328Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1569369600000 maxt=1569376800000 ulid=01DNK6034ETSZHJ9MK0036HN9N
level=info ts=2019-09-25T12:39:49.049159101Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1569376800000 maxt=1569384000000 ulid=01DNKCMQ9P9Y4Z9ZNF0X2BHM1K
level=info ts=2019-09-25T12:39:49.05096651Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1569384000000 maxt=1569391200000 ulid=01DNKKHHD4563MQRWAMTTQG77N
level=info ts=2019-09-25T12:39:49.052796539Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1569391200000 maxt=1569398400000 ulid=01DNKTC5VWTTRYYXJ5VVDNEWBZ
level=info ts=2019-09-25T12:39:49.054617459Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1569398400000 maxt=1569405600000 ulid=01DNM18PDZ0WYB4B1Y2M017DNC
level=warn ts=2019-09-25T12:39:49.059296468Z caller=wal.go:116 component=tsdb msg="last page of the wal is torn, filling it with zeros" segment=/prometheus/wal/00000022
level=warn ts=2019-09-25T12:40:05.623939925Z caller=head.go:440 component=tsdb msg="unknown series references" count=257
level=info ts=2019-09-25T12:40:06.331713625Z caller=main.go:635 msg="TSDB started"
level=info ts=2019-09-25T12:40:06.331781676Z caller=main.go:695 msg="Loading configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml
level=info ts=2019-09-25T12:40:06.338060633Z caller=kubernetes.go:201 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2019-09-25T12:40:06.339485399Z caller=kubernetes.go:201 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2019-09-25T12:40:06.340583012Z caller=kubernetes.go:201 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2019-09-25T12:40:06.341829041Z caller=kubernetes.go:201 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2019-09-25T12:40:06.342957358Z caller=main.go:722 msg="Completed loading of configuration file" filename=/etc/prometheus/config_out/prometheus.env.yaml
level=info ts=2019-09-25T12:40:06.342979664Z caller=main.go:589 msg="Server is ready to receive web requests."
level=error ts=2019-09-25T12:40:09.055629106Z caller=notifier.go:481 component=notifier alertmanager=http://alertmanager-operated.cattle-prometheus:9093/api/v1/alerts count=0 msg="Error sending alert" err="Post http://alertmanager-operated.cattle-prometheus:9093/api/v1/alerts: dial tcp: lookup alertmanager-operated.cattle-prometheus on <IP_ADDRESS>:53: no such host"
runtime: failed to create new OS thread (have 14 already; errno=12)
fatal error: newosproc

I have the same issue with v2.2.8. After I enabled monitoring, the prometheus pod kept restart itself. And I check the log found same error message:
level=info ts=2019-12-17T08:01:49.233493553Z caller=main.go:589 msg="Server is ready to receive web requests." level=error ts=2019-12-17T08:01:53.109803368Z caller=notifier.go:481 component=notifier alertmanager=http://alertmanager-operated.cattle-prometheus:9093/api/v1/alerts count=0 msg="Error sending alert" err="Post http://alertmanager-operated.cattle-prometheus:9093/api/v1/alerts: dial tcp: lookup alertmanager-operated.cattle-prometheus on <IP_ADDRESS>:53: no such host"
I searched and there was no alertmanager workload installed.

We solved the problem by increasing the memory ResourceLimits for the installed helm chart.

Same problem on the cluster, can you post a screenshot with your current values @ChrisHaPunkt?

We placed the values in Rancher -> Cluster -> Tools -> Monitoring ->

The values itself are actually depending on your workloads.

I had the same issue after clicking the magic istio button :) ... bumping Prometheus limit to 2048 stopped crashing (for now) and everything just feels snappier now ...
Should the defaults be higher maybe? Button to recalculate the settings?

I had the same issue and to solve it I just manually deployed a prom/alertmanager pod called alertmanager-operated in the same namespace cattle-prometheus and the error message disapeared.

This is still seemingly an issue, there's literally no alertmanager pods deploying whatsoever n my cluster, and Prometheus crash loops itself looking for it.

rancher v2.4.5  This problem still exists
2020/08/25 09:26:12 [ERROR] Failed to send alert: Post http://alertmanager-operated.cattle-prometheus.svc.cluster.local:9093/api/v1/alerts: dial tcp: lookup alertmanager-operated.cattle-prometheus.svc.cluster.local on <IP_ADDRESS>:53: no su
E0825 09:26:20.495601      33 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted


New helper when

For output html, often need to add classname depending of different terms, for example:

some html

New helper function will do this more clearly and conveniently:

some html

@rorc we discuss proposals in https://github.com/laravel/internals/ repo. Create an issue there.

@Dylan-DPC Done, thx)
https://github.com/laravel/internals/issues/487


Facebook Instant Games issue - can't add plugin

Node 9.2.1
npm
.yo-rc.json
{
"generator-phaser-plus": {
"meta": {
"createdWith": "3.0.0-beta.7",
"creationDate": "2019-06-03T18:33:31.708Z"
},
"objects": {
"dest": "app/scripts/objects/"
},
"plugins": {
"dest": "app/scripts/plugins/"
},
"scenes": {
"dest": "app/scripts/scenes/",
"index": {
"name": "app/scripts/scenes/index.js",
"requirePath": "./"
}
}
}
}
I have added

to the index.html file head. I also tried adding it to the body above the comment line where scripts are injected.
I also added
new webpack.DefinePlugin({
//  Required by Phaser: Enable Canvas and WebGL renderers.
'CANVAS_RENDERER': JSON.stringify(true),
'WEBGL_RENDERER': JSON.stringify(true),
'PLUGIN_FBINSTANT': JSON.stringify(true)
}),
to the gulpfile.js/webpack/config/plugins.js file but I get the FBInstant is not defined when I add this to my index.js file
export function boot() {
return new Phaser.Game(config);
}
FBInstant.initializeAsync().then(function() {
FBInstant.setLoadingProgress(100);
FBInstant.startGameAsync().then(function() {
boot();
});
});
Any idea what I'm doing wrong? Other development environments do this and it works great, but this generated development environment is the best I've found yet and I would love to make it my go to project creator.
Thanks for your work on this generator.

Sorry about that.
It's been a while since I made any significant updates to the generator template, and Facebook Instant Games are a recent addition to Phaser I'm not aware of yet. I'll have a look into this issue later.

There are two parts to this plug in. The Facebook SDK referenced by the URL created by Facebook and also Phaser's own plugin phaser-facebook-instant-games.js that exposes calls (this.facebook.call()) within the game code. Thank you for your response.

Amazing project generator, btw. I wish Phaser would officially adopt this structure and provide all code examples compliant with this set up.


Feeling of demotivation after seeing some good projects

Hello sir, when I develop some projects, I get a feel of confidence after seeing it working great. But, when I look at other people's projects, sometimes I get demotivated and start questioning my level of development. How to get better in this situation?

Yes, this happens with me too.

+1


[Publisher] Validation needs to be added in create document page

Validation to show name and description fields are required
Validation to the URL added as document URL
Validation to show duplicate document name

Refer below images:

Fixed in https://github.com/wso2/carbon-apimgt/pull/7035