Tuning Alerts from Security Hub

User Story
As a MP Engineer
I want to tune the alerts we are receiving from Security Hub to reduce duplication
So that it's easier to see the important findings from the service
Value / Purpose
As part of ticket #8076 we've enabled alerting for security hub issues that are categorised CRITICAL or HIGH.
Lots of alerts are being sent to the #modernisation-platform-security-hub-alerts slack channel and lots of incidents are being raised in the Security Hub Alerts - Modernisation Platform PagerDuty service.
Issues:

We seem to be getting multiple alerts for the same type of finding e.g. when you have a SSH port open to <IP_ADDRESS>/0 it triggers about 4 separate alerts as we have multiple standards applied e.g. AWS Foundational Security Best Practices, CIS AWS Foundations and PCI DSS v3.2.1. Here are some example alerts of this particular issue. This could be fixed by using consolidated control settings
We are getting multiple alerts because config rules associated with the standards are run periodically checking after the first time an issue is raised and triggering a follow-up alert. We could consider removing config rules or using SecHub Automation we can update statuses etc.
We are getting common alerts for rules across the 5 enabled regions of the baseline.

To consider:

Do we turn off some of the standards due to the overlap?
Do we suppress specific alerts where there is overlap?
Do we turn on consolidated control settings?
Can we stop receiving alerts from other regions (currently we are alerting from all 5 enabled baseline regions) if cross-region aggregation is already enabled?
Should we use Security Hub Automation rules to control the statuses of findings etc. to tame the alerts?

Useful Contacts
No response
Additional Information
No response
Definition of Done

[ ] Review alerts for overlap/duplication
[ ] Determine best ways of suppressing/taming alerts based on suggestion above
[ ] Take action on the agreed process
[ ] Test/Review alerts after making changes for improvement

Findings on Tuning Duplicate Alerts:
Enable Consolidated Settings:
By enabling consolidated settings within the organisation-security account, we can reduce duplicate alerts triggered by different standards, improving alert accuracy and clarity. This setting will allow all relevant standards to aggregate alerts into a single finding, thus minimizing noise and redundancy. Implementation of this consolidated setting needs to be coordinated with various teams, and Dave has taken the responsibility to discuss this with the respective teams to ensure a smooth rollout across our organization.
Security Hub in All Regions:
Although our infrastructure is not currently deployed in all regions, we are opting to keep the AWS Security Hub enabled in these regions. This approach ensures that we still receive alerts if any new infrastructure is created in currently unused regions, particularly if these resources present any security vulnerabilities. This proactive setup will alert us of any potential issues immediately, even in regions with minimal or no active infrastructure.
Suppression of Duplicate Findings:
Currently, there is no direct way to suppress duplicate findings within AWS, which contributes to alert noise. AWS has acknowledged this limitation and is actively working on a feature to address it. However, until AWS releases this functionality, we will need to manage duplicate findings as best we can with the existing configurations.
Automation Rules Limitations:
We explored using automation rules to reduce alert volume by adjusting severity levels. However, AWS automation rules only allow us to change an alert’s severity level; they do not support changing the workflow status once alerts are delivered to Slack. This limitation restricts our options for automatically managing alerts after initial notification, as we cannot adjust workflow statuses through automation rules alone.
Workflow Status to Slack:
We considered using a Lambda function to automatically change the workflow status to “notified” after sending alerts to Slack. However, we’ve decided against this approach. Receiving these alerts daily in Slack without changing their status allows our team to consistently monitor security alerts in real time. This daily feed ensures visibility, allowing us to stay on top of new and existing issues as they arise.
EventBridge and SNS Adjustment:
To further reduce duplicate alerts, I have removed EventBridge and SNS configurations in all regions except eu-west-2, as all alerts are already being centralized there. Previously, having EventBridge and SNS configured in multiple regions caused duplicate findings, as alerts were coming in from both eu-west-2 and the respective local region. With this change, we now receive findings from a single, centralized source, significantly reducing redundant alerts.

I updated the baseline module to restrict Security Hub alerting resources to eu-west-2 only, released a new tag, and updated the module reference in the MP repository. This ensures all alerts are centralized in eu-west-2, reducing duplicates across regions.
https://github.com/ministryofjustice/modernisation-platform-terraform-baselines/pull/650
https://github.com/ministryofjustice/modernisation-platform/pull/8425

I've reviewed this and am happy it's complete.


Feature Request: make a function that verifies the validity of a cookie?

Not too sure how to go about that considering there's no documentation for how the cookie is made.
Best I could do is check for ltuid to be numeric and ltoken to be alphanumeric but that sounds pretty useless.


Adds function to decode a String to a Double

thanks for the PR @Broich. my take, though, is that this is a bit too specific to include in the core library. rather, its a good candidate for a custom transformation in your own application.

@hkellaway Agreed and thanks for the link to the custom transformations. However, do you have an idea why the CI tests failed. I could not really make sense out of the error log other than it is some kind of a configuration issue.

hm, the CI fail is on a test that isn't impacted by your change. tests pass locally before the change and i also pulled your branch and tests passed locally as well. not sure why Travis isn't handling it :confused:


PullImageError

I'm getting the following error when attempting to create a pod in the environment:
bash-3.2$ kubectl get po
NAME             READY     STATUS           RESTARTS   AGE
ingester-hgdcr   1/2       PullImageError   1          29m

The image in question is just a public image being pulled from dockerhub:
      containers:
      - name: nginx
        image: scottganyo/nginx-sandwich-nginx

I've deployed the same file in other Kubernetes environments without issue... is there something different you're doing here than the standard setup?
Related: Is there a way I can just use local docker images in this K8s environment instead of going through docker hub?

@theganyo there is no difference then other Kubernetes cluster setups, coreos + k8s cluster bootstrapped via fleet units with go binaries. Not pulling images cloud be related to your network setup.
Do ssh to node1 (via menu) and try docker pull scottganyo/nginx-sandwich-nginx, it did work for me fine.
you can point to any local registry by e.g. <IP_ADDRESS>:5000/scottganyo/nginx-sandwich-nginx
btw my CoreOS OSX VM does have the private registry, the IP I showed above, and you can build your docker images there, this is what I'm exactly doing in my local setup

Thanks. I'm not an expert on Docker / Kubernetes (at least not yet). Are you saying that a private registry automatically runs by default in this system? Is that started on the k8smaster-01 box?

No worries, this is why we are learning new stuff.
I'm saying that private registry runs in this App https://github.com/TheNewNormal/coreos-osx on <IP_ADDRESS> IP where you can build your docker images.

Ah. Gotcha. I'll look into using that.
On more sort of related question that hopefully you can answer off-the-cuff: I'm trying to expose an external port to nginx from k8s. What IP address would I specify in my externalIPs setting?

externalIPis only for cloud providers which are supported by kubernetes, for local/bare-metal you need to use hostPort in replication controller

Thanks for your help!

no worries :)

Sorry, one more quick thing... you mentioned you use a local docker registry for your work... could you share your process for pushing your images into it and referencing it?

On CoreOS OSX VM shell do this:
bash-3.2$ docker pull alpine
Using default tag: latest
latest: Pulling from library/alpine
Status: Image is up to date for alpine:latest
bash-3.2$ docker tag alpine <IP_ADDRESS>:5000/alpine
bash-3.2$ docker push <IP_ADDRESS>:5000/alpine
The push refers to a repository [<IP_ADDRESS>:5000/alpine] (len: 1)
2314ad3eeb90: Image successfully pushed

ssh to one of kube-cluster nodes and do:
core@k8snode-01 ~ $ docker pull <IP_ADDRESS>:5000/alpine
Using default tag: latest
latest: Pulling from alpine
2314ad3eeb90: Pull complete
Status: Downloaded newer image for <IP_ADDRESS>:5000/alpine:latest

in Kubernetes replication controller then use image prefixed with <IP_ADDRESS>:5000/


Add Team Member to _data/team.yml Alvaro de Labaluce

This request is to add a new team member to the _data/team.yml file.
This will populate the project information pages, the team page card stack, and (potentially) the alumni page card stack.
1. Please upload a professional headshot to this issue.
2. Please fill in the content listed below:
❗Note
Fields that are bolded are required content, fields that are not formatted are recommended.
de Landaluce:
Alvaro:
Associated Projects:
**https://github.com/Landaluce**:
**https://www.linkedin.com/in/alvarodelandaluce/**:
3a. Please check the boxes that best match the description of the team member. Multiple boxes can be selected at once.
You can check a box by adding an x within the brackets.

[ ] Alumna
[ ] Alumnus
[x] Graduate Student
[ ] Undergraduate Student
[ ] Student Leader

3b. If Student Leader was selected, type in the role that they have:
Student Leadership Position:
4. What is the title that the member has in SSL? If unsure, leave blank.
Default Title:


Touchmove and srolling

Hi,
I do have an issue on touch devices when I call the draggable-points plugin :
On a touch device, I use a finger in the middle of the screen to scroll up or down. However, when there is a chart (and that de plugin is called - whether I made the chart draggable or not), I can't scroll over the chart. Or to be more accurate, if I'm scrolling I can continue over the chart, but if I stop, have a look at the chart, click on it, and then try to scroll away from it, it doesn't work.
The reason for that is that we have assigned something to the touchmove event : addEvent(container, 'touchmove', mouseMove);
So,

Can we let the touchmove event "free" for charts without draggable points ?
For a chart with draggable points, could we assign the touch event only to the columns for example (and not the full container) ?
Or could we think of an other way to handle the drag/drop events on smartphone (like double clicking, or cliking on the target position) ?

Thank you in advance for your help,
Adrien

Thank you for the reporting.
Please let me know if problem appears also without draggable-points extension? What kind of touch device do you use (platform / OS).

It does not happen if draggable-points is not called
I tried with Andoid 7.0 and Firefox 53.0, or Chrome 58.

I tested the demo https://jsfiddle.net/18jp1ste/show/ in two devices:

iPhone 6s (OSX 10.2.x)
Android 6

In both cases, when Im tapping on text Im able to scroll up and down, but when I touch on the chart, dragging is triggered. Let me know if we discuss about the same "behaviour".

Yes exactly. The dragging is trigger and consequently I can't scroll when I touch the graph

This is related with events, which are added on whole container:
addEvent(container, 'mousemove', mouseMove);         addEvent(container, 'touchmove', mouseMove);         addEvent(container, 'mousedown', mouseDown);         addEvent(container, 'touchstart', mouseDown);         addEvent(document, 'mouseup', drop);         addEvent(document, 'touchend', drop);         addEvent(container, 'mouseleave', drop);

Yes its actually due to the addEvent(container, 'touchmove', mouseMove); event.
Is there something we can do about it ?

I found a workaround :)
Replace :
function mouseMove(e) {             e.preventDefault();             if (dragPoint) {
by :
function mouseMove(e) {             if (dragPoint) {                           e.preventDefault();
It's good enough for what I need :)
Try with this website (sorry no english version yet) : http://www.electricite-2050.fr/


trimStart(Char) variant isn't exposed in Udon sharp anymore

Assets/USharpVideoSubtitles/Scripts/SubtitleManager.cs(545,75): Method is not exposed to Udon: 'text.Substring(text.IndexOf(@"{\an") + 6).TrimStart(' ')'

this could be fixed by changing to a different trimStart variant, my VRChat SDK - Worlds version is 3.5.0

v1.2.0 beta does not use that method so I would try that instead
Days since VRChat broke something in the SDK: 0

v1.2.0 beta does not use that method so I would try that instead I haven't updated to Unity 2022 yet so I'm not up to date with the changes

oh i just use TrimStart(new char[' ']) in the meantime, seems to work exactly like original code

This will be fixed in future release of VRChat: https://feedback.vrchat.com/sdk-bug-reports/p/350-beta1-stringtrim-stringtrimstart-and-stringtrimend-are-not-exposed-to-udon-i

Fixed in VRChat SDK 3.6.0.


Expand Testing: Compare Code dialog

Description:

The Compare Code Generation under the Testing menu needs to be extended. It should add support for Ruby and XRC code generation. Project.GetOutputType() should be used to disable radio buttons that would have no effect (e.g., there's no point in comparing Python files if none of the forms have a prop_python_file specified).

Comparison for Ruby has been added, but it's still missing XRC comparison.
Since this is for testing, and in most cases multiple languages are being tested (i.e., SID or QuickTest), I don't think disabling buttons for the other languages is important.


Add a Gitter chat badge to README.md

redfin/react-server now has a Chat Room on Gitter
@gigabo has just created a chat room. You can visit it here: https://gitter.im/redfin/react-server.
This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

Thanks @gitter-badger.  Closing in favor of #172.


chore: bdd-sbhg-1586630718 to 0.0.1

chore: Promote bdd-sbhg-1586630718 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


General support for connectionParams during websocket connection initialisation when operating Subscriptions

This PR makes changes that allow for the definition of connectionParams thereby facilitating token-based authentication over Subscription requests through websocket connections.
Following the standard set by the Apollo-GraphQL ecosystem, all Strawberry WS protocols now read the payload of the initial connection message as an internal connection_params field. This connection_params field is then injected into the context object with a general implementation for dictionary-based contexts at the top-level and specific implementations for each integration that define a more complex class-based context object, i.e. FastAPI and Channels.

Description

When the handle_connection_init hook is invoked by an instance of either BaseGraphQLTransportWSHandler or BaseGraphQLWSHandler, depending on the chosen protocol, the payload of the message is assigned to the self.connection_params instance attribute of either instance. This payload can be anything sent by the user, hence the loose typing Dict[str, Any].
Subsequently, when the Subscription is formally begun, by handle_subscribe in the case of BaseGraphQLTransportWSHandler and handle_start in the case of BaseGraphQLWSHandler, the self.connection_params instance attribute is injected into the context object as returned by self.get_context() on the condition that self.get_context() has returned a dictionary as its result.
The single reason for this condition surrounds the fact that some integrations, i.e. FastAPI and Channels, define class-based contexts rather than dict-based contexts. In these cases, specific functionality is added within their integration implementations to inject the self.connection_params variable into the class-based context within the hierarchy of Handler inheritances.
When an HTTP request is made, e.g. a Query or a Mutation, then dict-based contexts will lack the connection_params key while class-based contexts will have the connection_params attribute equal to None. Only when a Subscription request is made will this variable be populated with the payload of the initial connection message.
Types of Changes

[x] Core
[ ] Bugfix
[x] New feature
[ ] Enhancement/optimization
[x] Documentation

Issues Fixed or Closed by This PR

Closes https://github.com/strawberry-graphql/strawberry/issues/2351

https://github.com/strawberry-graphql/strawberry/issues/1703, potentially also. @kristjanvalur, let me know if you feel this PR alleviates this issue and whether connection_params achieves a similar goal to your PR.
Checklist

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] I have tested the changes and verified that they work and don't break anything (as well as I can manage).

Thanks for tagging me.  good stuff!
I'd like to remind you that queries and mutations also can happen over WS connections.  This is often the best way to do them in case you've already established a ws connection to deal with subscriptions.
in #1703 I was trying to find a way to reject a connection based on initially provided data.  Does the Appollo-GraphQL ecosystem have a way to do that?

@kristjanvalur, no problem at all!
You raise a good point, the connection_params field is populated whenever a websocket connection is initialised. If this is done using the GRAPHQL_TRANSPORT_WS_PROTOCOL then it will also be populated for Queries and Mutations since these will be transmitted over the websocket connection.
There is a key point here, however, which is that headers sent on Queries and Mutations will not work if the underlying protocol is GRAPHQL_TRANSPORT_WS_PROTOCOL due to the potential pitfalls as outlined in https://github.com/strawberry-graphql/strawberry/issues/2351 with headers over websockets. The client will need to be aware of this, do you agree?
With regards to your second point, I cannot find anything easily about rejecting subscriptions but my intuition would be that it is done using connection_params. For example, some credentials are sent on the websocket connection initialisation that is injected into the info.context object as connection_params. You can then check these credentials within your Subscription resolvers similarly to how one might check headers within the request object of info.context when working with Queries and Mutations over HTTP.
This may not be exactly what you're looking for, however, since this doesn't outright reject the connection but only rejects subsequent messages following the initial connection.

Great, just wanted to mention Queries and Mutations because of the way the PR description is worded.
Indeed, a client must be aware that it is making requests over WS, and that there can be no headers sent with individual requests in such a case.  (it physically cannot, there is no place to put headers when sending a WS message)
This brings up a third point:  If GRAPHQL_TRANSPORT_WS_PROTOCOL is enabled, then your resolvers need to work with both schemes.  This means that they cannot rely on headers to provide data to individual queries/mutations.  Headers and connection payload can only really be used for authentication and to provide constant options, and defaults for options that can be overridden by parameters.  An API needs to be designed with this in mind.
On a slightly related note, see issue #1754 illustrating how the get_context() FastAPI dependency cannot be assumed to have a single-query scope (and thus cannot be used to provide, e.g. database sessions)

Thanks for adding the RELEASE.md file!

Here's a preview of the changelog:

This change allows clients to define connectionParams when making Subscription requests similar to the way Apollo-Server does it.
With Apollo-Client (React) as an example, define a Websocket Link as so:
import { GraphQLWsLink } from '@apollo/client/link/subscriptions';
import { createClient } from 'graphql-ws';

const wsLink = new GraphQLWsLink(createClient({
  url: 'ws://localhost:4000/subscriptions',
  connectionParams: {
    authToken: user.authToken,
  },
}));

and the JSON passed to connectionParams here will appear within Strawberry's context as the connection_params attribute when accessing info.context within a Subscription resolver.

Here's the preview release card for twitter:

Here's the tweet text:
🆕 Release (next) is out! Thanks to Tommy Smith for the PR 👏

Get it here 👉 https://github.com/strawberry-graphql/strawberry/releases/tag/(next)

I agree with your comments, there's certainly some trickery involved with going between Queries, Mutations, and Subscriptions due to the difference between the underlying HTTP and WS protocols. Perhaps these problems might be alleviated if we were to support SSE instead of WS as requested here: https://github.com/strawberry-graphql/strawberry/issues/2091.

On a slightly related note, see issue #1754 illustrating how the get_context() FastAPI dependency cannot be assumed to have a single-query scope (and thus cannot be used to provide, e.g. database sessions)

This is especially true within the current context, so thanks for linking it!

I just want to add an example, to try to hammer home a point as part of this general discussion (not in the context of this PR):
We are using Strawberry in the context of a game project.  A client will establish a WS connection to listen to events.  Since it already has this connection, it will use it too to issue queries and mutations.   A separate, simpler, command line utility also uses the same GraphQL endpoint.  It doesn't required Subscriptions and uses HTTP to issue Queries and mutations.
GraphQL is specified independently of the transport.  And the resolvers in GraphQL cannot assume that they are being called with this transport or that.  This means that we cannot rely on per-request headers to provide metadata.  Similarly, any connection_params cannot be assumed to exist (except for Subscriptions, and only assuming that they only remain possible via WS).
For our use-case, it means that we restrict ourselves to HTTP Headers, and use those only to perform Authentication.
Because GraphQL is transport-agnostic, transport-specific data is tricky to use.

@tsmith023 thanks for this PR 😊
I haven't checked it properly yet because I was hoping @DoctorJohn would get to it (he's the most knowledgable person on subscriptions in Strawberry :D). But I'll try to review it over the  next few days :)

@patrick91, the type checkers failed on tests/pyright/test_union.py  because of this:
At index 0 diff: Result(type='error', message='Invalid expression form for type alias definition (reportGeneralTypeIssues)', line=14, column=26) != Result(type='information', message='Type of "UserOrError" is "Type[User] | Type[Error]"', line=16, column=13)

which is that the type attributes of each Result class are different here. Any idea why this could be?

@DoctorJohn, the type checkers test fail on the, seemingly unrelated, test: tests/pyright/test_union.py. It fails locally too!
Do you have an idea why this could be?

@tsmith023 this might be due to a pyright update, feel free to ignore it, I'll see if I can get #2212 in at some point this week :)

@DoctorJohn, what's the status of this PR? Are there any unresolved issues or outstanding changes that should be addressed?


Can't install with Luarocks

Hey, I tried to install this through with https://luarocks.org/ package manager. But it says
fatal: remote error:
#0 90.96   The unauthenticated git protocol on port 9418 is no longer supported.
#0 90.96 Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information.
As I further dug into this and found out that GitHub has updated their security measures and they no longer accept the git:// for package URL. Only HTTPS:// is allowed. As your Luarocks config file, it is still with the older version. I know you guys haven't worked with this for long now, but this is a bit important for me can you guys update this. I can add a PR if you want.

Done! There haven't been any changes to this because the underlying project is stable. Thanks for the detailed issue report...


Graceful exit before full execution for queries that would have very large responses

@andrewsu proposes finding a way to detect when a query is going to return a ridiculously high number of results and return an error (will return too many results, please narrow the scope of the query) and gracefully exit.
Examples include demo queries B.3a (around 100k results), C3 (45k results), and D6 (61k results). Look here for those queries
Andrew proposed that this could be detected during the sub-querying step: when a sub-query returns a massive amount of records. However, there are some concerns:

this would likely need to scale w/ the number of input IDs (1 ID in a subquery returning 10k records is diff from 1000 IDs in a subquery returning 10k records (which then means an average of 10 records per ID)
this wouldn't take into account post-query filtering (node attributes)
this wouldn't take into account query topology: perhaps there are a lot of records / output IDs for a step, but there will be an intersection (with another step or with the qNode's IDs) that will filter / whittle down the records a lot during edge-management...

@ariutta proposed calculating the potential size of the results, right before the results assembly step....and stopping there. The concern there would be that queries may still execute for a while (scale of 10-30 min) before hitting this step, so we may want to find where we can intervene earlier....

Builds on the caps issue and promiscuous node issues, but a different angle + intervention point...

If we went back to the older way that queryResult.update(records) worked where it send partial, incremental batches of records as they were received, we could potentially fail before waiting to get all records from every endpoint.

note: either have strict thresholds (say no we can't handle it) or give an option for user to override (say "no I want the super large response, give it to me")

Just as an FYI, here's a rough draft paragraph I put together to describe this issue:

Some queries return too many results to be useful and/or practical to handle in memory, and often these queries fail after a long time in processing. For a better user experience and lower load on our compute resources, it would be beneficial for a query of this type to fail early and let the user know that a more tightly constrained query is required. In cases where the number of results is large but still capable of being handled in memory, we may send the user a warning with an option to terminate the query, but in cases where the query is certain to fail, we would send the user an error message and immediately terminate the query. In both cases, the message to the user could include suggestions for how to modify the query to avoid the explosion in the number of results, such as pinpointing which part(s) of the query graph are most responsible for the explosion.

Leaving this here as a reminder for a future discussion. We may be able to efficiently detect whether a result set will explode by using bloom filters or cuckoo filters. For each endpoint, normalize the IDs it contains, load sets of IDs into a bloom filter and then serialize that bloom filter to save it for future use. Whenever a BTE instance is launched, it will load the serialized bloom filters and perform set intersections for the IDs from endpoints for each QNode in order to get an upper limit of number of results that can be returned. It won't give the actual results, just the number of results. The bloom filters will require less memory than trying to actually save the full sets of IDs.

@andrewsu and I had a discussion about this yesterday, a relatively simple solution that we will try and see how it works is to limit the results per edge in call-apis. This takes advantage of the behavior I noted in slack, where the call-apis package buckets the queries into groups to limit it to 3 concurrent calls per api. It then builds up an array of results as these 'query buckets' return results. When the length of this results array reaches a certain threshold, we can stop querying for this edge and send back the partial results. We can then query the next edge using these partial results and continue repeating this process. In theory, if we have a limit of 1000 results per edge, then a query that goes 1 -> 2000 -> 50000 could be turned into 1 -> ~1000 -> ~1000. To give the user control of this, we could let them set this parameter using a query parameter.
@ariutta Interesting idea, I don't know how bloom filters work but do you need to make the queries first to get the number of results or is this done another way?

@ericz1803 I have tried the linked PRs and discussed with @andrewsu...there are 3 areas that need clarifying / some more discussion.
First, to clarify:

What is being used for the restriction / limitation? It looks like records (which are later transformed into Edges)...
It would help to have clearer documentation on the PR (see above point). Also, it would help to say that the default is 1000 and one would adjust this by making setting a query parameter as max_results_by_edge with value as an int (-1 to retrieve all).

Second, I ran the demo queries and this code does not seem to be working as-intended.

I believe we wanted a limit on queries that had >1000 results or so. However, the limit is kicking in for queries that have <1000 results and leading to fewer results...
This limit doesn't lead to a predictable number of results, which makes it a bit difficult to work with and explain.
Andrew and I were wondering:

increasing the default limit
trying a different approach like:

doing ID-resolution in increments - which may help us stop at "1000 output nodes" for each hop
"incremental results-assembly", which would allow a limit on the number of results

This is my analysis, with concerning behavior in bold:

A2: 966 results (VS 4344 w/o restriction). Hit 1000 limit on 2nd hop.
B.2a: 797 results (VS 1441 w/o restriction). One-hop query.
B.2b: 613 results (VS 657 w/o restriction). One-hop query.
B.2d: 805 results (VS 1181 w/o restriction). One-hop query.
B.2e: 783 results (VS 1202 w/o restriction). One-hop query.
C2: 20 results (VS 93 w/o restriction). Hit 1000 limit on e4?
C3: 609 results (VS thousands w/o restriction). Hit 1000 limit on e3?
D1: 1 result (VS 743 w/o restriction)
D2: 108 results (VS 2565 w/o restriction)
D6: 76 results (VS thousands w/o restrictions)

Third, the demo query D3 hits an error....and it looks like it's related to the restriction and getting 0 results

logs
```
  bte:call-apis:query id annotation completes +1s
  bte:call-apis:query Query completes +0ms
  bte:biothings-explorer-trapi:batch_edge_query BTEEdges are successfully queried.... +9s
  bte:biothings-explorer-trapi:batch_edge_query Filtering out any "undefined" items in (1000) results +0ms
  bte:biothings-explorer-trapi:batch_edge_query Total number of results is (1000) +0ms
  bte:biothings-explorer-trapi:batch_edge_query Start to update nodes... +0ms
  bte:biothings-explorer-trapi:batch_edge_query Update nodes completed! +1ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (6) Storing results... +9s
  bte:biothings-explorer-trapi:UpdatedExeEdge (6) Applying Node Constraints to 1000 results. +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (6) No constraints. Skipping... +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Updating nodes based on edge results... +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Updating Entities in "e00" +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Collecting Types: "["SmallMolecule"]" +1ms
  bte:biothings-explorer-trapi:UpdatedExeEdge Collected entity ids in results: ["SmallMolecule"] +4ms
  bte:biothings-explorer-trapi:NewQNode (8) Node "n00" restored curie. +9s
  bte:biothings-explorer-trapi:NewQNode Node "n00" intersecting (6)/(314) curies... +0ms
  bte:biothings-explorer-trapi:NewQNode Node "n00" kept (0) curies... +3ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Updating Entities in "e00" +4ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Collecting Types: "["Disease"]" +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge Collected entity ids in results: ["Disease"] +12ms
  bte:biothings-explorer-trapi:NewQNode Node "n01" intersecting (3)/(1) curies... +13ms
  bte:biothings-explorer-trapi:NewQNode Node "n01" kept (1) curies... +0ms
  bte:biothings-explorer-trapi:edge-manager 'e00' Reversed[true] (0)--(1) entities / (1000) results. +9s
  bte:biothings-explorer-trapi:edge-manager 'e00' dropped (1000) results. +12ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (6) Storing results... +12ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (6) Applying Node Constraints to 0 results. +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (6) No constraints. Skipping... +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Updating nodes based on edge results... +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Updating Entities in "e00" +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Collecting Types: "["SmallMolecule"]" +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge Collected entity ids in results: [] +0ms
  bte:biothings-explorer-trapi:NewQNode Node "n00" saving (0) curies... +12ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Updating Entities in "e00" +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge (7) Collecting Types: "["Disease"]" +0ms
  bte:biothings-explorer-trapi:UpdatedExeEdge Collected entity ids in results: [] +0ms
  bte:biothings-explorer-trapi:NewQNode Node "n01" intersecting (1)/(0) curies... +0ms
  bte:biothings-explorer-trapi:NewQNode Node "n01" kept (0) curies... +0ms
  bte:biothings-explorer-trapi:edge-manager Updating all other edges... +0ms
  bte:biothings-explorer-trapi:main (10) Edge successfully queried. +9s
  bte:biothings-explorer-trapi:edge-manager (11) Collecting results... +0ms
  bte:biothings-explorer-trapi:edge-manager (11) 'e00' keeps (0) results! +0ms
  bte:biothings-explorer-trapi:edge-manager ---------- +0ms
  bte:biothings-explorer-trapi:edge-manager (12) Edges ["e00"] resulted in (0) results. No complete paths can be formed. +0ms
  bte:biothings-explorer-trapi:edge-manager (12) Collected results for: []! +0ms
  bte:biothings-explorer-trapi:edge-manager (12) Collected (0) results! +0ms
  bte:biothings-explorer-trapi:edge-manager (13) Edge Manager reporting combined results... +1ms
  bte:biothings-explorer-trapi:Graph Updating BTE Graph now. +8m
  bte:biothings-explorer-trapi:edge-manager (13) Edge Manager reporting organized results... +0ms
  bte:biothings-explorer-trapi:QueryResult Updating query results now! +8m
  bte:biothings-explorer-trapi:error_handler TypeError: Cannot destructure property 'connected_to' of 'dataByEdge[queryEdgeID]' as it is undefined.
  bte:biothings-explorer-trapi:error_handler     at QueryResult._getPreresults (/Users/colleenxu/Desktop/bte-trapi-workspace/packages/@biothings-explorer/query_graph_handler/built/query_results.js:109:17)
  bte:biothings-explorer-trapi:error_handler     at QueryResult.update (/Users/colleenxu/Desktop/bte-trapi-workspace/packages/@biothings-explorer/query_graph_handler/built/query_results.js:237:14)
  bte:biothings-explorer-trapi:error_handler     at TRAPIQueryHandler.query (/Users/colleenxu/Desktop/bte-trapi-workspace/packages/@biothings-explorer/query_graph_handler/built/index.js:185:27)
  bte:biothings-explorer-trapi:error_handler     at runMicrotasks ()
  bte:biothings-explorer-trapi:error_handler     at processTicksAndRejections (node:internal/process/task_queues:96:5)
  bte:biothings-explorer-trapi:error_handler     at async task (/Users/colleenxu/Desktop/bte-trapi-workspace/packages/@biothings-explorer/bte-trapi/src/routes/v1/query_v1.js:38:13)
  bte:biothings-explorer-trapi:error_handler     at async /Users/colleenxu/Desktop/bte-trapi-workspace/packages/@biothings-explorer/bte-trapi/src/controllers/threading/threadHandler.js:86:38 +0ms
```

To address some of your points:

The restriction is being applied on the call-apis results (which is pre-filtering/intersecting). This is why the ones marked with the triangle behave the way they do. Call-apis returns 1000 query results, which then might get dropped/filtered in the process of querying by the query graph handler/edge manager leading to less than 1000 results being sent back to the user.
Increasing the default limit: Easy as changing a number in the config. The thing we would have to consider with finding the ideal limit is balancing getting as many good results back as possible with setting it too high and sort of defeating the purpose if it takes just as long.
D3: You're right, it is being caused by the 1000 query results leading to 0 actual results.

Different approach:

I do agree that it would be better behavior to stop at returning 1000 output nodes back to the user instead of a variable and unpredictable number.
The pros of this current solution is that it is the simplest to implement with the current code and it limits the number of unnecessary api queries which is one of the most time consuming parts. However, the con is unpredictability for the user as you mentioned, sometimes the 1000 gets filtered down to 966 and other times it gets filtered down to 0.
Incremental ID Resolution: Maybe I'm misunderstanding the meaning of doing ID resolution in increments, but right now it is done in call-apis right after getting back the results and it doesn't change the size of the results array sent back to the query graph handler.
Incremental results assembly: I think we would probably have to take the incremental results assembly approach if we want to get this desired behavior. Maybe someone can explain the details of how it would work exactly because I don't understand it too well. My questions for it would be if we can limit the API calls without having to make all of them and how it would perform intersections incrementally.

Incremental results assembly may be the long-term solution, but I'd be hesitant to try getting that done in the next few days. It could involve quite a few changes to different parts of the codebase.

Is there a way to make the limit on the edge dynamic (like 7000 records per input ID)?
The number above comes from what I'm running: I'll post an update soon but the limit needed to get all 16 results for demo query D3 is between 15k-20k (a one-hop explain w/ 6 IDs on one node, 3 IDs on the other node)

Call-apis returns 1000 query results, which then might get dropped/filtered in the process of querying by the query graph handler/edge manager leading to less than 1000 results being sent back to the user.

To me, this dropping/filtering step is the key. I was hoping/assuming that the consolidation of edges to results would be at least somewhat predictable; maybe 1000 edges generally gets consolidated to ~900 results.  But clearly in practice, that varies quite a bit.  In deciding the path forward, I'd like more clarity on two issues:

What is happening during the dropping/filtering step?  I was imagining that this was primarily the consolidation of multiple edges/records with the same subject-predicate-object (SPO) triple into a single result, but that wouldn't explain how 1000 edges corresponds to zero results.
What would be the workflow for doing incremental building of results?

My guess is that the second item (and possibly the first) will be easiest to discuss live, perhaps at our Wednesday meeting...

I checked a couple of the queries and it looks like you're right, consolidation in the results building step looks like the main reason for the lower number of results. For B.2b, 1000 records are sent to the query results builder which turns them into 613 results sent back to the user. For predict queries (the first 5 that @colleenXu listed), this looks like the main cause. For explain queries (the rest), my guess is that it is a combination of intersecting IDs and this results consolidation. For D3 specifically, it is a one hop explain so that is probably why that is happening.

Here's my findings from increasing the limit (see the full google sheet here), with key findings with this symbol 📗
My impression is that a flexible limit dependent on the number of input IDs for the edge may be helpful.

For A2 (two-hop Predict), one gets the full results (4344) when 5k < limit <= 16k.
One-hop Predicts (B.2a, B.2b, B.2d, B.2e, C1) had full results when 1k < limit <= 5k.
For C2 (complicated topology), one gets the full results (93) when 1k < limit <= 5k.
📗 C3 (complicated topology) and D6 (3-hop Explain) run faster, get fewer results with a limit set
📗 D1 (2-hop Explain, "find everything related to both of these well-annotated diseases") gets full results (743) only when 41k < limit <= 42k
D2 (2-hop explain) got the full results (2565) when 16k < limit <= 20k
📗 D3 (1-hop Explain) gets a result/no error at 9k. It gets full results (16) when limit ~ 16k

Also! Setting the current limit to "1000 results out of the call-apis module) for a 1-hop Predict query doesn't result in...

1000 output nodes
1000 edges
1000 results (already mentioned above)

I found it surprising that it didn't lead to 1000 edges...
I got this info by checking the knowledge-graphs for one-hop predicts (B2a / B2b / B2d / B2e)...

Notes from meeting:

@ariutta looks at query-results.js in query-handler and adds logs....to help us figure out what is happening to the "1000 results of call-apis module". Specifically, how does this become <1000 edges?

From my perspective...

setting a high limit like 20k might achieve what we want

The current approach seems to revolve around "cutting off" the sub-querying / api calls. However, I'm a bit uncomfortable after seeing it in action: see C2 and D1 for examples.

For an Explain-type query / more complicated topology where intersections of nodes will happen, it is difficult to predict how high to set the limit to get the nodes that will survive the intersecting process

For the "assemble results as we do the sub-querying for an QEdge", I see a lot of difficulty. I think we hit the same issue as above (we don't know how high to set a limit when an intersection is going to happen)

we may want to use different approaches like setting a lower cap than we currently have or pruning promiscuous nodes at the end of each hop.

@ericz1803, @colleenXu and @marcodarko, here's a list of lines that might be useful for logging in query_results.js:

L284: debug(`${preresults.length} preresults found`); This is the number of connected subject, predicate, object, ..., subject, predicate, object paths along the query graph. It corresponds to the number of results if we would get if we never did any consolidation.
L448: debug(`${consolidatedPreresults.length} consolidatedPreresults found`); This is the number of results after consolidation.

We could add additional logging if desired to indicate which preresults are consolidated based on multiple edges for the same primary IDs, but that could be overwhelming, because there could be a large number of these.

Regarding incremental results assembly, we discussed two ways it could be done:

call queryResult.update every time we get records from an API
call queryResult.update every time we get all records for a QEdge

I think both are technically possible, but 2) would probably be a little more complex in terms of developer time required. If done properly, I think for both cases the runtime performance would be the same or slightly better than our current algorithm.
We also discussed two issues for why the number of results can be less than the specified cutoff limit -- consolidation and dead-ends.
Regarding consolidation, is this usually a big deal? If we set a limit of 1000, and some queries return 1000 results where each edge_binding has one item but other queries return 500 results where each edge_binding has two items, does that matter?
Regarding dead-ends, this may be uncommon, because it only happens for more complicated query graphs, like the one illustrated in the figure below. In this case, if we made API calls to get records for every QEdge except e3, incremental results assembly would allow us to skip making any API calls to get records with subject U for e3. Without incremental results assembly, we currently would make the "wasted" API call that gets record U->P.

not relevant given recently implemented limits on time and record counts


bun_python plugin freezes and segfaults

Following the instructions in the README, I created bunfig.toml and preload.ts. However running the demo file causes bun to freeze and then
362550 segmentation fault (core dumped)  bun run index.ts

The non-plugin version works fine (For now, I am making do with it and with a slightly bulkier .d.ts file)

I think it is some bug in bun itself, the plugin system has some bug (especially for return object one)... (I suggest use debug version of bun to get verbose error info, and report it to oven/bun)

@codehz
I seem to also be getting a segfault when python code tries to callback into Javascript code, or return values to Javascript code.
Seems like it is related to bun_python missing these important calls:
c.PyEval_InitThreads()
or
c.PyGILState_Ensure()
Related discussion:
https://stackoverflow.com/questions/4866701/python-pygilstate-ensure-release-causes-segfault-while-returning-to-c-from-p
My environment is bunjs on Ubuntu Jammy x86-64.
Also happens sporadically on another machine which is a MacBook Pro M1.

// initialize the Python interpreter
Py_Initialize();

PyEval_InitThreads();

/* Swap out and return current thread state and release the GIL */
PyThreadState tstate = PyEval_SaveThread();

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

PyRun_SimpleString("import random\n");

PyGILState_Release(gstate);

From: https://bugs.python.org/msg57619

Submitted PR #2

I just noticed that there may be another cause of the crash... not just the GIL issue

Care to elaborate?

@heri16 has been fixed, related to gc
https://github.com/codehz/bun_python/commit/e28c1824505eba9c90fcd6864e32fc6a86518bdd#diff-08b76b1156c14219681dc54823e0b39fd0e2e719ead10a5ebda8aeeb312fdf4aR224-R228

Found the problem, it is due to python.callback() not returning a Callback that holds a reference to the PyObject, causing GC to happen on the PyObject.
Will be submitting a PR soon.

Submitted PR #4


Feature/esp button

PTAL

@alibukharai I tried to build the example.ino but got many errors. Did you try this before?

@alibukharai I tried to build the example.ino but got many errors. Did you try this before?

Yes, I just checked, and it appears that some headers are missing. To resolve the issue, we should make the following changes:

Include <Arduino.h> in a code because we are using constants like "Pullup" from it.
#include <Arduino.h>

In  .cpp file, we need to include the "Button.h" header at the beginning. This ensures that the class definition is visible to the implementation code:
#include "Button.h"

These changes should resolve the compilation errors. I will push the changes with test app on monday. Happy weekend :)


recreate rigging flags for mesh data

[x] max influences
[ ] unk1

Unk1 is sometimes set to 0 for meshes with no bone influences and is always 1 for meshes with bone influences. Setting unk1 to 0 for meshes with bone influences causes crashes in Ryujinx. Setting unk1 to 0 for meshes without bone influences doesn't seem to have any noticeable effect.


'optional' file not found

Description
Xcode is throwing

'optional' file not found
'utility' file not found

errors during compilation(after new architecture is enabled). From the initial looks, this seems to be a C++ compiler compatability issue, but couldn't narrow down why or how to fix it.
Steps to reproduce

Create a React Native project with 0.72.6
Configure RN Bootsplash
Enable New architecture

React Native Version
0.72.6
Affected Platforms
Runtime - iOS
Areas
Codegen
Output of npx react-native info
System:
  OS: macOS 13.0
  CPU: (10) arm64 Apple M1 Pro
  Memory: 100.47 MB / 16.00 GB
  Shell:
    version: 5.8.1
    path: /bin/zsh
Binaries:
  Node:
    version: 16.15.0
    path: ~/.asdf/installs/nodejs/16.15.0/bin/node
  Yarn:
    version: 3.2.0
    path: /usr/local/bin/yarn
  npm:
    version: 8.5.5
    path: ~/.asdf/plugins/nodejs/shims/npm
  Watchman:
    version: 2<IP_ADDRESS>
    path: /opt/homebrew/bin/watchman
Managers:
  CocoaPods:
    version: 1.13.0
    path: /Users/ananthakumar/.asdf/shims/pod
SDKs:
  iOS SDK:
    Platforms:
      - DriverKit 22.4
      - iOS 16.4
      - macOS 13.3
      - tvOS 16.4
      - watchOS 9.4
  Android SDK:
    API Levels:
      - "27"
      - "28"
      - "29"
      - "30"
      - "31"
      - "32"
      - "33"
    Build Tools:
      - 28.0.3
      - 29.0.2
      - 30.0.2
      - 30.0.3
      - 31.0.0
      - 32.0.0
      - 32.1.0
      - 33.0.0
    System Images:
      - android-28 | Google APIs ARM 64 v8a
      - android-29 | Google Play ARM 64 v8a
      - android-32 | Google APIs ARM 64 v8a
      - android-33 | Google APIs ARM 64 v8a
    Android NDK: Not Found
IDEs:
  Android Studio: 2022.2 AI-222.4459.24.2221.10121639
  Xcode:
    version: 14.3.1/14E300c
    path: /usr/bin/xcodebuild
Languages:
  Java:
    version: <IP_ADDRESS>
    path: /usr/bin/javac
  Ruby:
    version: 2.7.4
    path: /Users/ananthakumar/.asdf/shims/ruby
npmPackages:
  "@react-native-community/cli": Not Found
  react:
    installed: 18.2.0
    wanted: 18.2.0
  react-native:
    installed: 0.72.6
    wanted: 0.72.6
  react-native-macos: Not Found
npmGlobalPackages:
  "*react-native*": Not Found
Android:
  hermesEnabled: true
  newArchEnabled: true
iOS:
  hermesEnabled: true
  newArchEnabled: true

Stacktrace or Logs
node_modules/react-native/Libraries/RCTRequired/RCTRequired/RCTRequired.h:8:10 'utility' file not found

node_modules/react-native/Libraries/TypeSafety/RCTConvertHelpers.h:8:9 'optional' file not found

ios/Pods/Headers/Public/RCTRequired/RCTRequired/RCTRequired.h:8:10 'utility' file not found

~/Library/Developer/Xcode/DerivedData/Jupiter-dpakkcsiqsoortecrrkkegznetck/Build/Products/Debug-iphonesimulator/React-Codegen/React_Codegen.framework/Headers/RNBootSplashSpec/RNBootSplashSpec.h:15:2 This file must be compiled as Obj-C++. If you are importing it, you must change your file extension to .mm.

Reproducer
https://github.com/zoontek/react-native-bootsplash/tree/5.1.1/example
Screenshots and Videos

Xcode:
version: 14.3.1/14E300c

Could you update to XCode 15 and try again @Ananthakr-Jupiter ?

Getting the same error in Xcode 15 @cortinico

We have abandoned the new architecture migration at that time. We are exploring to move to 0.76 again now, if we got any further updates, will update in your thread.

@Ananthakr-Jupiter Be sure that your AppDelegate.mm correctly compile as ObjC++


Esmfold

Description
Adds support for retrieving predicted structures from ESMFold.
Pull Request Checklist

[ ] Added a note about the modification or contribution to the ./docs/sources/CHANGELOG.md file (if applicable)
[x] Added appropriate unit test functions in the ./biopandas/*/tests directories (if applicable)
[x] Modify documentation in the corresponding Jupyter Notebook under biopandas/docs/sources/ (if applicable)
[x] Ran PYTHONPATH='.' pytest ./biopandas -sv and make sure that all unit tests pass (for small modifications, it might be sufficient to only run the specific test file, e.g., PYTHONPATH='.' pytest ./biopandas/classifier/tests/test_stacking_cv_classifier.py -sv)
[ ] Checked for style issues by running flake8 ./biopandas

@a-r-j It looks all good to me! Just added source='esmfold-v1' as a requirement if someone fetches a sequence. Just so that people now what they are fetching it from.
I noticed that the PDB jupyter notebook is corrupted though. Couldn't open it on my computer, and I also see that that it's the same for Gh rendering: https://github.com/rasbt/biopandas/blob/4515e39861a422c10a8aff7677ecd5aacb031508/docs/tutorials/Working_with_PDB_Structures_in_DataFrames.ipynb
Do you have a working version on your computer still? Othewise I will restore it from an earlier commit.

Oh gosh, let me see what I can do. Keeping notebooks under version control is always a nightmare.

Seems I hadn't resolved all the conflicts when merging. Should be fixed now :)

Thanks, I will try to check that out today! I will also take care of the docs. But just if you are curious, there are a few notes here: https://biopandas.github.io/biopandas/CONTRIBUTING/#notes-for-the-developers
PS: I transferred the repo to a new BioPandas org. Looks like everything worked pretty smoothly so far

I think you undid my commits where I added source='esmfold-v1' as a requirement including unit tests 😢

Oh gosh, my bad. Looks like I showed up to amateur hour :sweat:
I can't find your changes in the commit history - any chance you have a local copy?

No worries, thinks happen. Unfortunately, I destroyed them when I pulled your recent branch ... arg


[NEW CONTENT TYPE REQUEST] COLLADA

What type of file would you like magika to detect?
COLLADA file (.dae)
What software can create/open these files?
Many 3D modeling softwares.
Where can these files be found?
COLLADA.zip
If possible, please provide a specification for this file type.
https://www.collada.org/2008/03/COLLADASchema
Additional context
The included files are from examples of complete COLLADA models, given in the COLLADA Schema.

Closing, see #152 for context. Thanks!


Azure get_blob_properties not ideal to check for blob existence

Currently, one can use get_blob_properties in order to check for existance of a certain blob. Unfortunately, it is not as straightforward as it could be.
Since client.get_blob_propertes(key) always returns a Blob, it's unclear how one can check for existance. The fact that Blob has a present? method would make it seem one can do client.get_blob_propertes(key).present?, but Http always raises an exception if the status code is not a success. Therefore, a 404 response raises an exception and never makes it here.
Currently, one has to rescue errors:
def blob_exists?(key)
  client.get_blob_propertes(key).present?
rescue AzureBlob::Http::FileNotFoundError
  false
end

Ideally, one should not need to use exceptions for flow control. It might be clearer to have a blob_exists? method on AzureBlob::Client, that currently might even rescue the exception like above, but hide such details from the end user.
What do you think?

I also realized that one could avoid catching the exception if get_blob_properties would internally call Http.new with , raise_on_error: false (which probably should be the case, for consistency with get_container_properties. Then, client.get_blob_propertes(key).present? would work to check for existence.
On the other hand, it would be better to select on 404 HTTP errors and raise on other errors. Calls to get_blob_properties and get_container_properties should still raise on anything that is not a 404, otherwise a 500 (or any other error) would be indistinguishable from "blob/container does not exist".
I feel that the best course of action would be:

Create methods called blob_exists? and container_exists?, which would call the API to get properties and return true/false depending on whether the response is a 200 or 404. Other 4xx and 5xx errors should still be raised as exceptions (e.g. if the API is down, or if the container or blob name is invalid)
The methods get_blob_properties and get_container_properties could raise on all errors (as get_blob_properties does currently) or, if necessary for compatibility with paperclip, catch only 404 responses and return a Blob/Container that returns false on present?. They should still raise other HTTP errors, as those should not be conflated with a missing container or blob.


how can i using this software after installed local, need login, cannot REGISTRATION

I have recently installed your software locally and I'm encountering issues with the registration process. , but I'm facing two main issues: I'm unable to complete the registration process due to connectivity problems.

The registration is processed through your local MYSQL database. Have you
built the MYSQL databases on your server?
On Sun, Nov 3, 2024 at 4:31 AM zengqd @.***> wrote:

I have recently installed your software locally and I'm encountering
issues with the registration process. , but I'm facing two main issues: I'm
unable to complete the registration process due to connectivity problems.
capture_20241103172632394.jpg (view on web)
https://github.com/user-attachments/assets/caaeb72e-b6d5-41a0-bcd6-ef4de77eb1c8
—
Reply to this email directly, view it on GitHub
https://github.com/whitehead/sirna/issues/1, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACI44XTNUFREBO7WJHMQTADZ6XUOLAVCNFSM6AAAAABRCR52C6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGYZTCMBYHAZDENA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Bingbing


chore(master): release 1.2.17

:robot: I have created a release beep boop
1.2.17 (2023-08-05)
Miscellaneous

deps: update dependency phpstan/phpstan-deprecation-rules to v1.1.4 (22dfa9d)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/wayofdev/laravel-symfony-serializer/releases/tag/v1.2.17 :sunflower:


[Sysman]Report memory and GPU power

To display GPU and Memory power for ARC control GUI
https://hsdes.intel.com/appstore/article/#/14019156039

Add new sensor enum - zes_power_domain_t.ZES_POWER_DOMAIN_MEM and zes_power_domain_t.ZES_POWER_DOMAIN_GPU

missed seeing this issue earlier, we could close this as duplicate of #214


FeatureRequest: autoconfig

Hi, if I get this working, I plan to add a web server container to render the autoconfig/mail/config-v1.1.xml files so email clients can be auto-configured.
Example of the xml file:
  <emailProvider id="domain.com">
    <domain>domain.com</domain>
    <displayName>My Mail</displayName>
    <displayShortName>mymail</displayShortName>
    <incomingServer type="imap">
      <hostname>mail.domain.com</hostname>
      <port>143</port>
      <socketType>STARTTLS</socketType>
      <authentication>password-cleartext</authentication>
      <username>%EMAILADDRESS%</username>
    </incomingServer>
    <outgoingServer type="smtp">
      <hostname>mail.domain.com</hostname>
      <port>587</port>
      <socketType>STARTTLS</socketType>
      <authentication>password-cleartext</authentication>
      <username>%EMAILADDRESS%</username>
    </outgoingServer>
  </emailProvider>

We could render that xml on the start-mailserver.sh script and share that file with the new web server, mounting it in both containers.
Do you like the idea of adding this to the project? I will have to do it anyway, so if you want I can PR it

Hi @citec, do you know which email client are currently supporting these xml files?

I have been using it in thunderbird and it works great, just put email and pass, press next and it finds out everything. For outlook a different file is needed, I have never done that, but it seems easy:
https://i-mscp.net/index.php/Thread/3771-Mail-autoconfiguration-for-Thunderbird-and-Outlook/

Ok. Something seems to exist for Apple Mail: https://developer.apple.com/library/ios/featuredarticles/iPhoneConfigurationProfileRef/Introduction/Introduction.html
Regarding your implementation, why a webserver and not a command line tool?

webserver because, the email clients query those files using http. Not sure about all the different email clients, buy I am sure that thunderbird does, for example.
We could do similar as you do with start-mailserver.sh into a new container for the web server: the xml files will be rendered based on domainname, hostname and the accounts.cf file.
I prefer new container as it is a different service, even if they both share common accounts.cf file

I do agree, nice feature but in another image.

Let me know when it's done, I'll test it and link it in the documentation.


Added the dev15-rc project system branch.

Tagging @dotnet/roslyn-infrastructure @mmitche for review. /cc @dotnet/project-system.

👍


Support for .php

VS Code doesn't autocomplete block comments by default so PHP would be super handy

My current implementation is not working in PHP for some reason. It works in every other language, so I wonder if PHP has a separate code engine or something. I'll have to look into it, but it's not possible for now.

Actually, I figured this one out, it was my fault. Expect this to be added in the next update.

The new update has PHP support. Please open a new issue if you find any new bugs.


Every project close operation fails to close cleanly

Noticed the following error every time a project is closed in Language Server:
Closing text file [object Object]: Language Server request 'text/closeFile' failed: Error: Request timeout request took longer than 15000 ms to resolve

The similar problem is described here: https://github.com/enso-org/enso/issues/10381#issuecomment-2194643132, Probably similar fix to #10421 is needed in ydocs.
The dashboard closes LS and GUI app, usually LS finishes first, and then ydocs tries to do cleanup and close the file - but the connection is gone.

After some debugging, this feels like like a GUI bug. By the time ydoc wants to dispose LS session, websocket is already closed. Besides, LS on its will ensure that file is closed on a clean shutdown.

After some debugging, this feels like like a GUI bug.

Yes, I'm actually surprised you were assigned to this :)

By the time ydoc wants to dispose LS session, websocket is already closed.

Yes, and that's because GUI called project/close. ydoc server does not know about it, however, and cannot distinguish closed connection because of project close from connection interruption (due to hibernation, for example).

LS on its own will ensure that file is closed on a clean shutdown.

I think ydoc server should still clean up. Technically, still other client could be connected.


Add X-Scope-Probe-ID header to POSTs

/cc @2opremio @paulbellamy

Would something like the hostname be more meaningful to the user?

Hmm, I can't think of a normal scenario where the hostname wouldn't work... will make the change.

Fix comment, please, then LGTM

Test failure seems to be GCE related.


[core] fix DP issue

What does this PR do?
The mini batching PR introduced a small bug that led to DP crash using accelerate as we tried to all reduce tensors that did not had the same shape.
The tensors that did not had the same shape were the logprobs tensors, that derive from the inputs.
The fix is to add a check, that padds correctly the model inputs, and corresponding attention masks, and fills them with the correct value using accelerator.pad_across_processes
cc @lvwerra
run for gpt2 sentiment with DP=2: https://wandb.ai/distill-bloom/trl/runs/p7pzc7yk?workspace=user-younesbelkada
run for t5 sentiment with DP=2: https://wandb.ai/distill-bloom/trl/runs/nb9as16x?workspace=user-younesbelkada

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

A CI test (test_ppo_step_with_no_ref_sgd_lr_scheduler) is surprisingly failing but completely unrelated to this PR, I thought it was an issue with PT 2.0 but still cannot reproduce locally. Will merge like this and investigate later


Edge Not Saving Cookies From Signin-Oidc

Issue / Steps to reproduce the problem
We have an Identity Server 4 implementation and our front end application on different subdomains of the same domain (e.g. identity.somedomain.com and sales.somedomain.com).
In Chrome and Firefox, we are able to login just fine without issues.  The Authorize middleware redirects us to identity when we first reach the site, it allows us to login, redirects us back to sales, and then it succesfully authorizes and loads the page that is behind the authorize tag.
In Edge and IE11, however, it will get in a redirect loop which Edge/IE11 detects and kills.  After login, it will navigate to sales, then authorize fails and it tries to renavigate to identity, which would see it's already authorized and redirect back to sales.
I pulled the network traffic, particularly focusing on the cookies because that's a likely suspect, and found that when the redirect reaches sales/signin-oidc:
IE:

Posts to sales/signin-oidc which responds with the authorize cookies I expect.
Gets to sales/signin-oidc with no cookies which returns a 500 (not sure why it does this)
Gets to sales, with no cookies, responds with a redirect to identity

Chrome:

Posts to sales/signin-oidc which responds with the authorize cookies I expect
Gets to sales, with the cookies, responds with the site

Edge noticeably does two things: does not set the cookies that the signin-oidc responded with, and makes an additional get call to signin-oidc.
The only major difference I see is that the chrome cookies have a samesite=lax; on them that the edge log does not have.
What I think is happening is something is preventing edge from saving the cookies.  However, I'm at a loss for what to even look for.
Logs attached.  Please let me know what other information would be useful.  This cannot be replicated locally, only on our servers (seems like edge "trusts" the cookies if they both come from localhost)
Versions:
.Net Core 2.1
IdentityServer4.AspNetIdentity 2.3.0
IdentityServer4.EntityFramework 2.3.2
Microsoft.AspNetCore.App 2.1.8
Relevant parts of the log file
Attached (looks like the upload ate the formatting, sorry :( )
chrome.txt
edge.txt

Perhaps relevant: https://brockallen.com/2019/01/11/same-site-cookies-asp-net-core-and-external-authentication-providers/


_hover selector for buttons doesn't alter background color

Hello! I was playing around with the _hover selector as described in the docs here, and noticed that the color scheme for the button's background doesn't change. Demo:
const links = [
  { text: 'Home', to: '/' },
  { text: 'About', to: '/about' },
  { text: 'Login', to: '/login' }
];

[REDACTED]

<For each={links} fallback={
Loading...
}>
  {(link) => (

      <HopeLink href={link.to} fontSize="10px" class="p-1 m-1">
        <Button colorScheme="primary" _hover={{ background: "white", color: "$danger9" }}>
          {link.text}
        </Button>
      </HopeLink>

  )}
</For>

produces:

When hovering over the "home" button.
I have also tested this with the  at the root of my app with the same effects.

I believe this line to be the culprit, as the selector used, which ultimately overrides the basic &:hover selector is &:not(:disabled):hover.
When changing the code to be the following:
const links = [
  { text: 'Home', to: '/' },
  { text: 'About', to: '/about' },
  { text: 'Login', to: '/login' }
];

[REDACTED]

<For each={links} fallback={
Loading...
}>
  {(link) => (

      <HopeLink href={link.to} fontSize="10px" class="p-1 m-1">
        <Button colorScheme="primary"
          css={{
            "&:not(:disabled):hover": {
              backgroundColor: "transparent",
              color: "black"
            }
          }}
        >
          {link.text}
        </Button>
      </HopeLink>

  )}
</For>

We get:

when hovering.

Hi, yes the selector &:not(:disabled):hover have higher specificity. The problem is if I set it to &:hover only, hover styles will be applied even if the button is disabled.

Maybe I can do something like this, i will check it out
backgroundColor: config.bgColor,

"&:hover": {
  backgroundColor: config.bgColorHover,
},

"&:hover:disabled": {
  backgroundColor: config.bgDisabled
}

fixed in v0.4.4. available on the next tag.

Amazing job! Thank you!


Server still not responding after upgrade

Hi!
Sorry for creating a new issue, but I don't know if you have notifications on new comments on old issues.
Do you have any update of what has happened and when it will work again?
Best regards,
Anders

Now server is responding with timezone definition on port 2342 so it seems ok!
Thanks!

**** Still a TIMEOUT now:
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 2 - STA_START
ezTime debug level set to INFO
Waiting for WiFi ... Querying pool.ntp.org ... ERROR: No network
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 4 - STA_CONNECTED
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 7 - STA_GOT_IP
[D][WiFiGeneric.cpp:419] _eventCallback(): STA IP: <IP_ADDRESS>, MASK: <IP_ADDRESS>, GW: <IP_ADDRESS>
Querying pool.ntp.org ... success (round trip 52 ms)
Received time: Friday, 17-Sep-21 10:53:20.624 UTC
Time is in sync
UTC:             Friday, 17-Sep-2021 10:53:20 UTC
Timezone lookup for: Switzerland/Zurich ... ERROR: Timeout
Switzerland:     Friday, 17-Sep-2021 10:53:22 UTC
*** When I change Remote port to 4243, the server does respond but timezone is NOT set.

It also works for my apps again, thanks!


texvc support

Merge the texvc specific support back to standard MathJax
This will allow to merge new version of MathJax to MediaWiki easier.

Closing this as per F2F discussion ages ago. @physikerwelt let's catch up on this some time?


Tests/llvm config

note: no squash needed on that one, it also fies a few issues

Excellent, thanks!


[ Relay ][ Frontend ][ Tensorflow ]add op add_n to relay/frontend/tensorflow.py

Thanks for contributing to TVM!   Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

LGTM

Thanks @soiferj @KimBioInfoStudio @yongwww !


Remove unneeded @Deploy's

FW-1369 Remove unneeded unit testing @Deploys
Does not impact test logic
Reduces runtime of mvn clean test from 2:16 to 1:06 on my development machine.

@plasticviking can you please fix merge conflicts and then changes lgtm!

merge conflicts

Hi @davidlacho, conflicts resolved


Allocators owns should take const void[]

owns should take a const void[] argument as it should also work with const and immutable memory chunks.
Added more unittests in order to ensure that allocators inherit from their parents.

Per the discussion with @edi33416 let's close this as it has low impact.

We can return to it later.


Add NavigationKey in GoRouter

Steps to Reproduce
I am using the go_router in my app and also the get package. The problem is that since I implemented the Go-Router I can no longer show the Get.snackbar. It is failing and throwing a null-error.
For the GoRouter to work I had to add the .router for my GetMaterialApp. And then navigationKey was no longer available:
This is my GetMaterialApp:
  child: GetMaterialApp.router(
    title: 'Wishlists',

    // navigationKey: Get.key,
    // .router does not have a navigationKey so showSnackBar fails because the key is null

    routeInformationProvider: router.routeInformationProvider,
    routeInformationParser: router.routeInformationParser,
    routerDelegate: router.routerDelegate,
    translationsKeys: AppTranslation.translationsKeys,
    locale: locale,
    fallbackLocale: fallbackLocale,

  ),

This is the error:

Error: Unexpected null value.
at Object.throw_ [as throw] (http://localhost:51492/dart_sdk.js:5080:11)
at Object.nullCheck (http://localhost:51492/dart_sdk.js:5399:30)
at [_configureOverlay] (http://localhost:51492/packages/get/get_navigation/src/snackbar/snackbar_controller.dart.lib.js:2970:53)
at [_show] (http://localhost:51492/packages/get/get_navigation/src/snackbar/snackbar_controller.dart.lib.js:3113:30)

I feel like it has something to do with the key inside the GetMaterialApp, but I have no idea how to solve this.
Is this a known bug? What am I missing here?

being able to customize go_router is  generally good, so I'm supportive of the idea
especially when it seems to be an easy fix
  GoRouter({
    /// ...
+  GlobalKey<NavigatorState>? key,
  }) {
  /// ...
    _routerDelegate = GoRouterDelegate(
+     key: key,
    );
    /// ...
  }

  GoRouterDelegate({
     /// ...
+  GlobalKey<NavigatorState>? key,
  }) : builder = RouteBuilder(
        /// ...
-        );
+       ),
+     _key = key ??  GlobalKey<NavigatorState>();

   /// ...
+ final GlobalKey<NavigatorState> _key;
-  final GlobalKey<NavigatorState> _key = GlobalKey<NavigatorState>();

Nonetheless this is yet another example of why devs should steer clear of getX!
something as easy as dysplay a snackbar
ScaffoldMessenger.of(context).showSnackBar(snackBar);

being "vendor locked" is just bad (together with many other reasons, including fudged benchmarks)

Hello @DieGlueckswurst. Thanks for filing this request. Looks like a valid one. Labelling as such.

The pr https://github.com/flutter/packages/pull/2453#discussion_r949604346 adds the navigator key to gorouter

Yes please we need this to be able to navigate with Firebase push notification click listener , with GoRouter we are not able to get the context to navigate

any updates on this? seems like a quick fix. would help me a lot :D

flutter/packages#2453 (comment)

you can find the updates here https://github.com/flutter/packages/pull/2453


Why doesn't 'traverseWithKey' of Data.HashMap.Strict ensure the values of the result in WHNF?

I find a strict version of traverseWithKey is missing from Data.HashMap.Strict.Base. Instead I only see following sentence:

-- TODO: Should we add a strict traverseWithKey?

Is there any particular reason not to implement it?
I recent chased a memory leak in my program, and eventually find out that the values of my result HashMap after calling traverseWithKey are not in WHNF. This somewhat breaks the meaning of the whole module.

No, I don't think there's a good reason. Let's fix that, shall we?

sure. I will make a PR :)

I find it actually difficult to defined a completely strict version. there are various issues.  The traverseWithKey only requires Functor, but there seems not similar thing to <$!> with Functor.
There is also a issue with <$!> with Monad here. https://stackoverflow.com/questions/9423622/strict-fmap-using-only-functor-not-monad

I'm not sure what you're saying. traverseWithKey deals with Applicative, not Functor. You're not going to want to implement the strict version using the lazy version; you'll want to copy the code and make the necessary changes. But remember: you don't need to force elements before producing an action (and you can't). You need to force elements before producing a result. Specification:
Strict.traverseWithKey f = fmap (Strict.map id) . Lazy.traverseWithKey f

i am sorry that i wasn't able to make a response earlier. I was thinking about Applicative, but typed Functor instead. I am just concerning about is there a universal way to force the value there.
I made following test case. I tried three different monad, IO, Identity, and a strict Identity. I find OOM is solved in case 1, case 3, but no case 2, with your fix.  The same occurs in my fix here https://github.com/pierric/unordered-containers/commit/b975384ee6c636248a25de8e2cb9c4b2735ab0b8
{-# LANGUAGE BangPatterns #-}
module Main where

import Control.Monad.Identity
import qualified Data.HashMap.Strict as M
import qualified Data.Vector.Storable as V

mkV :: Int -> IO (V.Vector Int)
mkV = V.replicateM (128 * 1024) . return

main :: IO ()
main = do
    vs <- mapM mkV [1..100]
    let m0 = M.fromList (map (\v -> (v V.! 0, v)) vs)
    --
    -- case 1. IO monad
    go 100000000 m0
    --
    -- case 2. Identity monad
    -- return $! runIdentity $ go 100000000 m0
    --
    -- case 3.  There's a bit of an inconsistency. Whereas Lazy.mapWithKey forces the structure of the tree, Lazy.traverseWithKey does not. For the strict traversal function to work right with Identity, we need to be stricter. Please try the version I just pushed; I think I probably fixed it.

I confirm that the OOM is gone with Identity monad. Great! Thank you very much  @treeowl

Thanks for helping me pin this down, @pierric!


Zero-byte mp3 files

See towards the end of: https://github.com/spotDL/spotify-downloader/issues/931
I tried re-installing ffmpeg but no improvement using v3.1.4, v3.1.3 or v3.1.2.
But after installing spotdl-3.2.0rc0 from next-rel-dev.zip, the conversion from mp4 to mp3 is working again.
(OK, I now get:
FileExistsError: [Errno 17] File exists: './Temp'
but I've knocked up a crude bash script to auto-resume on error and the job completes)

See towards the end of: #931
I tried re-installing ffmpeg but no improvement using v3.1.4, v3.1.3 or v3.1.2.
But after installing spotdl-3.2.0rc0 from next-rel-dev.zip, the conversion from mp4 to mp3 is working again.
(OK, I now get:
FileExistsError: [Errno 17] File exists: './Temp'
but I've knocked up a crude bash script to auto-resume on error and the job completes)

Could you explain what the error was?

``https://open.spotify.com/album/7leAx67b8jDAXvIBy5IkHq
Fetching Album...
80%|████████████████████████████████████████████████▊            |ETA: 04:19, 1.44min/songmultiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 47, in starmapstar
return list(itertools.starmap(args[0], args[1]))
File "/usr/local/lib/python3.6/site-packages/spotdl/download/downloader.py", line 105, in download_song
on_progress_callback = displayManager.pytube_progress_hook
File "/usr/local/lib/python3.6/site-packages/pytube/main.py", line 103, in init
self.descramble()
File "/usr/local/lib/python3.6/site-packages/spotdl/patches/pyTube.py", line 149, in descramble
self.player_config_args = get_ytplayer_config(self.watch_html)["args"]
File "/usr/local/lib/python3.6/site-packages/spotdl/patches/pyTube.py", line 241, in get_ytplayer_config
raise RegexMatchError(
NameError: name 'RegexMatchError' is not defined
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/spotdl", line 11, in
sys.exit(console_entry_point())
File "/usr/local/lib/python3.6/site-packages/spotdl/main.py", line 94, in console_entry_point
downloader.download_multiple_songs(songObjList)
File "/usr/local/lib/python3.6/site-packages/spotdl/download/downloader.py", line 306, in download_multiple_songs
for song in songObjList
File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 274, in starmap
return self._map_async(func, iterable, starmapstar, chunksize).get()
File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
NameError: name 'RegexMatchError' is not defined

I still didn't get it, how is that related to fileAlreadyExists?

Oops, I'd not noticed that I'm now getting a different error.
I've just run the same URL 10 times. Twice the album was downloaded error free. The other times I got 'RegexMatchError'.

Actually,  wait for 2 -  days, were doing away with multiprocessing  you can check the related PR under the PR tab


Multiple Keyword Error

Hi All,
We have 2 repos with same resource file name in the git. While running testcases from one repo we are hitting multiple keywords error. Could any one give the solution for it.
Multiple keywords with name 'SSH-Set-Timeout' found. Give the full name of the keyword you want to use:
SSH-Wrapper.SSH-Set-Timeout
SSH-Wrapper.SSH-Set-Timeout
Thanks
Krishna

Do I get it right that you have two resource files with same name and same keywords in use? That results to an unresolvable conflict. You need to either rename either files or keywords.


Placeholder doesn't change when placeholder or customPlaceholder change

Hey,
When I change the value of the props placeholder or customPlaceholder, it doesn't change the state of IntlTelInputApp, and therefore doesn't rerender. I would need to dynamically change either one of them (e.g. hiding placeholder and displaying error when there is one).
Would you mind if I filed a PR for this change?
Thanks for this project! :+1:

Definitely! 👍 Thanks for your help!


Issue finding Inhabited instances in partial definition

Prerequisites

[X] Put an X between the brackets on this line if you have done all of the following:

Checked that your issue isn't already filed.
Reduced the issue to a self-contained, reproducible test case.

Description
The following definition fails to compile because Lean can't infer that m (Unit × β) is inhabited.
variable {α β ε : Type _} (m : Type _ → Type _) [Monad m] [MonadExcept ε m]

partial def test (q : m β) : m (Unit × β) :=
  let rec loop (p : Unit) : m (Unit × β) :=
    try
      return (p, ← q)
    catch _ => do
      loop p
  loop ()

The following workaround does compile.
variable {α β ε : Type _} (m : Type _ → Type _) [Monad m] [MonadExcept ε m]

partial def test' (q : m β) : m (Unit × β) :=
  let inst := Inhabited.mk do return ((), (← q))
  let rec loop (p : Unit) : m (Unit × β) :=
    try
      return (p, ← q)
    catch _ => do
      let _ := inst
      loop p
  loop ()

Both lets are necessary to compile. The second let is especially confusing, I think just fixing that would already be a significant improvement.
Versions
On MacOs with Lean (version 4.0.0-nightly-2023-04-20, commit f9da1d8b55ca, Release)

After a discussion on Zulip I think that the first let instance is not that problematic. I also understand that the second let instance is because the let rec doesn't automatically capture all local instances by design. That, I think, is problematic in this context. Adding the first let instance does not change the error message, which suggests that a different inhabited instance is necessary. When I first encountered this issue, I spent a long time checking that I got the right types and adding different instances to no avail. It's only by accident that I came across the solution.
Perhaps this can be remedied with a better error message: can the partial definition checker point out where the inhabited instance is missing? Or perhaps there is a way to force Lean to consider all local instances for the purpose of checking partial definitions?

I think we should perhaps consider special-casing Inhabited and Nonempty instances when compiling partial functions, propagating them to lets.
Here's another program with this issue:
partial def outer' [Nonempty α] (i : Unit) : α := inner' i
where
  inner' : Unit → α
    | () => inner' ()

failed to compile partial definition 'outer'.inner'', failed to show that type is inhabited and non empty

It can be fixed by causing a spurious use of the instance:
partial def outer'' [Nonempty α] (i : Unit) : α := inner'' i
where
  inner'' : Unit → α
    | () => have : Nonempty α := inferInstance; inner'' ()

but this is a quite obscure transformation.


Fix copypasta in JacksonGenerator causing maps to be lists

Also parse both in a safer way that makes it more clear where errors lie.
Contributing to Twilio

All third party contributors acknowledge that any contributions they provide will be made under the same open source license that the open source project is provided under.

[x] I acknowledge that all my contributions will be made under the project's license.

right?  i feel super dumb now.


First Try: timeout

On Raspberry Pi 2, using Raspbian and Python 3.5.3, I installed hyper and tried to run your simple example which failed on the conn.request('GET','/get'):
from hyper import HTTPConnection

conn = HTTPConnection('http2bin.org:443')
conn.request('GET', '/get')
resp = conn.get_response()

print(resp.read())

Error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pi/.local/lib/python3.5/site-packages/hyper/common/connection.py", line 103, in request
    method=method, url=url, body=body, headers=headers
  File "/home/pi/.local/lib/python3.5/site-packages/hyper/http11/connection.py", line 169, in request
    self.connect()
  File "/home/pi/.local/lib/python3.5/site-packages/hyper/http11/connection.py", line 119, in connect
    sock = socket.create_connection((host, port), 5)
  File "/usr/lib/python3.5/socket.py", line 712, in create_connection
    raise err
  File "/usr/lib/python3.5/socket.py", line 703, in create_connection
    sock.connect(sa)
socket.timeout: timed out

I can't ping http2bin.org, but the host does resolve to an IP address.

http2bin.org is down for a long time now. Try with another one, like google.com
from hyper import HTTPConnection

conn = HTTPConnection('google.com:443', secure=True)
conn.request('GET', '/')
resp = conn.get_response()

print(resp.read())

thanks...change your docs, please
On 9/3/17 11:41 PM, Kostya Esmukov wrote:

http2bin.org is down for a long time now. Try with another one, like
google.com
|from hyper import HTTPConnection conn =
HTTPConnection('google.com:443', secure=True) conn.request('GET', '/')
resp = conn.get_response() print(resp.read()) |
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Lukasa/hyper/issues/349#issuecomment-326859493,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABFf1EBnK4zJ2y3zASNqIKFsmPJxuckJks5se3F3gaJpZM4PLamH.


error crashed code 5

Does this issue occur when all extensions are disabled?: No

VS Code Version:

OS Version:

Steps to Reproduce: *.dmp file is here
vscode-crash.zip

Does this issue occur when all extensions are disabled?: No

If this not reproducible with all extensions are disabled that it's likely caused by an extension.
From command palette (F1) run >Help: Start Extension Bisect and follow steps.
See details: https://code.visualstudio.com/blogs/2021/02/16/extension-bisect

I cannot reproduce errors when some extensions are disabled


? and ! statements spread across multiple lines

I'm trying to create a MAUI XAML block of code when clicking on a Frame.
So for example I would want the  tag to be hidden if the node.type is NOT a frame.
I could put {{?node.type=frame}} before each line, but I also want it to hide all the code inside the code block.
Is there a way span the {{?node.type=frame}} statement across muliple lines to indicate when it starts and finishes?
Current approach:
{{?node.type=frame}}
{{?node.type=frame}}{{?css.height}}HeightRequest="{{css.height}}"
{{?node.type=frame}}
Ideal approach (just one of many examples):
{ {{?node.type=frame}}
{{?css.height}}HeightRequest="{{css.height}}"
 }
{{?node.type=frame}}
{{?css.height}}HeightRequest="{{css.height}}"
 {{?node.type=frame}}

See #14 For how I think you should handle this. Essentially, you should hardcode your frame template into your own version of this codebase if youre creating global templates like this and conditionally rendering on node.type.
Point taken though on the idea of blocks for the conditionals. I want to see how long I can go before implementing something that spans lines. One strength of the language is that each line is rendered in complete isolation which makes it a lot simpler to write and process.


names and spaces

if the file has a space at the end of the filename aka "test123 .html"  the page displays almost nothing
i admit it's bad form to have a space but it happened by mistake and i found this issue.

ok. now it's loading correctly. grrrr


sometimes factorio_logistic_network_items does not include science packs

I'm not sure why, but sometimes the factorio_logistic_network_items has plenty of stuff in it, but science packs stop being written out to game.prom.
I don't know how to reproduce, I don't know how to trigger it, but I have millions of science packs in logistics chests, and none of them show up in game.prom.  They were working, then they weren't.   ¯\_(ツ)_/¯

ah, I think running factorio headless includes the science pack data, and running normally doesn't.

Hello,
I Will try to look at it this week-end

I should add that I'm using Windows for both the GUI client and the headless server.

This may be a bug in Factorio, actually.  Switching back to full game client from standalone server shows the same save file now correctly reporting the science packs in the logistic network items dataset (or whatever the term is).
I expect that these will stop being emitted by your mod at some point, and I will again see "no data" on those gauges.  I don't know what the trigger was yesterday, so I don't know when I'll hit it today, but if I can associate it with something I'm doing, I'll let you know.


fix #167 use latest version of code-server

Hello!
This PR fixes https://github.com/cdr/sshcode/issues/167
There was a bit of a jungle with https://github.com/cdr/sshcode/pull/178 and https://github.com/cdr/sshcode/pull/181, so let me explain what happened.
Big thanks to @Merith-TK I realized, that the different PR order would make more sense. However we misunderstood each other a bit with the PR openings. Anyway - everything is clear now, so I am making the PRs in the right order. My apologies for the PR jungling :)

@nhooyr would you mind taking a look?

there is a button on the side that requests a reveiw. or down below here, i forgot

I do not see any button. There is a section on the right side, but I can't use it. Maybe you can because you have collaborator status?

@gyzerok Thanks for contributing, sorry for the late response.
I'll take a look at this soon.

It seems that after 3.3.0 update it got broken for my environment at least. So I invite you to help me figure out the most proper solution before we merge it. I am willing to incorporate any suggestions to make it the best experience for others.
Zero idea what it's about, but maybe you can help - Error: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.28' not found`.

See https://github.com/cdr/code-server/issues/1656

@gyzerok Can you hold off on further work here?
We're discussing deprecating sshcode in favour of a bash script that we maintain as part of code-server with most of the same functionality. I'll keep you posted on it. Don't want you to waste time on something that might not be merged.

@nhooyr
I would recommend making it an shell script instead of a bash script. As not all remote systems use bash, nor do all clients have bash installed.

@Merith-TK Yup, sorry I meant a shell script.

@nhooyr right now no further work is happening. I just have added ability to pass additional flags to the code-server and were planning to make it a separate PR.
Thanks for notifying me. Will be awaiting for you decision :)

See https://github.com/cdr/sshcode/issues/185

Please see #185!


Location is shifted on the map

Hello I am using Xamarin.Forms.BaiduMaps component in my mobile app. By deafult user can see his location on the map. This function is working well in several countries but in China position is shifted.
This is how map initialized:
IMapManager mapManager = DependencyService.Get();
mapManager.CoordinateType = CoordType.GCJ02;
Can you help me with this trouble?

Maybe your coordinate is WGS84. And you are using GCJ02 in your code, so you must translate WGS84 to GCJ02 first.
Then the 'shift' will disappear.


⚠ AlphaRatio has degraded performance

In 18c9893, AlphaRatio ($AR) experienced degraded performance:

HTTP code: 200
Response time: 1124 ms

Resolved: AlphaRatio performance has improved in 50f9dda after 415 days, 12 hours, 9 minutes.


[WFLY-11222] Remove the check for the -secmgr in the JAVA_OPTS. This …

…is not a valid JVM option and should never be in the JAVA_OPTS.
https://issues.jboss.org/browse/WFLY-11222

@jamezp Thank you, btw I don't see any regression caused by this PR ...


added config for IPP machine with Debian oldoldstable

I added the config for my IPP machine with Debian oldoldstable.

There is a merge conflict in the setup_cmake
<<<<<<< debian_oldoldstable
    cmake -DCMAKE_BUILD_TYPE:String=$BUILD_TYPE             \
          -DUSE_SSH=ON                                      \
          -DBUILD_PARVMEC=ON                                \
          -DCMAKE_Fortran_FLAGS="-fallow-argument-mismatch" \
          ../

elif [ $MACHINE_ID == "sv-e5-da-1" ]
then
    # Config for J. Schilling's IPP machine.
    # This is Debian oldoldstable (9.13).
    # -fpermissive is needed for xwout_diff.
    # This needs the debian_oldoldstable branch of LIBSTELL,
    # where libblacsF77init is linked explicitly.
    cmake -DCMAKE_BUILD_TYPE:String=$BUILD_TYPE             \
          -DUSE_SSH=ON                                      \
          -DBUILD_PARVMEC=ON                                \
          -DCMAKE_CXX_FLAGS="-fpermissive"                  \
=======
    cmake -DCMAKE_BUILD_TYPE:String=$BUILD_TYPE           \
          -DUSE_SSH=ON                                    \
          -DBUILD_PARVMEC=ON                              \
          -DCMAKE_Fortran_FLAGS="-fallow-argument-mismatch" \
>>>>>>> master

@cianciosa :
Hi Mark,
I fixed the merge conflict (sorry for that). Could you please have a second look?
Best,
Jonathan


Permission denied when running as a docker container

Docker log error
Failed to scrape socket: dial unix /run/php/php7.2-fpm.sock: connect: permission denied
Path is correct but I cannot read data

Did you mount the sock into the docker container?
Example from readme
SOCK="/run/php/php7.2-fpm.sock"; \
docker run -d -p 9253:9253 -v $SOCK:$SOCK  \
lusotycoon/phpfpm-exporter \
--phpfpm.socket-paths=$SOCK

I ran exactly the example from the readme, sorry I don't have any other debug info

is php-fpm serving the status page?
please try to download the binary and run it directly against the php-fpm host instead of using another container

I can get to the status page via browser, I will try the binary directly. Thanks

OK, the binary doesn't throw the permission error but it returns the same info in /metric with or without the error
HELP go_gc_duration_seconds A summary of the GC invocation durations.
TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0
go_gc_duration_seconds{quantile="0.25"} 0
go_gc_duration_seconds{quantile="0.5"} 0
go_gc_duration_seconds{quantile="0.75"} 0
go_gc_duration_seconds{quantile="1"} 0
go_gc_duration_seconds_sum 0
go_gc_duration_seconds_count 0
.....

I am getting permissions issues when running https://github.com/hipages/php-fpm_exporter on <IP_ADDRESS>:9000 so safe to say I am the problem not your code.
Thanks for your help.


feat: isInfraUp

Ensures that terraform apply is only run if terraform plan shows changes to the infrastructure
fixes #265

This LGTM.i can't merge anything ATM because my name is typoed in the codeowners


Add firewall-traversing iptables rules even if firewalld isn't running

People may have non-firewalld-based firewall rules. (In particular, our ansible install sets up such rules.)
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1273659

LGTM


Modified hibernate-validator code and dependency to avoid exception in consumers.

Having this module as dependency in callers sometimes results in javax.validation.ValidationException: HV000183
Modified the code and the dependencies in an attempt to avoid this

Closing in favour of https://github.com/hypertrace/document-store/pull/65


Can not update settings.kubernetes.pod-pids-limit by userData in Bottlerocket

Description
What problem are you trying to solve?
I want to update the pod-pids-limit by useData in Bottlerocket, but it does to take effect.
amiFamily: Bottlerocket
userData:  |
[settings.kubernetes]
"pod-pids-limit" = 9999
cat /etc/kubernetes/kubelet/config
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
#snippet
podPidsLimit: 1048576
#snippet
How important is this feature to you?

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Karpenter currently only supports a subset of the Bottlerocket configuration, there's another open issue, #6259, which proposes either automatically generating a new schema for each Karpenter release, or only only defining the values Karpenter needs to reason about and passing the remaining values through untouched. We would accept a PR to add support this specific field in the interim though.


Create User Authentication/Accounts

This seems like it could be a good starting point: https://scotch.io/tutorials/build-and-understand-a-simple-nodejs-website-with-user-authentication

#46

https://github.com/ucsb-cs48-w19/5pm-ucsb-rideshare/tree/oauth