Investigate supporting defaults in AuthPolicy

Use Case
As a gateway administrator I want to lock down access to any host attaching to this gateway. By default I want to apply a default DENY ALL policy at the gateway level.
When a policy is created targeting a HTTPRoute, I want that policy to then take precedence over my default policy.

[ ] Validate the existing behaviour supports this use case
[ ] Move authschema to default
[ ] Create a Demo showing two services and default policy. Show how access is only allowed when a policy has been defined

Based on initial thoughts on how to go about solving not only defaults but also overrides, I think it makes sense for the kuadrant controller to create an internal index of the hierarchy of resources present on the cluster. This would be similar to the tree representation that authorino uses for its cache but customized for controller use cases.
We need this because of the relational nature of AuthPolicy, and defaults/overrides require taking all the relations in one place to decide what behavior needs to be reconciled and configured within authorino.
let's go over a non-canonical examples:
target: gateway
default:
  hosts: ["*.pets.com"]
  authscheme:
    ac-gw-deny-all
override:
  hosts: ["cdn.pets.com"]
  authScheme:
    ac-gw-allow-all
---
target: route
default:
  hosts: ["dogs.pets.com"]
  authScheme:
    ac-rt-custom-default
override:
  {NOT_ALLOWED_YET} // Not sure why override is required at the route-level

This would look like following in the kuadrant controller.
                                   *
                                   │
                                   ▼
                                  com
                                   │
                                   │
                                   ▼
         ┌────────────────────────pets─────────────────────────┐
         │                         │                           │
         │                         │                           │
         ▼                         ▼                           ▼
         *                        cdn                         dogs
┌──────────────────┐     ┌───────────────────┐     ┌─────────────────────────┐
│Gateway:          │     │Gateway:           │     │Route:                   │
│  Default:        │     │  Override:        │     │  Default:               │
│    ac-gw-deny-all│     │    ac-gw-allow-all│     │    ac-route-cust-default│
└──────────────────┘     └───────────────────┘     └─────────────────────────┘

Each KAP would add information to the index tree and after each change to the tree, reconciliation would happen to a single resource from each node based on predefined rules.
Q: What happens if there are two route-level KAPs with same host?
A: In case of conflict, timestamp of creation will be used and status of rejected resource will be updated. During rebuilding of index same information will be used.
Q: What are the rules of creation for AuthConfigs?
A: Following algorithm is still in progress as we come across more cases.
If children nodes consist of a `start node` (`*`):
  reconcile(star node); // rest should be ignored at that level;
  delete(other nodes);
else: // there is no overlap
  reconcile(each node);

reconcile(node):
  if node contains gateway level override then reconcile only that;
  else if it contains route and gateway level defaults; reconcile only route-level defaults;
  else reconcile default which is present;

Just on

override:
{NOT_ALLOWED_YET} // Not sure why override is required at the route-level

We could certainly add a webhook that would reject an override at the route level with a useful message such as: overrides when targeting HTTPRoutes are not allowed, please set a default
It may not be required. One thought that comes to mind is if we were to allow setting a policy at the service level for example rather than just the route, but I don't see a good usecase for that right now.

I think it makes sense to build up a representation of the relationships between authpolicies. This will avoid an expensive list all auth policies and end up building a temporary structure for each change. I think the actual rules need some refinment

@rahulanand16nov, a couple extra observations from my side as well.

In the diagram, I believe you meant for the root to be a dot ('.'), not a star ('*').

Q: What happens if there are two route-level KAPs with the same host?
A: In case of conflict, the timestamp of creation will be used and the status of rejected resource will be updated. During the rebuilding of the index, the same information will be used.

Which one prevails, the newest or the oldest resource?
Do default or override have any implication on the above?
Can wildcards be used in route-level policies?

Q: What are the rules of creation for AuthConfigs?
A: The following algorithm is still in progress as we come across more cases.
If children nodes consist of a `start node` (`*`):
[...]

Do you mean star node?

Q: What would happen to the example shown above?
A: All of them would be created.

Do you mean as 3 separate AuthConfigs? (I imagine so; just double checking.)
Moreover, we're considering disabling Authorino's host name collision prevention for Kuadrant (https://github.com/Kuadrant/authorino/issues/329). If that is the case, then all fine. Otherwise, in the case of 3 separate AuthConfigs for the example above, let's have in mind that *.pets.com would have to be the last AuthConfig to be applied, but only if host name collision prevention and the AuthConfigs are from different namespaces – like this will never be the case for Kuadrant.

It also would be interesting to see an example of the AuthConfig operations induced by each step into building the example above (IWO, what 'reconcile' means exactly in the algorithm), and them permutations of the steps (i.e. same steps in different order).

In the diagram, I believe you meant for the root to be a dot ('.'), not a star ('*').

Yep, it's supposed to be dot('.');

Q: What happens if there are two route-level KAPs with the same host?
A: In case of conflict, the timestamp of creation will be used and the status of rejected resource will be updated. During the rebuilding of the index, the same information will be used.

Which one prevails, the newest or the oldest resource? Do default or override have any implication on the above? Can wildcards be used in route-level policies?

the oldest resource would prevail; route-level policies can have wildcards; default or override will have no implications; Note: These are the ideal expectations but as we move forward and find more problems, might need to pivot on those expectations.

Q: What are the rules of creation for AuthConfigs?
A: The following algorithm is still in progress as we come across more cases.
If children nodes consist of a `start node` (`*`):
[...]

Do you mean star node?

Yep

Q: What would happen to the example shown above?
A: All of them would be created.

Do you mean as 3 separate AuthConfigs? (I imagine so; just double checking.)

Ideally, yes.

Moreover, we're considering disabling Authorino's host name collision prevention for Kuadrant (Kuadrant/authorino#329). If that is the case, then all fine. Otherwise, in the case of 3 separate AuthConfigs for the example above, let's have in mind that *.pets.com would have to be the last AuthConfig applied, but, again, only if host name collision prevention is off and the AuthConfigs are from different namespaces – likely this will never be the case for Kuadrant.

We can control the order of creation for each authconfig depending on how we traverse the tree so I hope it shouldn't be an issue.

It also would be interesting to see an example of the AuthConfig operations induced by each step into building the example above (IWO, what 'reconcile' means exactly in the algorithm), and them permutations of the steps (i.e. same steps in different order).

:+1:

I just realized the algorithm doesn't even work for the example I wrote above :facepalm: but second iteration does.
if children nodes have a star node (`*`):
  if it has a `gateway-override`:
    just reconcile `gateway-override`;
    delete other nodes;
  elif it has `route-default`:
    reconcile `route-default`;
    reconcile(other nodes and delete `gateway-default` in them);
  else
    reconcile `gateway-default`;
    reconcile(other nodes);
else
  for each node: reconcile (node);

reconcile(node):
  reconcile in order of `gateway-override` > `route-default` > `gateway-default`;

cc @maleck13 @guicassolato

@rahulanand16nov, here's my take on the algorithm to reconcile an AuthoPolicy for a particular host.
I took into account 8 types of AuthPolicies (i.e. 8 different cases), based on the 3 aspects (variables) of the AuthPolicy.
Variables:

target → Gateway | HTTPRoute
host → <wildcard> | <literal>
scheme → default | override

Cases (2 targets ⅹ 2 hosts x 2 schemes = 8):

GWD: Gateway <wildcard> default
GWO: Gateway <wildcard> override
GLD: Gateway <literal> default
GLO: Gateway <literal> override
RWD: HTTPRoute <wildcard> default
RWO: HTTPRoute <wildcard> override
RLD: HTTPRoute <literal> default
RLO: HTTPRoute <literal> override

Note: I find useful to have RLD and RLO both supported, e.g. for the use case where admins can define RLO policies, while devs are restricted to RLD.
Precedence:
GWD < RWD < GLD < RLD < GWO < RWO < GLO < RLO
Description of the algorithm:
reconcile(kap, host) checks if another authpolicy exists, of any type that takes precedence over the resource being reconciled, targeting the same host (wildcard overlaps included); if thats’s the case, it rejects the resource for this particular host. Otherwise, it updates the status of any existing resource that would be superseded, and deletes/creates/updates the authconfigs accordingly
Pseudocode:
PolicyType = enum{GWD, RWD, GLD, RLD, GWO, RWO, GLO, RLO}

func reconcile(kap AuthPolicy, host string) error
  var supersededTypes []PolicyType

  switch kap.PolicyType()
    case GWD:
      if existOthers(kap, host, []PolicyType{RWD, GWO, RWO})
        return updateAuthPolicyStatus("%host superseded", kap)
      supersededTypes = []PolicyType{GWD}
    case RWD:
      if existOthers(kap, host, []PolicyType{GWO, RWO})
        return updateAuthPolicyStatus("%host superseded", kap)
      supersededTypes = []PolicyType{GWD, RWD}
    case GWO:
      if existOthers(kap, host, []PolicyType{RWO})
        return updateAuthPolicyStatus("%host superseded", kap)
      supersededTypes = []PolicyType{GLD, RLD, GWO}
    case RWO:
      supersededTypes = []PolicyType{GWD, RWD, GLD, RLD, GWO, RWO}
    case GLD:
      if existOthers(kap, host, []PolicyType{RLD, GWO, RWO, GLO, RLO})
        return updateAuthPolicyStatus("%host superseded", kap)
      supersededTypes = []PolicyType{GLD}
    case RLD:
      if existOthers(kap, host, []PolicyType{GWO, RWO, GLO, RLO})
        return updateAuthPolicyStatus("%host superseded", kap)
      supersededTypes = []PolicyType{GLD, RLD}
    case GLO:
      if existOthers(kap, host, []PolicyType{RLO})
        return updateAuthPolicyStatus("%host superseded", kap)
      supersededTypes = []PolicyType{GLD, RLD, GLO}
    case RLO:
      supersededTypes = []PolicyType{GLD, RLD, GLO, RLO}

    # somehow as an atomic transaction
    supersededAuthPolicies := findOthers(kap, host, supersededTypes)
    updateAuthPolicyStatus("%host superseded", supersededAuthPolicies...)
    deleteAuthConfigs(hosts(supersededAuthPolicies, host) - host)
    createOrUpdateAuthConfig(host, kap.AuthScheme)

func findOthers(kap AuthPolicy, host string, policyTypes []PolicyType) []AuthPolicy
  return []AuthPolicy{ other authpolicies targeting the same host (wildcard overlaps included) and whose type is in 'policyTypes' }

func existOthers(kap AuthPolicy, host string, policyTypes []PolicyType) bool
  return len(findOthers(kap, host, policyTypes)) > 0

Updating my own suggestion here, so it's consistent with the Gateway API (GW-API) spec.
I said before:

Precedence:
GWD < RWD < GLD < RLD < GWO < RWO < GLO < RLO

GW-API establishes that the hierarchy (which I called "precedence") for the different types of resources should be exactly the reverse between defaults and overrides.

Given our 4 types of resources:

GW: Gateway <wildcard>
GL: Gateway <literal>
RW: HTTPRoute <wildcard>
RL: HTTPRoute <literal>

The hierarchy in our case should actually be:
GW[D] < RW[D] < GL[D] < RL[D] < RL[O] < GL[O] < RW[O] < GW[O]
(Now noting default (D) and override (B) within square brackets.)
Rewriting the above with inverted inequality symbol (to match how it's represented in the GW-API docs):
GW[O] > RW[O] > GL[O] > RL[O] > RL[D] > GL[D] > RW[D] > GW[D]
Finally, the updated pseudocode for the hierarchy above:

Warning! I haven't tested this.

PolicyType = enum{GWO, RWO, GLO, RLO, RLD, GLD, RWD, GWD}

// reconcile(kap, host) reconciles a policy for a particular host.
// It looks for other policies targeting the same host (wildcard overlaps included), that are higher in the hierarchy and
// that according to the rules would be favoured to protect the host. If that is the case, it rejects the resource; otherwise,
// it updates the status of any existing policy that would be superseded by the resource regarding the the host, and
// deletes/creates/updates the authconfigs accordingly.
//
// Hierarchy:
// GWO > RWO > GLO > RLO > RLD > GLD > RWD > GWD
//
// General rules:
// 1. **Older policy rule:** Between policies of the same type, targeting the same host, the older policy (created first) wins, i.e. older (existing) supersedes newer (challenging).
// 2. **Strict subset rule:** Literal hosts (L) never supersede wildcards (W), because L ⊂ W; they can leave side by side (different AuthConfigs).
// 3. **Default rule:** Defaults (D) only supersede other defaults (lower in the hierarchy).
// 4. **Override rule:** Overrides (O) supersede everything that is lower in the hierarchy, as long as it doesn't violate any of the other rules.
//
// Combined rules:
// - xWO rule:
//   - supersedes all types that are lower in the hierarchy (rule 4)
//   - is superseded by older self and all xWO that are higher in the hierarchy (rules 1, 2, 3 and 4)
// - xLO rule:
//   - supersedes all xLD and xLO that are lower in the hierarchy (rules 2 and 4)
//   - is superseded by older self and all xWO and xLO that are higher in the hierarchy (rules 1, 3 and 4)
// - xLD rule:
//   - supersedes all xLD that are lower in the hierarchy (rules 2 and 3)
//   - is superseded by older self and all types that are higher in the hierarchy (rules 1, 3 and 4)
// - xWD rule:
//   - supersedes all xWD that are lower in the hierarchy (rule 3)
//   - is superseded by older self and all xWD and xWO that are higher in the hierarchy (rules 1, 2, 3 and 4)
func reconcile(kap AuthPolicy, host string) error {
  var supersededTypes []PolicyType

	switch kap.PolicyType() {
	case GWO:
		if existOthers(kap, host, []PolicyType{GWO}) {
			return updateAuthPolicyStatus("cannot apply policy for %host", kap)
		}
		supersededTypes = []PolicyType{RWO, GLO, RLO, RLD, GLD, RWD, GWD}
	case RWO:
		if existOthers(kap, host, []PolicyType{GWO, RWO}) {
			return updateAuthPolicyStatus("cannot apply policy for %host", kap)
		}
		supersededTypes = []PolicyType{GLO, RLO, RLD, GLD, RWD, GWD}
	case GLO:
		if existOthers(kap, host, []PolicyType{GWO, RWO, GLO}) {
			return updateAuthPolicyStatus("cannot apply policy for %host", kap)
		}
		supersededTypes = []PolicyType{RLO, RLD, GLD}
	case RLO:
		if existOthers(kap, host, []PolicyType{GWO, RWO, GLO, RLO}) {
			return updateAuthPolicyStatus("cannot apply policy for %host", kap)
		}
		supersededTypes = []PolicyType{RLD, GLD}
	case RLD:
		if existOthers(kap, host, []PolicyType{GWO, RWO, GLO, RLO, RLD}) {
			return updateAuthPolicyStatus("cannot apply policy for %host", kap)
		}
		supersededTypes = []PolicyType{GLD}
	case GLD:
		if existOthers(kap, host, []PolicyType{GWO, RWO, GLO, RLO, RLD, GLD}) {
			return updateAuthPolicyStatus("cannot apply policy for %host", kap)
		}
	case RWD:
		if existOthers(kap, host, []PolicyType{GWO, RWO, RWD}) {
			return updateAuthPolicyStatus("cannot apply policy for %host", kap)
		}
		supersededTypes = []PolicyType{GWD}
	case GWD:
		if existOthers(kap, host, []PolicyType{GWO, RWO, RWD, GWD}) {
			return updateAuthPolicyStatus("cannot apply policy for %host", kap)
		}
	}

	// somehow as an atomic transaction
	{
		supersededAuthPolicies := findOthers(kap, host, supersededTypes)
		updateAuthPolicyStatus("can no longer apply policy for %host - now taken by %kap", supersededAuthPolicies...)
		deleteAuthConfigs(hosts(supersededAuthPolicies, host) - host)
		createOrUpdateAuthConfig(host, kap.AuthScheme)
	}
}

func findOthers(kap AuthPolicy, host string, policyTypes []PolicyType) []AuthPolicy {
  return []AuthPolicy{ other authpolicies targeting the same host (wildcard overlaps included) and whose type is in 'policyTypes' }
}

func existOthers(kap AuthPolicy, host string, policyTypes []PolicyType) bool {
  return len(findOthers(kap, host, policyTypes)) > 0
}

Ok, here goes my 2c regarding this topic…
It’s an effort that aims to make the algorithm as simple as possible, redefining the precedence (hierarchy) order and the actual “compilation” of the policy rules. It aspires to also unify and be applied to the Rate Limit Policies
Hierarchy
It’s based on the GW API and Gui’s take, with some changes:
GL[O] > GW[O] >  RL[O] > RW[O] > RL[D] > GL[D] > RW[D] > GW[D]
My understanding is that if you have a GW Literal Override, means that the level of specificity that the Cluster Admin had in mind needs to overtake any wildcard that most likely are general rules, therefore would always have more weight than any less specific one. Which would be the opposite when it comes to Defaults.
Considerations

One Auth/RL Policy pero Networking Object. First applied, others rejected.
Defining the combination scheme:

Append
Replace
Merge

Ideally it would be an identifier per policy rule applied, which would make it simpler if merge is needed.

Reconciliation Algorithm

Check if the incoming policy target has already a policy attached, if so, ditch it.
Get all the policies
Sort them based on their hierarchy/precedence
Apply the combination scheme to obtain a single policy to apply. Then depending on the scheme:
a. Append: the resulting policy will add every other into a single one
b. Replace: The highest hierarchy wins.
c. Merge: Based on their order, the merging algorithm will replace and combine accordingly.
Profit!

I know it’s a bit simplistic, and might be even naive… but hopefully this will help to craft a good algorithm that could be applied for both types of policies (RL/Auth)
The most difficult, evidently, is the merge approach… but there are good libraries in Golang, such as Mergo, that could help us out.

My contribution:

hostnames for a given authpolicy are read from spec.authScheme.hosts. This spec.authScheme.hosts is optional, if not set, the hostnames are read from the network resource the KAP is targeting. Policy rules should not play any role defining the hostnames.
I would not define any order of preference between "literal" and "wilcard" per se. For example, two policies of type R[O], one for *.com and the other petstore.com, they both apply. Authorino implements longest path match, so both policies would be enforced depending on the workload. They can live side by side in two different authconfigs. Thus, the hierarchy would be straightforward from the Gateway API def: GO > RO > RD > GD
How kuadrant should proceed when there are conflicts (overlapping domains from different policies)

Higher precedence
Lower precedence
Applied policies

*.example.com
*.com
*.example.com, *.com

*.com
*.example.com
~*.example.com~ *.com

Authconfig schemes are not considered as units. Merge strategy can be applied on each authconfig section (patterns, conditions, identity, metadata, authorization, response, denywith). In the previous table, when *.com has higher preference than *.example.com, the policy *.example.com would still apply, however  previously it would had been modified merging with higher level policies. If there is a conflict, the one with higher preference will prevail. A request to a.example.com would hit the modified version of the policy and a request to other.com would hit the unmodified version of the *.com policy.

Surely I am missing tons of things, but food for thought is always  welcome (I guess).

@eguzki, as just discussed offline.

Policy rules should not play any role defining the hostnames.

I believe the policy rules should play a role to the host names in the AuthConfig, otherwise we enable a situation where the AuthConfig covers more that what it suppose to, e.g.:
kind: HTTPRoute:
spec:
  hostnames: [`*.petstore.com`]
---
kind: AuthPolicy
spec:
  rules:
  - hosts: ["api.petstore.com"]

We don't want *.petstore.com in the AuthConfig for the above, just like IMHO we don't want it in the Istio AuthorizationPolicy.

This spec.authScheme.hosts is optional

Maybe authpolicy.spec.authScheme.hosts should not even be an option. authconfig.spec.hosts makes sense and needs to remain required, but the abstraction on top (i.e. the KAP) can read the hosts from the rules and from the network object, as described before.
I can't see any use case where the user would want the ext-authz service triggered but have corresponding AuthConfig (effectively resulting in a 404 Not Found in this case). Perhaps to "lock" the backend service via authorization layer? E.g.
kind: HTTPRoute:
spec:
  hostnames: [`*.company.com`]
---
kind: AuthPolicy
spec:
  rules:
  - hosts: ["*.petstore.company.com"]
  authScheme:
    hosts: ["api.petstore.company.com"]

Meaning:

All requests to *.petstore.company.com trigger the authz service
Requests to api.petstore.company.com passes depending on the evaluation of the AuthConfig
All requests to other subdomains of *.petstore.company.com are denied by the authz service (with a 404)

I would not define any order of preference between "literal" and "wilcard" per se.

That was the same reasoning that previously made me think on RW[O] > GL[O], i.e. a route winning over a gateway in the hierarchy. However, I too was thinking that, in Authorino, both can co-exist, because one is specific (literal), while the other is generic (wildcard).
I will try later to rethink this so it reflects the correct hierarchy between the different types of resources as specified in GW-API and pointed out by @didierofrivia (i.e. Gateway override wins over HTTPRoute override, and the other way around for defaults), and Authorino's specificity that both policies can co-exist and most specific wins.

I think I'll come somewhat out of left field on this but:
Given the Gateway API's mandate that:

policy resources may omit Override or Default fields, but at least one of them MUST be present.

I'd like to propose a more "aggressive" approach and go for one or the other and build from there… If we really want to use the policy attachment concept, we don't really have a choice today anyways… Given the spec's:

GW[O] > RW[O] > GL[O] > RL[O] > RL[D] > GL[D] > RW[D] > GW[D]

I'd be tempted to start out with Default, so that RL wins. And iterate from there… Today that Default at the RL could completely replace whatever a GW policy would declare. Which raises interesting in itself:

how do I disable a RLP at my RL that was inherited from the GW, not make it some arbitrary large number, but completely remove it?.
how would one declare that a RLP has to be present on all RL?
how would one mitigate a DDoS?
...

I don't think that Defaults only will suffice to solve all of the use cases we're coming up with, but it'd be interesting to see how far we can get with "only" Defaults and what problem we face along the way…
So, not to invalidate any of what's been said so far, nor to pun on Overrides, but rather take an applied hands-on approach to one of them and "figure things out as we go"… wdyt?


Asset library: invalid commit hash

Hi,
You gave an invalid commit hash ("Download Commit") on the Godot Asset Library: https://godotengine.org/asset-library/asset/edit/198
You used v0.1 but you have no tag with this name, so it won't work. You should either tag version v0.1 so that it works, or use a commit hash instead, e.g. a6bf1285e0fb9ea1b4abef75e7234360d92164d4.
Note that you can edit your submission directly from the interface.

Ping.

Had to reject the edit since it would not work: https://godotengine.org/asset-library/asset/edit/198
But please feel free to submit it again with a proper commit hash as explained above :)

Oh, sorry. I haven't been to Github this year. I'll fix that.


custom timezone (moment)

Clear and concise description of the problem
I don't want to change all project time zone by moment.tz.setDefault().
Suggested solution
pass timezone when creating momenLocalizer
const localizer = momentLocalizer(moment, timezone);
Alternative
No response
Additional context
No response
Validations

[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Would you like to open a PR for this feature?

[X] I'm willing to open a PR

@ariel22411 So you're saying that your calendar will use one timezone, but you would have other date elements, on your active screen, using a different timezone? Theoretically?

Let's say there is a customer who lives in Los Angeles, and he is looking at his calendar and sees an event between 8 AM and 9 AM. He reports to you that there is some issue. You connect to his user account, but you are located in France, and the times you see on the calendar are different because your time zone is France and his is Los Angeles. So, once we adjust the time zone that you want to display on the calendar, the issue will be resolved.

@ariel22411 I follow so far, I just don't understand your issue. So, you connect to this user's account. Your localizer setup (typically in a useMemo of some kind) would include moment.tz.setDefault(user.timezone), setting all dates in the calendar to use the user's timezone. Somewhere you would also have
// reset default timezone when interface is unmounted
useEffect(() => {
  return () => {
    moment.tz.setDefault()
  }
}, [user.timezone])

Am I missing something? I think our timezone example code even included similar...

The moment.tz.setTimezone function is not recommended for several reasons:

Global impact: Calling moment.tz.setTimezone affects the timezone globally for your entire application. This can lead to unexpected behavior and make it difficult to manage and debug issues related to timezones.

Concurrency issues: If you have multiple components or modules that rely on different timezones, setting the timezone globally can create conflicts and inconsistencies. It becomes challenging to maintain accurate and reliable time calculations.

Unpredictable side effects: Changing the timezone globally can have unintended consequences, especially if you have dependencies or libraries that rely on the default timezone. It can affect other parts of your codebase, causing confusion and introducing bugs that are hard to trace.

Using the moment.tz.setTimezone function is very dangerous, even though I initialize it when I unmount the component. I have a dropdown with a list of teams, and each team can be from a different country. I prefer to pass a timezone prop instead of changing the timezone for the entire application.

We have been using this proposed localizer(PR #2406) for months now, and its been just fine up until recently we realized there is some issues with Daylight Savings Time, I commented with details on this PR: #2406


Does Thruway support secure websockets server (wss://)?

Does Thruway support secure websockets server (wss://)?

This worked for me (not on nginx 1.4.6, but it did after I upgraded to 1.8.0):
  location ^~ /thruway {
    proxy_pass http://localhost:8080;
    proxy_set_header Host $host;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }

Then I could use wss://example.com/thruway for the websocket URL.

This seems to work for me in apache:
Listen 4043 <VirtualHost *:4043>   ServerName example.com   SSLCertificateFile /etc/ssl/example.com.pem   SSLCertificateKeyFile /etc/ssl/example.com.key   SSLEngine On   SSLProxyEngine On   ProxyPreserveHost On   ProxyRequests Off   ProxyPass / ws://localhost:8080 retry=0 keepalive=On   ProxyPassReverse / ws://localhost:8080 retry=0 </VirtualHost>
Then I could use wss://example.com:4043

I used the Apache proxy at first but I was getting random 502 errors: Broken pipe: [client xxx.xxx.xxx.xxx:xxxxx] AH01084: pass request body failed to yyy.yyy.yyy.yyy:8080. Switching to Nginx proxy seems to work flawlessly.

I want to connect to crossbar.io server wss using the thruway client, its seems all the solutions are for a server
how can is use thruway client (NOT SERVER) for wss connections?

@ofumbi - This should not be an issue.

Actually it is. wss://myserver.com. freeszes. BUT ws://myserver.com. is fine. I had to solve this by removing TLS options  then moving the crossbar.io server behind  a proxy and then using the proxy to redirect traffic from  wss://myserver.com =>ws://myserver.com. This way the thruway client connects using   ws://myserver.com  and google chrome (which now requires wss links for Https sites can also connect ) using wss://myserver.com.

If you can't connect to wss:// it's a bug.  Can you open a new issue, for the client, with the steps to reproduce?


[TEVA-2093] GH Actions tweaks for check runs

Jira ticket URL
https://dfedigital.atlassian.net/browse/TEVA-2093
Changes in this PR:

Make deploy app check look for correct name
Switch to DFE-Digital turnstyle action, with initial-wait-seconds set to 20
Remove unused TF VAR environment variable in monitoring step

Review app deployed to https://teaching-vacancies-review-pr-3198.london.cloudapps.digital


Take a look at yazi

Hey,
I noticed that yazi has a fairly comprehensive image display module that might be worth drawing some inspiration / code from (it's MIT licensed).
Specifically start at https://github.com/sxyazi/yazi/blob/main/adaptor/src/adaptor.rs and then look at how it progresses from there to image then {iterm2,kitty,sixel,uberzug} There's some logic that seems specific to the yazi app loading and caching files, but the core of this seems pretty robust.
Pinging @sxyazi for awareness
P.s. I wonder if you'd want to change the name this project to ratatui-image rather than ratatu-image for discoverability and eventual inclusion in the ratatui-org github org? (My fat fingers were surprised when https://crates.io/crate/ratatui-image was 404).

Hi @joshka, thanks for your appreciation of Yazi. I'm more than happy to see Yazi's code being used, and If there's anything I can do to help, please feel free to ask!
Finally, I'd like to thank you for your hard work on Ratatui; it's a great project, and it's because of Ratatui that Yazi exists!

Thank you @joshka! I can definitely use yazi's code for the missing implementations here and general reference.
I will change the crate's name for discoverability, thanks for the suggestion.


Add debounce to transporter search

:rocket: Feature Request
Within the manifest form, I think it would be nice from a UX perspective to not require five characters before returning transporter results. I was experimenting with removing the character count requirement and instead adding a debounce filter on the input instead to limit requests to the API.
In the following example, I paused my search twice to refine my search and made two requests to the server - however, I noticed there is a third request with an empty epaId querystring after selecting the transporter.
https://github.com/USEPA/haztrak/assets/21269876/6d0f6146-2e62-40f7-a173-8d02ba92e127

This is a great idea, and I agree that it would improve the user experience and has the added benefit of reducing excessive traffic between Haztrak and the e-Manifest system (also I appreciate the video). We do need to consider the constraints of the e-Manifest Site Search service which we call on the back end if the user's org has provided API credentials. The epaSiteId field needs to be at least 2 characters.
I believe we still need to implement that constraint in our back end. I see a couple options:
Option A:

Implement the proposed debounce filter
Remove the 5-char minimum in our front end
Implement the 2-char minimum validation in our back end. If validation fails, just return the sites found in the Haztrak DB (same logic as if the user's org has not supplied e-Manifest API credentials).

Option B:

Implement the proposed debounce filter
Implement the 2 character minimum validation in our back end
Reduce the 5-char minimum to a  2-char minimum in our front end.

I'm partial to the first option A, and seems inline with your initial proposal.
Let me know what you think @sheckathorne.

Honestly, I don't think I understand the implications of A versus B as well as you, so I would lean with you toward option A if you think that's best.
With that said, I think the implementation on the back end would look like this. Let me know if you had something different in mind:
# /apps/rcrasite/services/rcra_site_services.py
class RcraSiteService:

   ...

  def get_or_pull_rcra_site(self, site_id: str) -> RcraSite:
      """
      Retrieves a rcra_site from the database or Pull it from RCRAInfo.
      This may be trying to do too much
      """
      MIN_EPA_ID_LENGTH: int = 2
      if (
          RcraSite.objects.filter(epa_id=site_id).exists()
          and len(site_id) < MIN_EPA_ID_LENGTH
      ):
          logger.debug(f"using existing rcra_site {site_id}")
          return RcraSite.objects.get(epa_id=site_id)
      new_rcra_site = self.pull_rcrainfo_site(site_id=site_id)
      logger.debug(f"pulled new rcra_site {new_rcra_site}")
      return new_rcra_site

We went ahead and implemented a check using the view's DRF Serializer class in #728 like so
    class HandlerSearchSerializer(serializers.Serializer):
        siteId = serializers.CharField(required=True)
        siteId = serializers.CharField(
            required=True,
            min_length=2,
        )
        ...

I also went ahead and implemented a new RcraSiteSearch class (maybe overkill) which validates a 2-character minimum before calling the e-Manifest site search service as well.
A future PR for this can also remove the logic in the front end for a 5-character minimum

closed with #729
Thanks again @sheckathorne. We appreciate your input on ways that we could enhance this project nad hope to see more awesome feature requests like these.


Improves integrations/jdbc/jdbc to better support future JPA improvements

Initial work.
Signed-off-by: Laird Nelson<EMAIL_ADDRESS>

This PR breaks a chunk off of my ongoing work (https://github.com/helidon-io/helidon/compare/helidon-3.x...ljnelson:helidon:nested-transactions-helidon-3.x) to improve JPA support.  This "chunk" pertains to JDBC only.
A lot of this PR is formatting and cleanup.
The rest of the PR is devoted to setting up a java.lang.reflect.Proxy-free set of "delegating" classes that Do The Right Thing™ when wrapping other JDBC constructs.
The most interesting class is ConditionallyCloseableConnection, which already existed in prior versions of Helidon, but here is significantly and necessarily enhanced for future JPA support.


timeout if pathio takes too long

how to write timeouts: https://github.com/golang/go/wiki/Timeouts

This wasn't quite the timeout I had in mind, but would probably also be valuable. What we've seen in the past indicates that we are able to fetch the Reader, but something hangs later on. I was talking about adding a timeout to calls to Read on the Reader returned by pathio. One way to do this would be a a ReaderWithTimeout function that takes in a Reader and returns a Reader with a timeout on Read.


Wish list for datasets

This is a wish list for new datasets that would be very helpful in benchmarking methods in a particular task. If you have anything to add, please comment here with the dataset description, and the task name.

@wes-lewis Those can be obtained quite easily... I believe scvelo even has example datasets in their tutorial. Or are you looking for something specific?


adlplug,opnplug: patch JUCE to fix build on Darwin

Description of changes
I had to cherry-pick several patches from JUCE upstream, since ADLplug depends on an older version. This will no longer be necessary after updating ADLplug to the latest commit on its master branch.
When I tried to do the update, I ran into linking issues related to undefined symbols, and I realized I don't know enough to resolve those yet. So that is future work.
ZHF: #172160
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Ah, sorry for the duplicated effort but I pushed a PR earlier that bumps those packages & fixes linking problems: https://github.com/NixOS/nixpkgs/pull/172702. It's a draft because I don't know where AU plugins are supposed to go on Darwin, but lemme just install them somewhere and un-draft it.

Sounds good and better than what I have. Thank you for fixing!


Update RStudio Server Notebook

Relate to: https://issues.redhat.com/browse/RHOAIENG-4374
Description
This pull request updates the following components:

The RStudio Server version from rstudio-server-rhel-2023.06.1-524 to rstudio-server-rhel-2023.12.1-402
Update R from 4.3.1 to 4.3.3
Update the RStudio related ImageStreams and kustomization artifacts
NOTE: I moved down the N-1 to N-2, the N to N-1 and the updated one to N. Shall we keep the N-2 or we can drop it?

How Has This Been Tested?
Way 1

Run podman run --network=host --name validation-container quay.io/opendatahub/workbench-images:rstudio-c9s-python-3.9-pr-453
Inspect the notebook

Way 2

Import the notebook in ODH

Lunch the newly added notebook , you should see the following

Merge criteria:

[x] The commits are squashed in a cohesive manner and have meaningful messages.
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has manually tested the changes and verified that the changes work

i do not think this is working, odh-rstudio-gpu-notebook-n-2 is not set in the params.env and rstudio-gpu-notebook-imagestream.yaml

CI validation of the params.env file for this PR failed but it got still merged :upside_down_face:  https://github.com/opendatahub-io/notebooks/actions/runs/8278818862/job/22651961748 - can I improve it anyhow so we can avoid this in the future? :slightly_smiling_face:

We will address the fix in a separate PR, shortly.


Refactor push update and notifications

As title.

Tests are still failing. CI fail is related.

replaced by #13381


Update infisical-secrets-check.yml

:white_check_mark: Build POC-dotnet-Asterisk 1.0.341 completed (commit https://github.com/GuilhermeStracini/POC-dotnet-Asterisk/commit/2ee4827f78 by @guibranco)


Ability to disable or change refresh time in polling

This got brought up in the component redirect PR by @frbor: https://github.com/adamghill/django-unicorn/pull/98#issuecomment-747920253.
Potential options:

a custom class that could be returned from the component action
a new modifier poll.stop or poll.disable which would provide a way to stop polling after it has started

I added a unicorn:poll.disable attribute that controls whether the poll is disabled or not in c437579. Example of how it can be used: https://github.com/adamghill/django-unicorn/commit/c43757988abb0488a94d15c92ff0969091aa2f1d#diff-ae29ce65d990223fc12fc1d24fcab92062ce7e202a38ed777aae534e74698b39R1-R7.

PR for changing polling options from an action method: https://github.com/adamghill/django-unicorn/pull/106.

#106 merged in and documentation updated in 196fa5073dc2924ca47482eb88bc2be493c9c966. This should get released in 0.14.0. Closing!

#106 merged in and documentation updated in 196fa5073dc2924ca47482eb88bc2be493c9c966. This should get released in 0.14.0. Closing!


Add a uniswap anchored view

This commit adds a Uniswap anchored Open Oracle view. We intend to propose switching the Compound Protocol to use this oracle, together with the Coinbase reporter feed.
The idea is that the price can only be set to a value coming from the reporter, but must be reasonably close to the anchor price. The configuration for the oracle is passed up entirely in the constructor, including which uniswap market to use as an anchor for each reported price.
Tokens can also be configured to have FIXED_ETH or FIXED_USD prices. FIXED_USD prices are constant (since the oracle returns prices denominated in USD). FIXED_ETH prices are a multiple of the ETH price, which must be reported in order to be used.
The view also directly exposes the interface for compatibility with the Compound Protocol, which currently expects to fetch prices by CToken address.
In the unexpected event that the reporter believes their key to be compromised, they may sign a 'rotate' message which can be used to invalidate the reporter. In such an event, the view is configured to fall back to using the anchor price directly.
In order to track the Uniswap price, the price accumulators those markets expose are tracked by block timestamp. Whenever prices are posted, the tracked accumulators are updated in such a way as to always have an accumulator from at least a certain period ago. When a current anchor price is needed, it is always the time-weighted average price over at least the configured anchor period.

Codecov Report

:exclamation: No coverage uploaded for pull request base (compound/0.6.6@c4c4683). Click here to learn what that means.
The diff coverage is n/a.

@@                Coverage Diff                @@
##             compound/0.6.6      #99   +/-   ##
=================================================
  Coverage                  ?   97.46%
=================================================
  Files                     ?        6
  Lines                     ?      553
  Branches                  ?      148
=================================================
  Hits                      ?      539
  Misses                    ?       14
  Partials                  ?        0

Merged to master


position argument in the  onData callback

I just wanted to confirm my understanding on the position argument. I have a bit different requirement, in our app. we have the muxer in a worker, the worker forwards the muxed codec chunks to the main thread. The actual file write is happening in the main thread, we are using Streamtarget with onData callback for this.
Is it a  correct assumption that the 'position' argument corresponds to the file write offset, the writer just needs to seek to the position and write the chunk at the position in the file ??
Please advise.

That is correct! The chunks need to be written at the provided positions in the final file, and in the same order in which they are spit out by the muxer, for the output file to be correct.

I tried the above approach , forwarding muxed chunks to the main thread from  web worker and writing them to the file. I am getting an unplayable file with some data and my mp4 player complains  "unrecognized" format. I am pasting below muxer settings. I verified that all chunks are being written to the file honoring the 'position' argument.
how does 'finalize' work in case of streamedTarget ??
are my muxer settings correct for my usecase ?
target : Mp4Muxer.StreamTarget
fastStart : false,
firstTimestampBehavior : 'offset'
Please advise.

I am not expert on mp4 ,
first 100 bytes of a valid mp4 file  look like this
od -x intro.mp4 -N 100
0000000 0000 2000 7466 7079 7369 6d6f 0000 0002
0000020 7369 6d6f 7369 326f 7661 3163 706d 3134
0000040 0500 e3b7 6f6d 766f 0000 6c00 766d 6468
0000060 0000 0000 0000 0000 0000 0000 0000 e803
0000100 0300 4088 0100 0000 0001 0000 0000 0000
0000120 0000 0000 0100 0000 0000 0000 0000 0000
0000140 0000 0000
0000144
in my case it is like this
od -x soundkone.net_recording_2023-12-12\ 15_22_05.mp4 -N 100
0000000 0000 1c00 7466 7079 7369 6d6f 0000 0000
0000020 7369 6d6f 7661 3163 706d 3134 0000 0100
0000040 646d 7461 0000 0000 0000 1000 fff8 f8fe
0000060 feff fff8 f8fe feff fff8 f8fe feff fff8
0000100 f8fe feff fff8 f8fe feff fff8 f8fe feff
0000120 fff8 f8fe feff fff8 f8fe feff fff8 f8fe
0000140 feff fff8
0000144

To see if there's something wrong with your config, switch out StreamTarget for ArrayBufferTarget and send over one large buffer once muxing is complete. If that works, then there's likely something wrong with how you write the chunks created by the StreamTarget. Are you absolutely certain you are writing the chunks in-order, and at the correct offsets? I've tested this feature time and time again and it should work correctly, so I doubt it's a bug in my library.

Thanks, let me try and get back

test with arraybuffertarget and it works fine. it seems the bug is in our code.

Have you found it yet? It can be tricky with writing files sometimes, sometimes seeking isn't happening.

Hi ,
Can you please try this at your end as well , after spending couple of days , i am still not able to figure out why such a simple thing is not working, let me describe our architecture and things i have tried at my end.
As we know , that worker cannot write to file system, we send the muxed chunks to the front end over a message channel, we have to move the muxer to the worker as we were not getting desired frame rate.  The offset ( position ) and the data ( arraybuffer ) are both send to the front end and the data is written to the file at the offset.
I have tried filesystemtarget and arraybuffer target. both work flawless.

I'm sorry this is causing you trouble. There's a definitive test you can do to check if this is an issue with StreamTarget or an issue with your file writing code. I always use this test to ensure it works correctly.
This is the test:
let buffer = new ArrayBuffer(0, { maxByteLength: 1e9 });
let bytes = new Uint8Array(buffer);

let muxer = new Mp4Muxer.Muxer({
    target: new Mp4Muxer.StreamTarget(
        (data, position) => {
            buffer.resize(position + data.byteLength);
            bytes.set(data, position);
        },
        () => {
            downloadBlob(new Blob([buffer.slice()]));
        }
    ),
    // ...
});

const downloadBlob = (blob) => {
    let url = window.URL.createObjectURL(blob);
    let a = document.createElement('a');
    a.style.display = 'none';
    a.href = url;
    a.download = 'davinci.mp4';
    document.body.appendChild(a);
    a.click();
    window.URL.revokeObjectURL(url);
};

It uses the StreamTarget to write directly into an Uint8Array, which simulates writing to a file. The final buffer is then downloaded.
Try using this code and sending the final buffer over to your frontend to download. If it works, then we know StreamTarget is not the culprit here. This code also shows nicely how writing is meant to happen, regarding byte offsets and so on.

Thanks for the input , i have tried sending a large buffer written by stream target to the main thread and it produced a perfect file. now , it is clear the library has no issues.
regards,

How are you writing to disk, like what API are you using?

i am using writable.write with offset

Maybe this can help: https://github.com/Vanilagy/webm-muxer/issues/25#issuecomment-1732441066

Please look at the screenshot of the first 16 bytes of the packet dump returned by the onData callback. Every thing works fine till line 5 in the image, from line 5 the packets go out of sync , orig is the packet given by the library and the copy is a copy packet, as we see they go out of sync for few packets like moov , mdat etc.
console.log is an asynchronous function it keeps a reference of the variables it is given and expands them when it prints,
so , copy is the actual data and the orig may have been modified by the library by the time it gets printed.
Copy remains out of sync with the orig and while all the data is perfect, the segment headers never get written to the final file. This segment information gets written in the finalize step which is never seen by the onData callback. I am collecting till the final packet. some thing to do with the finalize step i guess. If the logic is being shared between webm and mp4 then likely we have  similar issue in the webm code as well.
for reference i am pasting the code of onData callback ,
  let target = null;
    let sendChunkToRecorder = ( data, position ) => {
      let sview = new DataView ( data.buffer );
      let abuf = new ArrayBuffer ( data.length );
      let dview = new DataView ( abuf );
      for ( let i = 0; i < data.length ; i++ ) {
        dview.setUint8 ( i, sview.getUint8 ( i ) );
      }
      //console.log ( '=> position : ', position, ' data.len : ',  abuf.byteLength );
      hexdump ( 'orig:', data.subarray ( 0, 16 ) );
      hexdump ( 'copy:', new Uint8Array ( abuf, 0, abuf.byteLength < 16 ? abuf.byteLength : 16 ) );
      recorderInputPort.postMessage ({ offset : position, data : abuf }, [ abuf ]);
    };
    if ( recordingContainerFormat == 'mp4' ) {
      target = new Mp4Muxer.StreamTarget (
        ( data, position ) => {
          sendChunkToRecorder ( data, position );
        }, //onData
        () => {
        }, //onDone
        { chunked : true, chunkSize : 4 * 1024 });

reopening issue for further investigation

I see there may be an issue here with incorrect reuse of buffers meaning that buffer contents change after they are emitted by the onData callback (can't confirm yet).
If you copy the buffer immediately after receiving it from the StreamTarget, and then write those into a file, is the resulting file correct? Should be, right? Since then you'd be doing exactly what this does.

I doubt the data emitted by the StreamTarget changes tho?? When new data is added to the StreamTarget internally, I do this:

The data is duplicated using slice and can't change again. And also your statement about console.log being asynchronous isn't exactly right; it is synchronous because it has to evaluate its arguments at the time it is called. You were probably referring to those "rich" logs, where it's an object you can expand by clicking an arrow, which are indeed evaluated only when you expand them. Unless your hexdump is async tho, I can't say if it is because you haven't included it.
I also think it's strange that it is your copies which are all zeroes? Like, shouldn't the copies contain proper data and the original gets messed up? I also notice that your copies only are all zeroes when they are shorter than 16 bytes, so what's up there? Strange stuff indeed.
Maybe it has something to do with postMessage. Since you're transferring the buffer, the buffer will get "detached" on the transferring side, meaning it gets truncated to length 0. Then, when you make a Uint8Array out of it, it displays all zeroes.

Thanks for the reply , I can wait till the end of holidays , please enjoy your holidays.
coming to the analysis. please look at the points below
i) The code works fine for all the packets and it fails for the last few packets, both the copy and orig are same byte by byte, only the last few packets go out of sync,  the packets are Trak, mdia, stbl, minf, moov.
ii) Yes the logs i were referring are rich logs, which contain complex objects. so, these are expanded lazily i guess, i might be wrong though, there is no clear evidence for this on internet and every browser does this differently.
iii ) The buffer being sent through postmessage is allocated afresh every time and is sent to the consumer as detached buffer.  abuf enclosed in []'s.
recorderInputPort.postMessage ({ offset : position, data : abuf }, [ abuf ]);
Below is the new dump which is dumping first 8 bytes of the buffer , again these are the same packets which are all zeros. only particular packets are all zero's [ the packets are enclosed in yellow square ]

here is the code for hexdump, i just lifted it out of internet.
function hexdump( uint8Array ) {
  const chunkSize = 8;
  for (let i = 0; i < uint8Array.length; i += chunkSize) {
    const chunk = uint8Array.subarray(i, i + chunkSize);
    const hexValues = Array.from(chunk, byte => byte.toString(16).padStart(2, '0')).join(' ');
    const asciiValues = Array.from(chunk, byte => (byte >= 32 && byte <= 126) ? String.fromCharCode(byte) : '.').join('');    console.log(
      `${i.toString(16).padStart(8, '0')} | ${hexValues.padEnd(chunkSize * 3 - 1)} | ${asciiValues}`
    );
  }
}

adding one more line in the onData callback , basically taking a copy of the data and using it solved the problem.  pasting the new callback for reference.
 let sendChunkToRecorder = ( data, position ) => {
      let data2 = new Uint8Array ( data );
      let sview = new DataView ( data2.buffer );
      let abuf = new ArrayBuffer ( data2.length );
      let dview = new DataView ( abuf );
      for ( let i = 0; i < data2.length ; i++ ) {
        dview.setUint8 ( i, sview.getUint8 ( i ) );
      }
      //console.log ( '=> position : ', position, ' data.len : ',  abuf.byteLength );
      hexdump ( 'orig:', data2.subarray ( 0, 8 ) );
      hexdump ( 'copy:', new Uint8Array ( abuf, 0, abuf.byteLength < 8 ? abuf.byteLength : 8 ) );
      recorderInputPort.postMessage ({ offset : position, data : abuf }, [ abuf ]);
    };

Glad this works, strange it didn't before. I think you could simplify your copying logic:
 let sendChunkToRecorder = ( data, position ) => {
      let abuf = data.buffer.slice();
      recorderInputPort.postMessage ({ offset : position, data : abuf }, [ abuf ]);
    };


很棒的项目，成功应用了，但是当我输入比较长的问题的时候，会报错说”内存不足“？

ggml_new_tensor_impl: not enough space in the context's memory pool (needed 819409712, available 805306368)
[1]    52017 segmentation fault  ./main -m zh-models/7B/ggml-model-q4_0.bin --color -f prompts/alpaca.txt -ins

看到你现在的n_batch=512，batch size 可以调小一些，其实对速度没有什么影响。
以下是llama.cpp官方的解释：
-   `-b N, --batch_size N`: Set the batch size for prompt processing (default: 512). This large batch size benefits users who have BLAS installed and enabled it during the build. If you don't have BLAS enabled ("BLAS=0"), you can use a smaller number, such as 8, to see the prompt progress as it's evaluated in some situations.

看到你现在的n_batch=512，batch size 可以调小一些，其实对速度没有什么影响。 以下是llama.cpp官方的解释：
`-b N, --batch_size N`: Set the batch size for prompt processing (default: 512).
This large batch size benefits users who have BLAS installed and enabled it during the build.
If you don't have BLAS enabled ("BLAS=0"), you can use a smaller number, such as 8,
to see the prompt progress as it's evaluated in some situations.

哇，非常感谢！！！现在可以了。想不到这么快就有回复了！！！


use Retry provider in forge script

Component
Forge
Describe the feature you would like
#1826 shows we can run into ratelimit issues in forge script
we should be able to solve this by using
https://github.com/foundry-rs/foundry/blob/7e9eba5656c9371679e8963fb83900f0961982c8/anvil/src/eth/backend/fork.rs#L393
for the provider in forge script
Additional context
No response

Of note that we may want to be more aggressive with how we submit requests and maybe nto want to have a exponential backoff but a linear backoff retry

Closing - fixed in #1824


Peer peer code review

Hi @rubydevi,

Great job is done so far some required changes

Required Changes ♻

Please try to use the ES6 Module Arrow function  in place of the normal function It's best to practice javaScript ES6.

@Chandan-devs-tech thanks for your review, I have resolved this issue.


Integration Tests

Overview
Builds a structure for creating rendering/integrations tests that are run with the rest of the unit tests. These are rendered in WebGL and Canvas renderers and then compared against a pre-existing solution.
Rendering solutions are generated by using this tool: npm run renders

Travis CI seems to not be able to run WebGL content natively (anyone have experience with this?), but it's still valuable to run WebGL renders locally. Code coverage is going to be slightly different in each case: local coverage is about 51.24%, on Travis: 47.92%.

Try specifying the Firefox version to see if a more up-to-date version works:
addons:
  firefox: "49.0"

If that doesn't work, you can use headless-gl: https://github.com/stackgl/headless-gl#how-can-i-use-headless-gl-with-a-continuous-integration-service

This approach isn't perfect and I hear you about the Mocha-inconsistency with the other tests. The trade-off here is that it is very easy to generate (or re-generate) a rendered solution from a bit of code because the rendering tests are authored as somewhat standalone. Having some GUI tooling for generating the test makes it easier to debug and preview the render tests.
It would be easy for these render tests to support sinon, chai, etc. That's a good suggestion.
I could also potentially move the pixi-tests-tool code/functionality here too.

Okay, I integrated the app for creating render solutions (npm run renders) and provided full-parity with Floss for running the render tests: (including sinon, chai, etc.).

full-parity with Floss for running the render tests: (including sinon, chai, etc.)

looks like the tests are still not using mocha/chai? Is that intended?

Yes, the render test code is now using chai's assert (they were using Node's before). It is intended that the render code be externalized so the solutions can be re-generated.
Someone could create additional integration Mocha tests, but with these I really wanted to validate that the output render is exactly what it's suppose to be. So these render Mocha test simply load a snippet of code and load a solution and compares the two. If they are within a tolerance, the test passes.

Why is it using assert there instead of expect-style like the rest of the tests?

Because I'm lazy :) I can change to expect


Sunstone milti IP view broken on the VM pages

Description
The screenshot says everything:

To Reproduce
Create 18 VMs (more than the default view of 10 per page), the first one with Alias IP
OpenVM's view and click on the next tab (or the number)
The result is in the screenshot of the description
Expected behavior
The expected dropdown menu/dialogue should appear.
Details

Affected Component: [Sunstone]
Hypervisor: [KVM]
Version: [5.12.7]

Additional context

Progress Status

[x] Branch created
[x] Code committed to development branch
[ ] Testing - QA
[x] Documentation
[x] Release notes - resolved issues, compatibility, known issues
[ ] Code committed to upstream release/hotfix branches
[ ] Documentation committed to upstream release/hotfix branches

Further observations:
Hitting the reload button, the page is fixed.
Then the issue re-appear when I switch back on the first page, again, hitting the reload button will fix the page.

This issue is duplicated, and solved in #5037

This issue is duplicated, and solved in #5037

Please note the reported version 5.12.7 - IMO it is not solved with #5037

Please note the reported version 5.12.7 - IMO it is not solved with #5037

@tinova PRs to be approved:

Code
Docs

Needs testing

@tinova PR to be approved:

Tests


make pd_object_namer more discriminating

This fixes #33 by not applying ~ to data object names

working fine thanks!


[BE][BUILD]Failed at compiling formats/orc/apache-orc

Failed at compiling formats/orc/apache-orc at my arm64 build testing. The error message seems to be not related to my patch.
Steps to reproduce the behavior (Required)
OS: openeuler 20.03LTS-SP3
gcc:10.3.0
cmake:3.20.1
python 3.7.9
./build.sh
Expected behavior
Build success
Real behavior (Required)
++++ <STARROCKS_HOME> is set. formats/orc/apache-orc is compiled in StarRocks env ++++
-- Build type: RELEASE
-- compiler GNU version 10.3.0
-- SNAPPY_HOME: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed
-- Found the Snappy header: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include/snappy.h
-- Found the Snappy library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib64/libsnappy.a
-- ZLIB_HOME: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed
-- Found the ZLIB header: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include/zlib.h
-- Found the ZLIB library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib/libz.a
-- Found the ZLIB static library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib/libz.a
-- ZSTD_HOME: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed
-- Found the zstd header: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include/zstd/zstd.h
-- Found the zstd library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib
-- Found the zstd static library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib64/libzstd.a
-- LZ4_HOME: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed
-- Found the LZ4 header: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include/lz4/lz4.h
-- Found the LZ4 library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib
-- PROTOBUF_HOME: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed
-- Found the Protobuf headers: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include
-- Found the Protobuf library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib/libprotobuf.a
-- Found the Protoc library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib/libprotoc.a
-- Found the Protoc executable: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/bin/protoc
-- Found the Protobuf static library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib/libprotobuf.a
-- Found the Protoc static library: /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/lib/libprotoc.a
-- Configuring done
-- Generating done
-- Build files have been written to: /home/hqs/starrocks-arm_neon-adaptation-patch/be/build_Release
[2/1110] Building CXX object src/formats/orc/apache-orc/c++/src/CMakeFiles/orc.dir/Adaptor.cc.o
FAILED: src/formats/orc/apache-orc/c++/src/CMakeFiles/orc.dir/Adaptor.cc.o
/usr/local/gcc/bin/../bin/g++ -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H -DUSE_AVX2KI -Isrc/formats/orc/apache-orc/c++/src -I../src/formats/orc/apache-orc/c++/src -I../src/formats/orc/apache-orc/c++/include -Isrc/formats/orc/apache-orc/c++/include -I../src -I../test -I../../gensrc/build -I/home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/open_jdk/include -I/home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/open_jdk/include/linux -I/home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed -I/home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include/zstd -I/home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include/lz4 -isystem /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include -isystem /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/starcache/include -isystem /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/gperftools/include -isystem /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/jemalloc/include -isystem /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include/thrift -isystem /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/include/event -isystem /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed/llvm/include -Wall -Wno-sign-compare -Wno-unknown-pragmas -pthread -Wno-register -Wno-strict-aliasing -fno-omit-frame-pointer -std=gnu++20 -D__STDC_FORMAT_MACROS -Wno-deprecated -Wno-vla -Wno-comment -fno-sized-deallocation -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -Werror=return-type -Werror=switch -DWITH_STARCACHE -march=armv8-a+crc -I /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed//ksl/include -L /home/hqs/starrocks-arm_neon-adaptation-patch/thirdparty/installed//ksl/lib -lavx2ki  -Wno-attributes -DS2_USE_GFLAGS -DS2_USE_GLOG -faligned-new  -Werror -fcoroutines -g -Wno-unused-local-typedefs -O3 -gdwarf-4 -DNDEBUG  -Wall -Wno-unknown-pragmas -Wno-conversion -Werror  -Wall -Wno-unknown-pragmas -Wno-conversion -Werror -O3 -DNDEBUG -fPIC -std=gnu++2a -MD -MT src/formats/orc/apache-orc/c++/src/CMakeFiles/orc.dir/Adaptor.cc.o -MF src/formats/orc/apache-orc/c++/src/CMakeFiles/orc.dir/Adaptor.cc.o.d -o src/formats/orc/apache-orc/c++/src/CMakeFiles/orc.dir/Adaptor.cc.o -c ../src/formats/orc/apache-orc/c++/src/Adaptor.cc
In file included from ../src/formats/orc/apache-orc/c++/src/Adaptor.cc:19:
src/formats/orc/apache-orc/c++/src/Adaptor.hh:134:11: error: ISO C++11 requires whitespace after the macro name [-Werror]
134 |   #define std::isnan(XXX) isnan(XXX)
|           ^~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:162:11: error: ISO C++11 requires whitespace after the macro name [-Werror]
162 |   #define std::mutex pthread_mutex_t
|           ^~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:162: error: "std" redefined [-Werror]
162 |   #define std::mutex pthread_mutex_t
|
src/formats/orc/apache-orc/c++/src/Adaptor.hh:134: note: this is the location of the previous definition
134 |   #define std::isnan(XXX) isnan(XXX)
|
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:11: error: ISO C++11 requires whitespace after the macro name [-Werror]
163 |   #define std::lock_guardstd::mutex LockORC
|           ^~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163: error: "std" redefined [-Werror]
163 |   #define std::lock_guardstd::mutex LockORC
|
src/formats/orc/apache-orc/c++/src/Adaptor.hh:162: note: this is the location of the previous definition
162 |   #define std::mutex pthread_mutex_t
|
../src/formats/orc/apache-orc/c++/src/Adaptor.cc:64:2: error: #error ("pread() undefined: unknown environment")
64 | #error("pread() undefined: unknown environment")
|  ^~~~~
In file included from ../src/formats/orc/apache-orc/c++/src/Adaptor.cc:19:
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:173:3: note: in expansion of macro ‘std’
173 |   std::string to_string(double val);
|   ^~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:174:3: note: in expansion of macro ‘std’
174 |   std::string to_string(int64_t val);
|   ^~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:14: error: expected identifier before ‘::’ token
163 |   #define std::lock_guardstd::mutex LockORC
|              ^~
src/formats/orc/apache-orc/c++/src/Adaptor.hh:163:16: error: ‘lock_guard’ in namespace ‘::’ does not name a template type
163 |   #define std::lock_guardstd::mutex LockORC
|                ^~~~~~~~~~
StarRocks version (Required)
arm_neon-adaptation-patch

it is related to your patch. cmake in https://github.com/StarRocks/starrocks/blob/main/be/src/formats/orc/apache-orc/c%2B%2B/src/CMakeLists.txt will do a simple compiling to check if the libstdc++ has the standard stl headers such as mutex, thread.
your change breaks the link, causes the cmake think these stl libraries are not available.


Proper escaped string support

Right now only quotes and slashes are handled properly. Other control chars and \u encoded unicode is not handled.
Refer to spec for more info on what has to be handled https://www.json.org/json-en.html

This is a known issue but I'm adding it here to be tracked. There are quite a number of things I want to change before this. So despite it being a major conformance issue its not actively being worked on at the moment.

Shouldn't we be unescaping unicode and control chars? #71 appears to just be storing the escaped versions of those chars.


Saigon not supported

Saigon jailbreak (10.2.1) isn't supported, and as I do no longer own a jailbroken device, I'd like for someone to add support for it.
Thanks !

I’m working on this, I’ll create a PR soon.

Commit https://github.com/lululombard/CertRemainTime/commit/8edbe0ccaf994259647f0c27dc1f2f201b1eebf7 closes this.


Solves issue 1

Fixes issue #1 Inconsistent Sidebar Styles by editing the _content.html.erb partial.

Please explicitly describe the changes you have made.

Fixes issue #1 Inconsistent Sidebar Styles by editing the _content.html.erb partial:

Edited the sidebar-title class title (previously sidebar_title)
Edited the sidebar-body class title (previously sidebar_body)


Thread Safety?

Hello :)
I'd like to use magic in my project to detect MIME types, in addition to tree_magic (which uses the freedesktop MIME database).
The problem is: I'm using multithreaded async Rust, which requires futures to be Send. Looking at the docs for magic, I can see that a Cookie is !Send + !Sync.
Usually, I'd just wrap stuff in Arc<Mutex<T>>, but that requires T: Send.
A working alternative I'm using at the moment is to encapsulate anything magic-related within tokio::task::spawn_blocking, but this is suboptimal because a) I need to re-initialize the Cookie every time and b) I need to move the data I'm analyzing into that block, which is costly.
I'm still a beginner w.r.t. Rust -- do you think it'd be possible to share a Cookie between threads safely?

Hi!
First of a bit of nitpicking;

https://github.com/robo9k/rust-magic-sys is the repo for https://crates.io/crates/magic-sys
https://github.com/robo9k/rust-magic is the repo for https://crates.io/crates/magic

That being said your issue concerns both of them, i.e.

magic_sys::magic_t
magic::Cookie

The data in magic_t is just an opaque handle from the libmagic C library. While it would be safe to copy the handle as plain data, the question remains whether the libmagic C library implementation can deal with that.
Unfortunately it seems like the answer is; no, the libmagic C library itself is not thread-safe at all.
Here's a couple indicators as to why:

the man page for libmagic (source, readable) never even mentions threads
the libmagic issue tracker has no entries for threads
except for one bug in the official Python binding
whose changes, while I'm not proficient in Python, do create new thread-local object instances
the libmagic mailing list only has one entry
which seems to be about issues where even analyzing different files from multiple threads would run into issues
Apache mod_magic_mime, which uses modified libmagic source code, has a note that states that after the changes the module "[..] should be reentrant now but has not been tested in a threaded environment"
which suggests the original code is not thread-safe
an old StackOverflow question has the simple answer of "No, you cannot share libmagic handle/descriptor between two or more threads, in this sense libmagic is not threadsafe" but without elaborating further

So tl;dr is no, I believe it's not safe to share nor send magic_sys::magic_t nor magic::Cookie between threads because the underlying libmagic library does not support this.
Since you are using tokio, have you looked at their bytes crate? It can allow creating a shareable instance which you should be able to use with magic::Cookie::buffer(), but I have not tested this myself.
Since you are using async and libmagic requires a handle-per-thread, maybe using a thread-per-core such as with Glommio could further help performance, but I have not explored this at all.
Note that as a micro optimization (but you'd better benchmark this), you could create a smaller magic database for use with magic::Cookie::load() which might make creating those instances slightly less costly, but only works if you know you don't need the full default database.
I'd like to hear which solution you end up with and also which parts of the magic and magic-sys documentation could be improved to explain the lack of Send and Sync.

Hey,
thanks for the in-depth analysis!
It's intuitive that using a libmagic handle from multiple threads in parallel is a bad idea, since libmagic does not explicitly allow this. However, I'm still not 100% sold on why it shouldn't be possible to encapsulate the handle within a Mutex and have calls to magic::Cookie::buffer() in the critical section. But in the end, you're probably right: libmagic does not explicitly allow this either, and all presented solutions seem to resort to one-handle-per-thread solutions.
After some benchmarks, opening the database takes only a handful of milliseconds, which is fine for my use case.
So in the end, I didn't come up with any fancy solution, and just do it like this for now.
If this comes up as performance-critical for me in the future, I might resort to something like a thread pool for libmagic handles, and communicating with them via channels, or something like that.
Thanks again!


New version: gileli121.windowtop version 5.22.0

Pull request has been created with WinGet Automation :rocket:
Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

2023-09-21 16:06:12.632   Exception message:
Internal error: Cannot expand "pf64" constant on this version of Windows.

Hi @vedantmgoyal2009,
This package has an unusual error that might need more attention.


[Question]  How do I use this docker with an internal Website?

How do I use this docker with an internal website?  I am following this guide.  https://blog.ssdnodes.com/blog/installing-nextcloud-docker/
I want to setup using your letsencrypt module but for an internal website.   What do I need to change to run this for an internal webiste?
My full setup has some external websites and some internal websites.

LetsEncrypt can't be used for internal web servers without using public DNS challenge responses. This proxy only supports LetsEncrypt via HTTP challenge responses. To learn more about how these LetsEncrypt challenges work please see: https://letsencrypt.org/docs/challenge-types/


Update Start-HistoricalSearch.md

https://github.com/MicrosoftDocs/office-docs-powershell/issues/4187

Docs Build status updates of commit 7237c4e:
:clock10: Full build: the average full build time is 11 min(s) 47 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 7237c4e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/mail-flow/Start-HistoricalSearch.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 40f8d13:
:clock10: Full build: the average full build time is 11 min(s) 47 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 40f8d13:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/mail-flow/Start-HistoricalSearch.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit b147eef:
:clock10: Full build: the average full build time is 11 min(s) 47 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit b147eef:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/mail-flow/Start-HistoricalSearch.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 4cda612:
:clock10: Full build: the average full build time is 11 min(s) 47 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 4cda612:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/mail-flow/Start-HistoricalSearch.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


addGeneratedAnnotation should not be configured in usage section of README

When addGeneratedAnnotation is set to true, a timestamp is introduced in the generated sources. This makes it impossible to reproduce built artifacts byte-for-byte.
Below are two invocations of the https://github.com/deweyjose/graphqlcodegen-example project, with no changes being made between build invocations. Despite this, the built artifacts are different.

Fortunately, the default is false. However, the Usage section of this project's README has it configured to true. Users are likely to copy-and-paste this without understanding the consequences. That line of the documentation should be removed.
Additionally, the default when creating a project on https://start.spring.io is true. It would be great to have that not be the case. I can create a separate issue for that at spring-io/start.spring.io.
See also:

https://reproducible-builds.org/docs/definition/
https://reproducible-builds.org/docs/timestamps/

Hi @erichaagdev
Thanks for the issue. However I don't necessarily agree with you that @Generated should be the non default case. In my experience having @Generated enabled solves more problems than creates. What is your rational behind this opinion?

Hi @erichaagdev,
Please feel free to open an issue in Netflix DGS codegen regarding the ability to suppress or configure the optional date element of the Generated annotation. This behavior is not controlled by this plugin.

I don't feel strongly about the usage comment so I removed it for now since there really are no other opinions in usage about overriding defaults.

Thanks. I did create a PR here. https://github.com/spring-io/start.spring.io/pull/1533
And I see it's removed from the Usage section now. Thanks!

@erichaagdev Interesting but @Generated is really useful for other tooling. Another way to solve for reproducible builds is to not generate code that has the optional date element set.

Yes, I should have been more specific. The @Generated annotation itself is not a problem, it's the inconsistent output as a result of setting the date field. Another option is to not include the date field.

@erichaagdev - I think you'll pulling on an interesting thread. Ultimately Netflix DGS codegen folks control the @Generated annotation. I'll create an issue there and reference this one. If the date was removed we wouldn't need spring to change and setting addGeneratedAnnotation true doesn't impact reproducible builds.

https://github.com/Netflix/dgs-codegen/issues/702

That would be a good change. Assuming the default value is false then having only addGeneratedAnnotation as true would still yield reproducible sources. I can leave a comment on my Spring PR that this option is being considered.


Expand tab doesn't work

The expand tab option is set incorrectly.
https://github.com/TroubleC0re/nvim/blob/f468be1f6651440ced69edd02f8ed457fc64e19b/plugin/options.lua#L13
The expandtab option is local to each buffer and thus must be set with vim.bo instead of vim.opt

When I go back home I will fix it, thank you!

No problem! If you would like you could simply merge my pull request to make those changes.

Thanks!


[KeyVault] Update the LRO link that polls on recovery of deleted secrets

https://github.com/Azure/azure-sdk-for-net/issues/31581 for more information.

This doesn't seem to be an issue in the Python SDK since our recovery pollers across KV libraries don't use the resource version in polling requests.
Secrets: https://github.com/Azure/azure-sdk-for-python/blob/47133e7957fe731c69ca7bcb4299b45d7da5e62d/sdk/keyvault/azure-keyvault-secrets/azure/keyvault/secrets/_client.py#L459-L462
Keys: https://github.com/Azure/azure-sdk-for-python/blob/47133e7957fe731c69ca7bcb4299b45d7da5e62d/sdk/keyvault/azure-keyvault-keys/azure/keyvault/keys/_client.py#L552-L558
Certificates: https://github.com/Azure/azure-sdk-for-python/blob/47133e7957fe731c69ca7bcb4299b45d7da5e62d/sdk/keyvault/azure-keyvault-certificates/azure/keyvault/certificates/_client.py#L335-L338


[Bug]: Safari renders transparency as solid colour

With PDF.js v4, Safari is rendering solid colour where a PDF contains transparent images.
Firefox and Chrome both render the transparency correctly with PDF.js v4, and all browsers render the transparency correctly with PDF.js v2.
Web browser and its version
Safari 17.6
Operating system and its version
macOS 14.6.1
PDF.js version
v4.5.136
Is the bug present in the latest PDF.js version?
Yes
Is a browser extension
No
Steps to reproduce the problem
Open the attached PDF in the demo viewer.
drop-shadow.pdf
What is the expected behavior?
The drop shadow is rendered as partially transparent.
Chrome:

Firefox:

What went wrong?
The drop shadow is rendered as a solid block of colour.
Safari:

Link to a viewer
https://mozilla.github.io/pdf.js/legacy/web/viewer.html
Additional context
No response

With PDF.js v4, Safari is rendering solid colour where a PDF contains transparent images.
Firefox and Chrome both render the transparency correctly with PDF.js v4,

Given that it works in both Firefox and Chrome this unfortunately points to either a bug or a missing feature in the Safari browser itself, rather than an issue in the PDF.js library, hence this probably needs to be reported the Safari developers instead.
Please also note that we only list Safari as "mostly" supported in the FAQ, see https://github.com/mozilla/pdf.js/wiki/Frequently-Asked-Questions#faq-support

and all browsers render the transparency correctly with PDF.js v2.

The SoftMask handling in PDF.js has been re-factored in more times, to improve performance, so it's likely that we're now using functionality that Safari doesn't support properly.

I suspect it might be the issue described in this StackOverflow post, where the globalAlpha setting is not being recognised in combination with globalCompositeOperation. In fact, each of Chrome, Firefox and Safari render that example differently.
The closest WebKit bug I can find is 202459.

I suspect it might be the issue described in this StackOverflow post, where the globalAlpha setting is not being recognised in combination with globalCompositeOperation.
Here's a very minimal reproduction which shows Safari not rendering the opacity when globalCompositeOperation is set to destination-atop (or source-out, but most other values for globalCompositeOperation seem fine):
const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");
ctx.globalCompositeOperation = "destination-atop";
ctx.globalAlpha = 0.2;
ctx.beginPath();
ctx.rect(0, 0, 800, 800);
ctx.closePath();
ctx.fill();

The closest WebKit bug I can find is 202459.

Reported upstream to WebKit: https://bugs.webkit.org/show_bug.cgi?id=278454

Another instance of this reported https://app.frontapp.com/open/cnv_cdzi5fl?key=PoLfZs4reYIsidQhLZDtDa6CTzoh337T

@mlevans0 why are you commenting on a closed bug, which isn't finally related to pdf.js, and to share a non-public URL ?


(android) fix: respect requested location accuracy on Android 12+

Platforms affected
Android
Motivation and Context

Prior to Android 12, both COARSE and FINE location permissions were treated synonymously - requesting one would grant the other.
Therefore this plugin currently checks/requests both permissions on Android.
However, when an app is built with an SDK version (and running on device with) Android 12 (API 31) or higher, Android differentiates between precise vs approximate accuracy using these permissions (similar to iOS 14+) - see the Android documentation.
So currently when the methods of this plugin are invoked on Android 12+, this results in both both COARSE and FINE location permissions being requested, regardless of the value set for enableHighAccuracy in PositionOptions, resulting in the user being asked to choose between precise vs approximate location permission:

Description

This PR changes the Android implementation to respect the value of enableHighAccuracy passed in PositionOptions so if enableHighAccuracy: false then only COARSE location permission is requested, resulting in the user only being asked to grant approximate location permission:

If the user grants this permission and enableHighAccuracy: true is subsequently requested, the user in prompted to grant an upgrade from approximate to precise location permission:

If the app's first permission request is for enableHighAccuracy: true, then the user will be presented with the choice between approximate and precise location:

I believe this PR resolves issues #188 and #235.
One further change this PR makes is to remove the check for location permissions on Android before invoking clearWatch() since this is unnecessary.
Issues with approximate location permission on Android
Note: As @jcesarmobile points out in this comment, there is a bug in the Chromium WebView which is present up until API 32 (empirical testing shows it to manifest in every API from 22 to 31 but not in 32+ where it appears to be fixed).
It happens when COARSE but not FINE location permission is granted and enableHighAccuracy: false is requested, and results in a {"code":3,"message":"Timeout"} error when getCurrentLocation() or watchPosition() is called, rather than returning the current location.
Due to this issue, this PR implements API version-conditional functionality so that even when enableHighAccuracy: false is specified, both COARSE and FINE location permissions will still requested on API 31 and below, while on API 32 and above only COARSE permission will be requested.
A further complication is specific to devices running API 31: even if the user grants precise (both COARSE and FINE) location permissions, if  enableHighAccuracy: false is passed in the PositionOptions object to the W3C Geolocation API function on the original navigator.geolocation by this plugin, then this also results in a {"code":3,"message":"Timeout"} error.
To prevent this, this PR modifies the native getPermission method on Android to return the current API level of the device.
The web layer of the plugin www/android/geolocation.js then checks this returned value and if it's equal to 31, ensures enableHighAccuracy: true is passed to to the W3C Geolocation API function, even if enableHighAccuracy: false was specified by the app.
There is one further edge case: on Android 12+ (API 32+), if enableHighAccuracy: true is requested but the user grants only approximate (COARSE but not FINE) location permission, this will result in a {"code":1,"message":"Illegal Access"} error.
This is best handled by the app informing the user that precise location permission is required but they only granted approximate location permission.
The test harness app I created for this PR demonstrates this.
Testing

Ran the tests in cordova-paramedic with the same results between current plugin master branch and PR branch:

Created a test harness app to test the current vs modified plugin behaviour:

https://github.com/dpa99c/cordova-plugin-geolocation-test
Tested on Android emulators with APIs: 22, 29, 30, 31, 32, 33

To run the test harness app with this PR branch of the plugin:
git clone https://github.com/dpa99c/cordova-plugin-geolocation-test && cd cordova-plugin-geolocation-test
cordova platform add android@latest
cordova run android

To run the test harness app with the apache/master branch of the plugin:
git clone https://github.com/dpa99c/cordova-plugin-geolocation-test && cd cordova-plugin-geolocation-test
git checkout apache_master
cordova platform add android@latest
cordova run android

Checklist

[x] I've run the tests to see all new and existing tests pass
[x] I added automated test coverage as appropriate for this change (NOT APPLICABLE)
[x] Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
[x] If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
[x] I've updated the documentation if necessary (NOT APPLICABLE)

hi all!, waiting update

@breautek are we good to merge this?

@breautek Can I help here , because waiting that changes?

@dpa99c can you check your MR ?

@breautek I've made the requested changes - can you re-review and confirm you are happy to merge?

thanks a lot

My geolocation plugin working fine before 2 day suddenly my one plus nord and many client compalnt application location not detect. it was previously working fine
"cordova-plugin-geolocation": "^4.1.0",
"@ionic-native/geolocation": "^5.36.0",

@js-kulkarni , @mobiliseapplabllp I am also facing the same issue. Since few days suddenly it stopped working on few mobile phone and in few others it is working properly.  Lets stay connected if someone finds a solutions then lets update here. Thank you.

Guys, Any update pls ? Any help for this topic will be appreciated.

I am facing the same issues in VIVO, OPPO, and MI
I have enabled enableHighAccuracy: true and it will ask for 1 popup with both permissions If I choose Approximate then the code breaks.

Good news, seems like Android have some update on Play Service which got download on VivoPhone automatically. I did restart my phone and the GPS is working fine now which earlier was not working on Vivo.
Can anyone please confirm on their phone, where GPS was not working.


Windows wt shortcut in run, start menu, and search (run command) does not work

Windows Terminal version
1.11.3471.0
Windows build number
10.0.22000.466
Other Software
No response
Steps to reproduce

Right click on Windows start, click Windows Terminal or Windows Terminal (Admin)

Search, type 'wt' and run

Windows + R, write 'wt' and enter

Expected Behavior
It should open WindowsTerminal.exe
It works when typing 'wt' from command prompt or 'Windows Terminal' app from search, just does not work from what i said above
Actual Behavior
Does nothing, and not showing anything

Also when i run wt.exe and WindowsTerminal.exe from C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.11.3471.0_x64__8wekyb3d8bbwe, it does not work.
Tried reinstalling, repairing, and resetting does not work too

Update:
I tried downgrading to 1.10.210826003-release1.10, and wt.exe fully working in these all issues.
Probably i'll stay in these version and waiting for fix in next release

I tried downgrading to 1.10.210826003-release1.10, and wt.exe fully working and have no that all issues.
Probably i'll stay in these version and waiting for fix in next release

There's no reason that 1.11 would have broke this, so if 1.10 is working for you, I'd suspect that's just coincidence 😅.
Have you possibly changed the permissions on the WindowsApps directory at all? This sounds a lot like #12139 which was a dup of #9452

I tried downgrading to 1.10.210826003-release1.10, and wt.exe fully working and have no that all issues.
Probably i'll stay in these version and waiting for fix in next release

There's no reason that 1.11 would have broke this, so if 1.10 is working for you, I'd suspect that's just coincidence 😅.
Have you possibly changed the permissions on the WindowsApps directory at all? This sounds a lot like #12139 which was a dup of #9452

Perhaps yes, it may coincidence. I tried updating from 1.10.210826003-release1.10 to 1.11.3471.0 with github release msixbundle, and i didn't encounter that issues.
Sadly i can't check if it was actually permission issue or not, so i'll close this issue because the issues solved