[ZF3] Refactoring Service Manager

Hi,
This is another PR against develop. But those deleted files are still present in the PR :/.

This is better but is not enough:

changelog overwrite 2.6
composer. json is a mesh
test/bootstrap must be restored

All those changes (change log, removing test/bootstrap) were asked by Matthew actually. I feel lost u_u
Envoyé de mon iPhone

Le 21 juil. 2015 à 15:24, Maks3w<EMAIL_ADDRESS>a écrit :
This is better but is not enough:
changelog overwrite 2.6
composer. json is a mesh
test/bootstrap must be restored
—
Reply to this email directly or view it on GitHub.

@bakura10 tell me the reason because you  have removed each line. Put a note on every one. If you justify the 90 percent I invite you to 1 beer

@bakura10 Matthew never told you to remove test/bootstrap. He explain you why phpunit is failling and you can workaround the issue changing the phpunit.xml file.

Several comments regarding this implementation:
I don't like options as part of service locator interface. It implies that service exists, even if it is transparently created behind the scene.
This addition makes certain edge cases easier to work with but open way for multiple contradicting bug-prone behaviors.
Service locator implementation does not use existing service acting strictly as a factory when options are supplied. It is not something implied by contract and goes directly against service locator pattern.
Injecting configured service will have no effect on fetching service with options provided, resulting in unexpected behavior.
Factory implementation gets volatile due to need to support all possible options. Swapping factories have high chance of introducing subtle integration bugs. I believe options must be linked to concrete factory and defined along with it in service manager configuration. Assuming options are factory options, not service constructor options.
Maintenance load associated with using zend-servicemanager is increased due to multiple (possible) conflicting usage scenarios. Swapping factory as black box is no longer enough.

Why do you want all classes to implement __invoke

@Maks3w the __invoke allows to do interesting optimizations as we do not need to have multiple conditions (check if implement interface, then check if callable...). Now the creation is unified around the callable. You can see in the code how it has simplified a lot of things.

callable can still having callable typehint, just use the format [$object, 'method']

because callbable can be anything and not just FooInterface then does not have sense to enforce the implementation of the method.

You're right, the whole implementation could simply remove the interface. But I find it really useful to have it (implements the interface, and IDE automatically fill the missing methods with the correct signature, and parameter...).

Regarding get() with options, i suppose it would be cleaner to introduce new interface, that will define createWithOptions($serviceName, array $options) instead?
That way it can be implemented only where it is needed for edge cases, such as in validator plugin manager.

This is a bit what we have currently. As of today, there is an obscure interface called MutableOptionsInterface (defined in ZF 2.2 iirc), that you must implement in your factories. Then you can use a method called "getOptions" that will allow to retrieve you the options. But this only work for factories and is super confusing to use (will my proposal make it unified and work for factories, delegators...)

I'm with @Xerkus on options being a bad idea. If you want this then simply create a real factory class and register it as a service. So maybe we should drop MutableOptionsInterface instead. That would mean that ZF2 would have to migrate away from using it: but I guess that would be for the better.

Hi @roelvanduijnhoven , I'm not sure to understand the alternative you're suggesting. The options is meant to be used with get. Let me show you a simple example: you want a validator and give some options:
$validator = $validatorPluginManager->get(StringLength::class, ['min' => 5]);

I don't see how creating "real factory" would solve this issue. Are you suggesting adding one service per variant? (so having a StringLengthWithMin50, StringLengthWithMin50AndMax100, String...). This is clearly not manageable (and what about when you need to create them at runtime?).
You could argue that I could simply set the parameters using setters:
$validator = $validatorPluginManager->get(StringLength::class);
$validator->setMin(50);

But the issue is that if Validators become stateless, we'll likely want to inject parameters at construction and not being able to change them (and actually, that's the case for a lot of immutable services).
Honestly, I can understand that options is not the most beautiful way, but until someone suggests a simple solution that does not involve creating 50 factories for the same service, that definitely sounds the simplest and sanest idea.

By adding the $options argument you turn the service manager into a factory manager. You now decide at runtime what the requested services looks like. Defining what a service is should be the task of the service manager, right?
Secondly, there is a problem with semantics. The get call should be idempotent (or be renamed). Thus calling get('class', ['min' => 3]) twice would yield the same instance. However calling get('class', ['min' => 3]) and get('class', ['min' => 4]) would yield two different instances. This behaviour is strange and not understandable to the user.
If you want a factory for validators why not simply do it like so:
$v = $sl->get('ValidatorFactory')->build(StringLength::class, ['min' => 5])

But let us indeed turn to the problem. I am in no way an expert with the validator component as is, thus not sure if this is the current ZF behaviour. Storing multiple instances of StringLength with some other min and max values does not seem a good idea to me! But isn't this a problem of the validator component?

I definitely agree with all your points! Once again, this was really a solution I’m not 100% proud but that solve some use cases I already had.
However I definitely like the idea of the ValidatorFactory. If we can introduce that sort of factory to every component that need that, it should definitely solve the issue in a much more elegant way. Your point @weierophinney?

I think if you need to pass options when build then you are coupled to specific implementation.
Basically it makes a proxy to the constructor method.
In resume I don't see the difference between ask to the service manager to call a constructor with X arguments or instante the class dire using "new".
$serviceManager->get('Foo')->build(arguments) === new Foo(arguments)

After some thoughts, I'm convinced to remove the options. This will simplify the code even more and remove any confusion of services not cached because of options.
@Maks3w : what if Foo has required dependencies coming from the service locator AND additional options that need to be passed at creation? This is where the "new" approach does not really work.

But let us indeed turn to the problem. I am in no way an expert with the validator component as is, thus not sure if this is the current ZF behaviour. Storing multiple instances of StringLength with some other min and max values does not seem a good idea to me! But isn't this a problem of the validator component?

<snark>I just love how people chime in on things they admit to not understanding.</snark>
Please take the time, then, to understand the use cases and current behavior, before suggesting changes that invalidate these use cases.
Now, for some explanations and background.
The AbstractPluginManager was created to handle several specific use cases:

Creation of instances that are all of the same basic type (usually, instances of a specific interface).
In the case of filters and validators, due to the fact that their behavior differs based on the options they encompass, we wanted the ability to return discrete instances of the same type. Per the examples above, you may use the Between validator in multiple inputs of the same form, once with a configuration of min = 5, max = 10, another time with min = 100, max = 1000, etc; these will validate completely different sets of input, and, as such, need to be discrete instances.

As such, the AbstractPluginManager has a toggle to determine whether it returns the same instance or a new instance by default. Additionally, it allows passing $options as a second parameter to get() to configure the instance being retrieved, if the factory that will return the instance understands it. By default, all filters and validators accept an array of options to the constructor, so "invokables" will be passed those at instantiation; several factories provided also understand them. Others do not. The fact is: the $options passed are not a guarantee, but official plugins we provide support the concept in order to be internally consistent. Is it a good pattern? Maybe, maybe not, but it's practical, and can already support the idea of stateless validators, which is a goal for the future.
Related, the ServiceManager allows determining on a service-by-service basis whether the instance is shared or not; if not, a new instance is returned on each retrieval of that service. One specific example of where this facility is used is with events; EventManager instances are not shared, while the SharedEventManager is. This allows per-instance EM instances, each with the shared SEM instance. (Please, let's not debate the practice here; I'm using it only to illustrate how the service manager works.)
What I'm trying to say is that it's not as simple as this statement might appear:

By adding the $options argument you turn the service manager into a factory manager. You now decide at runtime what the requested services looks like. Defining what a service is should be the task of the service manager, right?

Or this one:

I think if you need to pass options when build then you are coupled to specific implementation.
Basically it makes a proxy to the constructor method.
In resume I don't see the difference between ask to the service manager to call a constructor with X arguments or instante the class using "new".
$serviceManager->get('Foo')->build(arguments) == new Foo(arguments)

(BTW, this latter is completely ignoring the fact that we may not just be passing the options directly to the constructor! In some cases, the options are processed by a factory in order to pull other services or determine which setters to call on the instance before returning it!)
The context in which a service manager exists matters. For plugins, being able to pass options makes a lot of sense, as multiple instances of a plugin, each with different state, are expected. Defining a service per instance type gets very complicated very quickly, if nothing else because of names!
Next, recall that not every plugin is an invokable! One big reason for using the AbstractPluginManager was to allow the same suite of options for plugin creation as we have for general service creation: factories, abstract factories, delegators, etc. This provides a ton of consistency to the framework: if it looks, acts, or smells like a plugin of some sort, then a plugin manager exists for it, and, since you know how to configure the service manager, you now also know how to configure new plugins for the component in question. It also simplifies consumers; they get the plugin manager and pull from it in a consistent way, without needing to know how it works.

Moving from plugins to the ServiceManager…
Unshared services are a desirable feature. Most IoC containers do support this idea, because there are services that you want to be able to inject into other services, but as multiple discrete instances, even if the initial configuration is the same. Is it necessarily a factory? Does it matter? IoC does not require that every retrieval of a given service returns the same instance.
As a new point: one of the goals @bakura10 has made is to reduce the number of factories developers need to build. Right now, because of the 1:1 relationship between a service and a factory (because factories do not know what service name was requested), we end up with a lot of factories that are essentially cut-and-paste code. Those factories are redundant, but also further slow the application due to I/O lookups and memory overhead (the single biggest factor in performance of ZF2 applications I've seen is number of objects; objects take up memory, but also mean filestat calls). My impression is that @bakura10 was using $options as one way to solve this problem.

So, with this background in mind, the question is: is there another way to accomplish the same goals, particularly with regards to having a clean, easy to accomplish, upgrade path for end-users?
This looks promising:

If you want a factory for validators why not simply do it like so:
$v = $sl->get('ValidatorFactory')->build(StringLength::class, ['min' => 5])

Considering that the majority of places that plugins are retrieved are inside other factories or in proxy/facade methods governed by the framework (e.g., Zend\View\Placeholder::plugin(), Zend\Mvc\Controller\AbstractController::plugin()), this would be a relatively easy migration. I'd actually suggest having such a factory use the get() method as well, as that will allow seamless migration in most cases:
$serverUrl = $services->get('HelperPluginManager')->get(ServerUrl::class, $options);

Note, however, that in order to facilitate the same spectrum of use cases that we currently have, this means that the AbstractPluginManager implementation needs to support the same features as the ServiceManager, which leaves us in exactly the position we are now!
I'll sum up:

A key feature of plugin managers is that they support the same creation mechanisms as service managers. This provides consistency within the framework, and simplifies learning (learn the system once, apply everywhere).
Plugin managers often need to support discrete instances based on the options provided at runtime. As such, they often act more like factories.
Adding $options to the application-level service manager may not be necessary or desired, but my understanding is that it's in the current incarnation in order to facilitate re-use for plugin managers and/or the ability to use the same factory for multiple services.

I appreciate the desire for theoretical purity, but we also have to be pragmatic. Having something re-usable in multiple contexts and easily understood are important goals as well. Additionally, as I've been pointing out everywhere lately, we must consider the migration impact of changes. Yes, we're talking about a new major version, but "break all the things!" is bad for adoption. If the migration will not be easily accomplished, people will not migrate.
Please read the above and understand the use cases. @bakura10 — you're doing great work.

@weierophinney Thanks for your comment, I think you have hit in one important point.  It's needed document the PRs with clear list of use cases.
My comment is related to the discussion thread along with the recent (BC Break) change in StaticValidator where are cases of constructor coupling.
I

Thanks @weierophinney for the lengthy explanation.

As such, the AbstractPluginManager has a toggle to determine whether it returns the same instance or a new instance by default. Additionally, it allows passing $options as a second parameter to get() to configure the instance being retrieved, if the factory that will return the instance understands it.

This has to be noted that as pointed before this can has the side-effect of NOT caching a service, even if a factory does not understand options (or, more accurately, does not use them, as factory interface now contains options in the signature to avoid the confusing MutableCreationOptions interface, as theorically now all factories understand options).
The basic rule is that once you give a non-empty option, caching will always be bypassed, no matter how the manager is configured.

Related, the ServiceManager allows determining on a service-by-service basis whether the instance is shared or not; if not, a new instance is returned on each retrieval of that service. One specific example of where this facility is used is with events;

This is not really true in my implementation: once options is given, it won't be cache, no matter how the service manager is configured.
So... are we good to go ? :) I'll try to find some time for writing documentation in addition of the changelog, but may that can be done after?

Any news on that @weierophinney ?

I'd hoped to start formal review and documentation yesterday, but to many
other tasks; should be able to start early next week, and get it merged to
develop.
On Aug 28, 2015 10:55 AM, "Michaël Gallego"<EMAIL_ADDRESS>wrote:

Any news on that @weierophinney https://github.com/weierophinney ?
—
Reply to this email directly or view it on GitHub
https://github.com/zendframework/zend-servicemanager/pull/8#issuecomment-135813403
.

This is not really true in my implementation: once options is given, it won't be cache, no matter how the service manager is configured.

This makes absolute sense to me. If you provide $options, the implicit agreement is you want a new instance, regardless of whether or not they are actually used. I consider that consistent behavior, as it sets up a consistent expectation.

I've gone and re-reviewed old comments and new ones, and have also been researching other implementations recently as we worked on zend-expressive. As such, I have a little bit of feedback to add at this point about options.
Options are typically useful only for plugin managers, and particularly when you allow or require new instances each time an instance is retrieved from it.
In many ways, it's analogous to what Aura.Di terms factories. This feature in Aura.Di has a separate method for instance retrieval, $container->newFactory('NameOfService'). When using that feature, you can pass items to the factory to pass on to the underlying constructor; they're passed as if you were calling ReflectionClass::newInstanceArgs(); in other words, sequentially. The pattern would work very well with our plugin managers.
To recap:

Have a separate "factory" feature for factories that create new instances and which can accept arguments/options to use during instance creation. This could be isolated to a plugin manager interface/implementation.

This could be as simple as passing the class name, and using its constructor.
Or allow passing any callable, and invoking the callable with the arguments provided at runtime.

Plugin managers would utilize this almost exclusively (if not completely exclusively). Instead of using the current get() method, we would use a factory() or similar method. This would pose a small challenge for people upgrading, but in most cases would be masked by the framework or established framework components (as the bulk of plugin retrieval/invocation is done via proxy methods).

@bakura10 Does the above sound good to you? It would resolve some of the issues raised on the thread (specifically reservations about having $options part of the get() signature), and create a clear separation between a standard container and a plugin manager.

@bakura10 Now that I've gone and reviewed the code directly, I'm in a quandary about the $options and plugin managers aspect again. If we take the route I suggested in my above comment, you would need to likely:

remove array $options = [] from the ServiceLocatorInterface.
remove the array $options = [] argument from all factories
potentially re-add a MutableOptionsInterface that AbstractPluginManager can implement.
alter AbstractPluginManager to remove implementation of ServiceLocatorInterface and/or implement any new interfaces; it could no longer extend ServiceManager.
create additional factory types that provide the array $options = [] argument

In other words, a lot of work, adding a lot more complexity (as you essentially have two systems to test now, one with options and one without).
My only real concern at this point with having $options in the interface and the implementation is conflicts with the container-interop interface. That said, I don't see that interface changing the get() signature, and adding optional arguments to an implementation and/or interface extension does not currently cause conflicts in any known PHP versions.
As such, I'm in favor of keeping $options in the main implementation. It keeps the code leaner, reduces complexity and relationships between the implementations, and will be more consistent in usage.
The only questions I have remaining are those I made in line comments today, specifically around container immutability. If you can address those, I can do a final review for merge.
Thanks, @bakura10 !!!!

and adding optional arguments to an implementation and/or interface extension does not currently cause conflicts in any known PHP versions.

It's something other containers do (https://gist.github.com/mnapoli/6159681), we saw that when discussing the ContainerInterface. Since, as you said, it's perfectly valid code to have more arguments in the implementation (it satisfies the interface), it was considered OK (the interface wouldn't have to contain it, but implementations were free to do it). So I'm just chiming in to +1 what you said.
Also want to point out another approach you mentioned (if I understood you correctly). Just like Aura, PHP-DI has a separate method (make()) for creating new instances with provided parameters (missing parameters are resolved by the container). It's defined in a separate interface (FactoryInterface) so that it can be type-hinted without being too coupled to the container itself. I'd love to see such an interface be added to container-interop at some point (and PSR eventually but we are still far from it): containers would be free to implement it or not, but containers that do (many) would at least have an interface for it. And since it's a different use case than get() IMO it's fine to have a separate interface/method.

I'd say that we should keep the same services and not clone them. That would make the withConfig extremely expensive is there are a lot of services.

Per a conversation with @bakura10 in IRC, get() does use resolveAlias($name), just at an earlier stage in the method not shown by the diff.


Not able to deploy a v2 app in a Project

On master-head - commit id: eb96d76c

on a HA setup, deploy Rancher backup app in Dashboard.
Select a project - System and enter S3 creds and deploy.
Error seen

Note:
Same issue is seen when CIS Benchmark v2 app is deployed
On clicking on "Install" again, it installs, with no error in the console

The same bug happens for monitoring v2
https://github.com/rancher/rancher/issues/29167

This behavior is seen with OPA Gatekeeper.
rancher\rancher:master-head version 2ae933e
Clicking install again will get past the error and the app is deployed.


Fix detail views not watching child components

Add subscribeStores calls to all relavent details

Add support for tracking overlapping subscribes as an optimization

Signed-off-by: Sebastian Malton<EMAIL_ADDRESS>fixes #4314

Doing a stress test, the dashboard UI eventually freezes (the cluster renderer process?) sorry for the long video.
Using kubectl, 200 namespaces are created, then they are removed in Lens. During removal the UI eventually freezes.
https://user-images.githubusercontent.com/40840436/143138138-47aca86d-af1d-40ce-82ed-6bb244db6f93.mov
This is relevant here because the dashboard UI doesn't freeze on master (the watch looks to be broken though) for the same test.
Another interesting fact is that the UI freeze happens on local clusters (tried minikube on mac, microk8s on linux) but everything is fine for applying the same test on spaces clusters.
Not sure how important this is for this PR...
some output (tail) from the main process:
[0] info:    ▪ [KUBE-API] watch (devcluster-1) retried /apis/entities.k8slens.dev/v1beta1/namespaces/default/devclusters?watch=1&resourceVersion=1522857 +167ms
[0] info:    ▪ [KUBE-API] watch (weblink-1) retried /apis/entities.k8slens.dev/v1/namespaces/default/weblinks?watch=1&resourceVersion=1522857 +3ms
[0] info:    ┏ [CLUSTER]: refresh +30s
[0] info:    ┃ [1] {
[0] info:    ┃ [2]   id: '87bac5783b9b6bb49d42137caf351757',
[0] info:    ┃ [3]   name: 'minikube',
[0] info:    ┃ [4]   ready: true,
[0] info:    ┃ [5]   online: true,
[0] info:    ┃ [6]   accessible: true,
[0] info:    ┃ [7]   disconnected: false
[0] info:    ┗ [8] }
[0]
[0] <--- Last few GCs --->
[0]
[0] [60096:0x3600000000]   880762 ms: Scavenge 3806.0 (3966.8) -> 3800.8 (3968.1) MB, 6.7 / 0.0 ms  (average mu = 0.910, current mu = 0.921) allocation failure
[0] [60096:0x3600000000]   880909 ms: Scavenge 3807.4 (3968.1) -> 3802.2 (3968.6) MB, 10.6 / 0.0 ms  (average mu = 0.910, current mu = 0.921) allocation failure
[0] [60096:0x3600000000]   881255 ms: Scavenge 3808.5 (3968.6) -> 3803.4 (3968.6) MB, 209.2 / 0.0 ms  (average mu = 0.910, current mu = 0.921) allocation failure
[0]
[0]
[0] <--- JS stacktrace --->
[0]
[0] info:    ┏ [SHELL-SESSION]: websocket for 87bac5783b9b6bb49d42137caf351757:7cc19f97-d4da-4adb-a096-f7cb2a76414a closed with code=GoingAway(1001) +13s
[0] info:    ┃ [ 1] {
[0] info:    ┃ [ 2]   cluster: {
[0] info:    ┃ [ 3]     id: '87bac5783b9b6bb49d42137caf351757',
[0] info:    ┃ [ 4]     name: 'minikube',
[0] info:    ┃ [ 5]     ready: true,
[0] info:    ┃ [ 6]     online: true,
[0] info:    ┃ [ 7]     accessible: true,
[0] info:    ┃ [ 8]     disconnected: false
[0] info:    ┃ [ 9]   }
[0] info:    ┗ [10] }
[0] error:   ┏ [IPC]: failed to send IPC message "extensions@f3691b58df3ee1ce8cfc5e7d2dc88221b297786eed2f772bf44d140b16f25de4:spaces" to view "remote=5"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +9s
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "extensions@f3691b58df3ee1ce8cfc5e7d2dc88221b297786eed2f772bf44d140b16f25de4:spaces" to view "window=1"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "catalog:items" to view "remote=5"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1s
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "catalog:items" to view "window=1"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +3ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "catalog:items" to view "remote=5"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1s
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "catalog:items" to view "window=1"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] info:    ┏ [CLUSTER]: refresh +6s
[0] info:    ┃ [1] {
[0] info:    ┃ [2]   id: '87bac5783b9b6bb49d42137caf351757',
[0] info:    ┃ [3]   name: 'minikube',
[0] info:    ┃ [4]   ready: true,
[0] info:    ┃ [5]   online: true,
[0] info:    ┃ [6]   accessible: true,
[0] info:    ┃ [7]   disconnected: false
[0] info:    ┗ [8] }
[0] error:   ┏ [IPC]: failed to send IPC message "cluster:state" to view "remote=5"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +78ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "cluster:state" to view "window=1"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "catalog:items" to view "remote=5"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +7ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "catalog:items" to view "window=1"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "cluster:state" to view "remote=5"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +14ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "cluster:state" to view "window=1"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "cluster:state" to view "remote=5"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "cluster:state" to view "window=1"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +0ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "catalog:items" to view "remote=5"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +1s
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] error:   ┏ [IPC]: failed to send IPC message "catalog:items" to view "window=1"'s subframe "frameInfo.processId"=10 "frameInfo.frameId"=5 +0ms
[0] error:   ┗ [1] { error: 'Error: Render frame was disposed before WebFrameMain could be accessed' }
[0] info:    ┏ [CLUSTER]: refresh +29s
[0] info:    ┃ [1] {
[0] info:    ┃ [2]   id: '87bac5783b9b6bb49d42137caf351757',
[0] info:    ┃ [3]   name: 'minikube',
[0] info:    ┃ [4]   ready: true,
[0] info:    ┃ [5]   online: true,
[0] info:    ┃ [6]   accessible: true,
[0] info:    ┃ [7]   disconnected: false
[0] info:    ┗ [8] }

Thanks for the stress test. I think I missed checking if the stores' API resource is cluster scoped. Therefore abort less often.
Also I think that I should hoist the cluster scope check higher up so that we don't abort and restart in this case.


Don't bail, but only warn if using --no-flash with defmt.

This PR makes probe-run not bail when a user uses defmt together with the --no-flash cli-option, but only emits a warning.
This allows for more felxibility during development, but should only be used when you know what you are doing.
Fixes #298.

bors r+


Introduce CHANGELOG.md

What changed?
Introducing changelog based on Keep a Changelog format.
Collected changes from previous 0.18.5 release.

Why?

How did you test it?

Potential risks

Pull Request Test Coverage Report for Build d5acf1ec-c84f-4091-a2cb-cc5825a869ee

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 63.9%

Totals

Change from base Build 27a5a12e-f930-4df7-a1ba-389c8c4dbb1d:
0.0%

Covered Lines:
12373

Relevant Lines:
19363

💛  - Coveralls

bleh. Sorry about the delay - this looks good! Lets do it.
There are some kinda-easy ways to mechanically transform these kinds of changelogs to mark a version release, think we should add a makefile target for it? Assuming we can find a script that doesn't look to bad, anyway.

I'm not against it, but marking the version isn't that hard to do manually either. IMHO we can live without it for now.


Starting point

See the following:

https://github.com/algotech/angular-boilerplate/issues/1
https://github.com/diegonetto/generator-ionic
https://github.com/thaiat/generator-angular-famous-ionic

https://github.com/loicknuchel/ionic-starter
https://github.com/apompa/ionic-ng-boilerplate


MQTT

It's been suggested the cloud connection for the grill is via MQTT. This guy says he's been successful hijacking DNS to have his devices connect to a local MQTT server instead of the vendor's instance.
If I get on the firewall and sniff the grill's MAC address, I can see the periodic broadcasts to udp:6667 and when I fiddle in the app, I see additional traffic on tcp:8886. Perhaps that's MQTT. I wonder if we could do something similar to get async comm to the grill instead of relying on polling.
# tcpdump -n -i lag0 ether host ec:fa:bc:6a:d1:0e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lag0, link-type EN10MB (Ethernet), capture size 65535 bytes
12:36:04.068514 IP <IP_ADDRESS>.49154 > <IP_ADDRESS>.6667: UDP, length 188
12:36:05.276415 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 46
12:36:09.068083 IP <IP_ADDRESS>.49154 > <IP_ADDRESS>.6667: UDP, length 188
12:36:12.451077 IP <IP_ADDRESS>.8886 > <IP_ADDRESS>.44017: Flags [P.], seq 676013862:676014075, ack<PHONE_NUMBER>, win 65535, length 213
12:36:12.595263 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [.], ack 213, win 3615, length 0
12:36:12.656234 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [P.], seq 1:214, ack 213, win 3615, length 213
12:36:12.739453 IP <IP_ADDRESS>.8886 > <IP_ADDRESS>.44017: Flags [P.], seq 213:282, ack 214, win 65535, length 69
12:36:12.842436 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [.], ack 282, win 3546, length 0
12:36:13.967322 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [P.], seq 214:427, ack 282, win 3546, length 213
12:36:14.049280 IP <IP_ADDRESS>.8886 > <IP_ADDRESS>.44017: Flags [P.], seq 282:351, ack 427, win 65535, length 69
12:36:14.080555 IP <IP_ADDRESS>.49154 > <IP_ADDRESS>.6667: UDP, length 188
12:36:14.092430 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [.], ack 351, win 3477, length 0
12:36:15.298710 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 46
12:36:16.154205 IP <IP_ADDRESS>.8886 > <IP_ADDRESS>.44017: Flags [P.], seq 351:564, ack 427, win 65535, length 213
12:36:16.336150 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [P.], seq 427:640, ack 564, win 3264, length 213
12:36:16.421209 IP <IP_ADDRESS>.8886 > <IP_ADDRESS>.44017: Flags [P.], seq 564:633, ack 640, win 65535, length 69
12:36:16.595469 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [.], ack 633, win 3195, length 0
12:36:17.999128 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [P.], seq 640:853, ack 633, win 3195, length 213
12:36:18.082948 IP <IP_ADDRESS>.8886 > <IP_ADDRESS>.44017: Flags [P.], seq 633:702, ack 853, win 65535, length 69
12:36:18.092447 IP <IP_ADDRESS>.44017 > <IP_ADDRESS>.8886: Flags [.], ack 702, win 3126, length 0
12:36:19.067961 IP <IP_ADDRESS>.49154 > <IP_ADDRESS>.6667: UDP, length 188
12:36:21.155307 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
12:36:21.445076 ARP, Reply <IP_ADDRESS> is-at ec:fa:bc:6a:d1:0e, length 46

I've got a packet capture from 2020-10 where I joined the grill to my AP and sent some various commands. It's definitely MQTT. I did some reading about it back then and probably forgot most. Here are some of my notes.
The grill made some initial DNS requests for a.gw.tuyaus.com and received 3 different IP addresses during the capture. The MQTT broker seems to be mq.gw.tuyaus.com. I get a not authorized error from that URL with MQTT Explorer when attempting.
MQTT seems to work with topics and has commands to interact like subscribe and publish. The topic broken out by wireshark looks like this: p1234567/mb/az12345678901234AbCd2. Based on what I found in the kDefaultTuyaUserV2 key of the plist I extracted, it appears to be partnerIdentity/mb/uid.

Dug into the capture some more and found what seems to be a key exchange from the grill in HTTP prior to the MQTT traffic.
Request query string to a.tuyaus.com/gw.json looks like this:
a=s.gw.token.get&gwId=xxx123xxx123&other={"token":"1abcdEFg","region":"AZ","tlinkStat":{"configure":"smartconfig","time":4,"source":"station_ap","path":"multicast"}}&t=9&v=3.0&sign=abc999abc999

Then comes another HTTP POST to a.tuyaus.com/gw.json. The gsId and token values are the same between this and the last exchange. The sign value changed.
Looks like this:
a=s.gw.dev.pk.active&gwId=xxx123xxx123&other={"token":"1abcdEFg"}&t=1600000000&v=3.0&sign=abc456abc456
Response to that one in JSON. Interesting note is the localKey in this response does not match the localKey I pulled from my iPhone. The uid is the same value as above in the MQTT topic.
"result":{"schema":"[{\"mode\":\"rw\",\"property\":{\"type\":\"bool\"},\"id\":1,\"type\":\"obj\"},{\"mode\":\"rw\",\"property\":{\"min\":0,\"max\":65535,\"scale\":0,\"step\":5,\"type\":\"value\"},\"id\":102,\"type\":\"obj\"},{\"mode\":\"ro\",\"property\":{\"min\":0,\"max\":65535,\"scale\":0,\"step\":1,\"type\":\"value\"},\"id\":103,\"type\":\"obj\"},{\"mode\":\"rw\",\"property\":{\"min\":0,\"max\":255,\"scale\":0,\"step\":1,\"type\":\"value\"},\"id\":104,\"type\":\"obj\"},{\"mode\":\"ro\",\"property\":{\"min\":0,\"max\":65535,\"scale\":0,\"step\":1,\"type\":\"value\"},\"id\":105,\"type\":\"obj\"},{\"mode\":\"ro\",\"property\":{\"min\":0,\"max\":65535,\"scale\":0,\"step\":1,\"type\":\"value\"},\"id\":106,\"type\":\"obj\"},{\"mode\":\"rw\",\"property\":{\"min\":-128,\"max\":127,\"scale\":0,\"step\":1,\"type\":\"value\"},\"id\":107,\"type\":\"obj\"},{\"mode\":\"wr\",\"property\":{\"type\":\"bool\"},\"id\":108,\"type\":\"obj\"},{\"mode\":\"ro\",\"property\":{\"type\":\"bool\"},\"id\":109,\"type\":\"obj\"},{\"mode\":\"ro\",\"property\":{\"type\":\"bool\"},\"id\":110,\"type\":\"obj\"},{\"mode\":\"ro\",\"property\":{\"type\":\"bool\"},\"id\":111,\"type\":\"obj\"}]","devEtag":"00000001ab","uid":"az12345678901234AbCd2","secKey":"zzzseckeyzzz","schemaId":"00000003f4","localKey":"zzzlocalkeyzz"}
In between the request and response in this comment, some tuya MQTT server sends a publish Message with the topic I mentioned in an earlier comment. The JSON response just above follows immediately after.

Edit in above: Correction on device. My notes had wrong IP addresses. My iPhone initiated all this traffic above. The grill didn't come in until later. Unfortunately, it looks like my iPhone and the grill communicated directly over wireless, so it didn't pass through my AP interface to capture in my packets. Seems the iPhone chats with the grill to get the important info, then negotiates with Tuya servers from the app. Once negotiated, it seems like the iPhone tells the grill to then query the mq.gw.tuysus.com domain with a connect and then subscribe MQTT message. Maybe not enough CPU in the grill controller than process all the key exchange stuff?
Lots of info to unpack here, but I think in short it would require a lot more reversing of the key exchange to get data from the MQTT broker. That or perhaps someone else who has experience with the Tuya API knows this is a more common exchange prior to MQTT.
On the plus side, I'm happy to see layers of security applied to what seems to be an otherwise "insecure" IoT protocol.

Looks promising. Making notes on investigations like this public is appreciated. Storage is cheap (free here) so may as well us it, right? I appreciate the effort and sharing.

If anyone can get the android app apk file, we could do a decompile of it and possibly find some of the routines for this process. Chances are pretty good it is implemented the same on both platforms.


"warehouse init config.py" failing when setting up

I'm not sure if this is a README bug, a missing dependency, a user error or an actual bug, but after using "mkvirtualenv -p /usr/bin/python3 warehouse" to create a virtualenv and running "pip install -r requirements.txt", I get the following when running "warehouse init config.py":
Traceback (most recent call last):
  File "/home/ncoghlan/.virtualenvs/warehouse/bin/warehouse", line 9, in <module>
    load_entry_point('warehouse==0.1dev1', 'console_scripts', 'warehouse')()
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python3.4/site-packages/pkg_resources.py", line 356, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python3.4/site-packages/pkg_resources.py", line 2431, in load_entry_point
    return ep.load()
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python3.4/site-packages/pkg_resources.py", line 2147, in load
    ['__name__'])
  File "/home/ncoghlan/devel/warehouse/warehouse/__main__.py", line 15, in <module>
    from configurations import management
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python3.4/site-packages/configurations/__init__.py", line 2, in <module>
    from .base import Settings, Configuration
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python3.4/site-packages/configurations/base.py", line 7, in <module>
    from .utils import uppercase_attributes
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python3.4/site-packages/configurations/utils.py", line 5, in <module>
    from django.utils.importlib import import_module
ImportError: No module named 'django.utils.importlib'

I initially tried Python 2, and that failed in the same way (just with a misleading exception message that initially made me think it wanted Python 3's importlib module):
Traceback (most recent call last):
  File "/home/ncoghlan/.virtualenvs/warehouse/bin/warehouse", line 9, in <module>
    load_entry_point('warehouse==0.1dev1', 'console_scripts', 'warehouse')()
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python2.7/site-packages/pkg_resources.py", line 356, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python2.7/site-packages/pkg_resources.py", line 2431, in load_entry_point
    return ep.load()
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python2.7/site-packages/pkg_resources.py", line 2147, in load
    ['__name__'])
  File "/home/ncoghlan/devel/warehouse/warehouse/__main__.py", line 15, in <module>
    from configurations import management
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python2.7/site-packages/configurations/__init__.py", line 2, in <module>
    from .base import Settings, Configuration
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python2.7/site-packages/configurations/base.py", line 7, in <module>
    from .utils import uppercase_attributes
  File "/home/ncoghlan/.virtualenvs/warehouse/lib/python2.7/site-packages/configurations/utils.py", line 5, in <module>
    from django.utils.importlib import import_module
ImportError: No module named importlib

I think you're installing Warehouse from PyPI which is a really old iteration of it or your branch is really old. Particularly confusing is there is no requirements.txt in the root directory anymore nor is there a warehouse init command anymore.

Generally the supported way to run an environment now adays is via Docker. You can run all the pieces on your own if you want, but it's easier to just do docker-compose build && docker-compose up. See: https://warehouse.pypa.io/en/latest/development/getting-started/

Note, that if you have an old clone of Warehouse, the new master branch is a wholly distinct orphan branch from the previous branches so you'd probably want to just blow away the old repository.

Oh, I bet what happened is I had an old fork on GitHub, so even though the local clone was new, the fork was old.

Yep, my fork is ancient, I just forgot about that because I'd only previously used it for online edits. Sorry for the noise.


Packaging a FeathersJS App

Can I add a gulp pipe to the flow of your packaging?

Hi! I did not manage to reproduce. Please post your pkg command line and output.

Can I add a gulp pipe to the flow of your packaging

You can create some gulp wrapper for pkg and use it.

Any example I could look at? thank you

No example. Just an idea. Think of it as running a custom app from gulpfile.


Rename service name to match enum

backend -> backendservices
Fix relative paths to be from .sample-metadata.json's location

ping @gngeorgiev

Please move the contents of the template/backend folder to the respective template/backendservices folder.

:+1:


MinGW failures - due to symlink

@eregon
Good day,  as you know, I'm the crazy MinGW guy.  At present, using make test-spec has two failures.   Running mspec has no failures.
I believe you touched on the issue of varying build systems in a ruby-lang issue.  Well, here's another.  I've mostly been building 2.5 trunk, but for various reasons, I've built other versions.  So, I've got a git repo of ruby, and that is the source for my build system.
Since my system is loosely based on the MSYS2 / MinGW packaging system, I have a symlink for src within the standard build system folders to the ruby git repo folder.
Hence, the two failures, where realdirpath and realpath do not match __FILE__.
Would you accept a patch that allows the two tests to determine if __FILE__ is a symlink?
A method like the following does so:
fn = 'E:/GitHub/ruby-loco/src/ruby/spec/rubyspec/core/file/realdirpath_spec.rb'

real = File.realdirpath(fn)
#real = File.realpath(fn)

def symlink_in_path?(fn)
  fn_strip = fn.dup
  found_symlink = false
  while fn_temp = File.dirname(fn_strip) do
    found_symlink = File.symlink?(fn_temp)
    break if found_symlink || fn_temp == fn_strip  # reached last dir or drive
    fn_strip = fn_temp
  end
  found_symlink
end

if real != fn
  puts symlink_in_path?(fn)
end

On Windows, one has to walk up the path to check for whether File.symlink? is true...

Yes, of course any improvement ro ruby/spec is welcome, especially if it fixes the specs on some platform/setup.
I am unsure, does File.symlink? and File.realpath work on Windows at all? Or just on mingw builds?
I think an easy way to solve this is to not use __FILE__ but a simple file in rubyspec_temp, like the spec below, except the file would exist:
https://github.com/ruby/spec/blob/ec1c161a8341a1ceb4075484e8a630be5fecada1/core/file/realdirpath_spec.rb#L97
But, if symlinks work on mingw, we should also test them similarly to how we do on other platforms.
Do you know the semantics of symlink-related Ruby methods on Windows? Is it documented somewhere?

@eregon
Getting a bit ahead of things.

I am unsure, does File.symlink? and File.realpath work on Windows at all? Or just on mingw builds?

I'll check, but it will be a bit.  I've got a mswin build from unak / mswin-build, I'll check with that.  I suspect both work.
On Windows, I believe symlink? and real* work fine,  The issue is creating symlinks, although I haven't checked all types.  On Windows, creating some types of symlinks require elevated privileges, which are enforced on most user systems, but can be disabled on servers.
So, I don't think there's anyway to test symlink? and real*, since there's no way for the test system to create a symlink.
Hence, I think using __FILE__ is fine, but a temp file would be easier, since we'd know that real* would be equal to the regular name.

https://github.com/ruby/ruby/commit/6f199eb803e2b3c75521a0aac95909d86923a916 should fix both failures (it's committed in ruby/ruby).
So essentially File.symlink raises an error on Windows user systems? What exception is raised?

@eregon
irb(main):001:0> File.symlink("E:/temp", "E:/Temp2")
Errno::EACCES: Permission denied @ rb_file_s_symlink - (E:/temp, E:/Temp2)
        from (irb):1:in `symlink'
        from (irb):1
        from C:/ruby25_64/bin/irb.cmd:19:in `<main>'

@eregon
Just checked the commit, both tests pass (running none parallel).
My solution was a bit complicated.  I guess I really don't like temp files in test code.  Problems in readline tests in test-all also...
Feel free to close.


changing .elixir_ls path

Environment

Elixir & Erlang versions (elixir --version): Elixir 1.13.4 (compiled with Erlang/OTP 25)
Elixir Language Server version: 0.11.0
Operating system: Gentoo Linux
Editor or IDE name (e.g. Emacs/VSCode): Neovim 0.7.2
Editor Plugin/LSP Client name and version: nvim-lspconfig and mason.nvim (elixir_ls)

Current behavior
When launching the Language Server, the .elixir_ls directory gets created in my $HOME.
Is there an environment variable or something that can change the path where this directory gets created?
Can it be disabled altogether?

How are you launching it?

It gets launched by nvim-lspconfig and mason-lspconfig.
From what I set up, following the guide, the executable just gets called

Can you post a detailed trace? I think your client is sending rootUri/rootPath/workspaceFolders pointing to your $HOME and elixir-ls is convinced that it opens a project that needs to be compiled and it needs.elixir_ls dir for that. I cannot reproduce it in vscode - when I open a project dir it creates .elixir_ls in that project dir. When I open a single file (rootUri is null) then .elixir-ls is not created at all.

Is there an environment variable or something that can change the path where this directory gets created?

No. How would you make it work? It needs to be relative to the open dir

Can it be disabled altogether?

I don't think we can do that

Running :LspLog yields:
[START][2022-10-06 17:00:21] LSP logging initiated
[WARN][2022-10-06 17:00:21] .../lua/vim/lsp.lua:1072	"server_request: no handler found for"	"workspace/diagnostic/refresh"

I also receive this message:
LSP[elixir][Info] Did not receive workspace/didChangeConfiguration notification after 5 seconds. Using default settings.

This looks like a configuration issue on your end. Please use forum for help or post a reproduction steps.


Passing location as a number

Library is working great, Thanks for all the effort.
Can't get onPress a map cluster to work on android, had to cast the location parameters numbers for it to work. And use a custom forked repo for it to work.
Notes:

It works just fine on ios with or without my modification.
Error message will be attached before modifications (on Android).
On apple maps when I zoom out far enough rendered clusters/pins disappear. (seems to work just fine on android google maps)
https://user-images.githubusercontent.com/42909451/52913939-6d005900-32cb-11e9-9ce6-18f9fbe86014.png

@HazemEdaily are you sure that you haven't substitute the onClusterPress behaviour. For me, Android onClusterPress are working perfectly. Maybe react-native-maps: 0.23.0 solves this issues (we are updating the components maps dependency)

@HazemEdaily I don't think this component is the right place where to cast string coordinates to number coordinates. At least It is not in general, we will consider adding a prop for forcing coordinates casting component-side.


fix: supporting unauthenticated version endpoint

Closes #315.

@sybnex I tested this personally and seems fixing what you're looking for.
$: curl -k https://<IP_ADDRESS>:9001/version
{
  "major": "1",
  "minor": "25",
  "gitVersion": "v1.25.3",
  "gitCommit": "434bfd82814af038ad94d62ebe59b133fcb50506",
  "gitTreeState": "clean",
  "buildDate": "2022-10-25T19:35:11Z",
  "goVersion": "go1.19.2",
  "compiler": "gc",
  "platform": "linux/amd64"
}

For the /version endpoint we're skipping the error in case of retrieval of authentication data, bypassing the user impersonation and decorating the request with the bearer token which would be required to perform the necessary user impersonation.
I wanted to give it a try also if we support a protected endpoint, and it seems to do the trick:
$: curl -k -H 'Authorization: Bearer <REDACTED>' https://<IP_ADDRESS>:9001/version
{
  "major": "1",
  "minor": "25",
  "gitVersion": "v1.25.3",
  "gitCommit": "434bfd82814af038ad94d62ebe59b133fcb50506",
  "gitTreeState": "clean",
  "buildDate": "2022-10-25T19:35:11Z",
  "goVersion": "go1.19.2",
  "compiler": "gc",
  "platform": "linux/amd64"
}

This would be required for those Kubernetes distributions which are protecting the version endpoint, thus a valid token, and a user along with groups, must be enforced.
It would be great if you could give it a try, ensuring it works as expected in your environment.

Hi,
thanks for your big effort! The part, you developed, with the version works as designed!
Unfortunately it didn't solved the problem. I increased all debug logs to the max but couldn't find any hint what's the problem with the dashboard. It is still not starting and hangs in a CrashLoopBackOff.
I even tried to create an own tenant for the dashboard with his serviceaccount as owner but wasn't successfully.
Last thing would be sniffing the network ...
Is there any change to increase the logs of the capsule-proxy by more than 10?

Unless sharing more details, such the reason for the crashloopbackoff, dashboard logs, and further details, I cannot help you.
I would suggest moving the discussion to the issue itself, since this place must be used mostly to address changes and technical discussions.

Stale, Reopen on activity


gulp protractor: Error: Server terminated early with status 1

When I run gulp protractor I get :
Error: Server terminated early with status 1

This seems to be a problem with the version of selenium webdriver that gulp-protractor installs.
https://github.com/SeleniumHQ/selenium/issues/1191
https://github.com/Swiip/generator-gulp-angular/issues/853
https://github.com/42Zavattas/generator-bangular/issues/31

now seems to work


Tubehax?

I've recently started using tubehax and I attempted to install homebrew browser but its not showing up. Am I installing it improperly? or is homebrew browser not supported on tubehax atm?

nvm I download the source files instead of the correct files -_(\

Oh? I'm glad to hear it's working; newer versions of ninjhax have been breaking it recently. I'll do some tests later today if I get a chance. Thanks for the heads up anyway. :)


Add CodeQL analysis for Java and JavaScript code

The code analysis didn't show anything useful. Won't merge for now!


Display Frame Number

In the player we currently see time as MM:SS. We would like to be able to see the frames as well. E.g MM:SS:FF.
Is this even possible?

did you mean like the FPS?

@leandromoreira: I mean fractional seconds converted into frame numbers, e.g. 0.4 @ 25fps = 00:00:01

@leandromoreira - This should be implemented using a timer and using the normal Clappr events for playback to know when to suspend/resume the timer. You start the timer and then you read the current timestamp from the video - with this and knowing the framerate(you need it, it cannot be guessed from the video easily, for example in HLS streams you cannot, so to have a generic solution, find a way to pass this from outside) you can roughly estimate the current frame number. But it is not absolutely precise(no SLA, but can be precise enough for your use case) due format limitations of current browsers supported media types.
I do not think this should be part of Clappr Core, unknown framerate is really a show stopper. Start  a plugin and I will gladly contribute @ronag

@iongion I'm doing a stats plugin for clappr which will inform the FPS by X seconds, is that helpful?

@leandromoreira allow milliseconds update rate - seconds is too much in many cases

@leandromoreira I saw you try to perform some framerate detection, please also allow it to be passed from outside for a better more imperative control, especially in the m3u8 playback

@iongion you're right, thanks!

I saw you try to perform some framerate detection, please also allow it to be passed from outside for a better more imperative control, especially in the m3u8 playback

I couldn't understand =/ can you provide an example?

@leandromoreira - _fetchExtras at line 120, while it works, if browsers support it, it would be nice to allow users to pass the frame rate as configuration option to the plugin and use that instead of the detection's - in this case you would also cover the cases when frame rate cannot be detected. Because there are a lot of of stats that can be computed on the frame-rate, a better control would be awesome, and is a good design choice too, no ? :+1:

Yes, how about something more "dynamic"?
{
 overrideMetrics: function(metrics) {
     metrics.counters.fps = 10;
 }
}

I think this could let users to update certain values instead of passing it statically at initialization, what do you think?

@leandromoreira absolutely much better :dancer:

done https://github.com/leandromoreira/clappr-stats/blob/master/index.js#L119 feedbacks are welcome

@iongion I'm thinking now that both functions can offer you the same final result, ins't?  Maybe, we should have only one.
        onReport: (metrics) => {console.log(metrics)}, //not mandatory: callback function
        overrideMetrics: (metrics) => {metrics.counter.fps = 30}

Any solution to this? - I want to show the elapsed time in greater granularity, e.g. currently it shows seconds, e.g. 00:02 / 00:30 - but what I want is 00:02.78 / 00:30.12. Anyone knows how to achieve this?


Look into upgrading gradle wrapper to version 5

On quick test, it looks like this will require upgrades of various dependencies.

looks like this shaves a few seconds off the build (without any other optimizations).

Alright if I take a look at this? Seems like a decent way to learn a bit more about Gradle.


AttributeError: 'RoutingModel' object has no attribute 'SetCumulVarSoftLowerBound'

Is SetCumulVarSoftLowerBound no longer supported in python? I am using 6.10.6025. The similar method for UpperBound works fine.
My use case is to try and shorten the time between a pickup and delivery node, as much as possible.

This method seems still here on master.
%make python
...
%grep --color -Hn "def SetCumulVarSoftLowerBound" ortools/gen/ortools/constraint_solver/pywrapcp.py
ortools/gen/ortools/constraint_solver/pywrapcp.py:3660:    def SetCumulVarSoftLowerBound(self, index: 'int64', lower_bound: 'int64', coefficient: 'int64') -> "void":


A11y: Dropdowns lose accessibility focus after selecting an item

b/142427914
expected: When you select a DropdownMenuItem and the popup closes, accessibility focus should return to the DropdownButton.
actual: On Android, focus gets lost, and then swiping right brings you to the first element on the page. On iOS, focus goes to the first element on the page.
Repro copied from https://github.com/flutter/flutter/issues/40101
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  String value = 'foo';

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: Scaffold(
        body: SafeArea(
          child: Center(
            child: Row(
              mainAxisAlignment: MainAxisAlignment.center,
              mainAxisSize: MainAxisSize.min,
              children: <Widget>[
                DropdownButton(
                  value: value,
                  items: [
                    DropdownMenuItem(
                      value: 'foo',
                      child: Text('Foo'),
                    ),
                    DropdownMenuItem(
                      value: 'bar',
                      child: Text('Bar'),
                    ),
                    DropdownMenuItem(
                      value: 'baz',
                      child: Text('Baz'),
                    ),
                  ],
                  onChanged: (String value) {
                    setState(() {
                      this.value = value;
                    });
                  },
                ),
                MaterialButton(
                  child: Text('Reset'),
                  onPressed: () {
                    setState(() {
                      this.value = 'foo';
                    });
                  },
                ),
              ],
            ),
          ),
        ),
      ),
    );
  }
}

Changing severity to S1 since we have a page in setup (the privacy settings page) that has many dropdowns and this issue makes getting through it a O(n^2) effort. It has been highlighted as a serious issue in accessibility testing.

It's unlikely that there will be any progress on this issue before the new year and before critical a11y issue #46625 is resolved.

Sorry, I meant to update this before the holiday. I have spent a fair amount of time looking into this, but I am not sure what we are doing differently than regular Android apps in this regard. We appear to be providing all the necessary a11y info with the appropriate flags set, but a11y focus doesn't appear to go anywhere when the dropdown is dismissed. I will see if I can get some help from the TalkBack team. At least they might be able to let us know what it is keying off of when it selects a new focused item.

@zanderso Do you know if @cbracken plans on working on this when he's back? If not, do you have a delegate in mind?

I don't think @cbracken will be working on this when he's back, no. I'm working on finding someone to pick this up.

I had to solve this issue and a related issue with pickers in a recent project. Did it by making two small hacks at the engine:

Added a new event handler in a11y bridge that allowed to request it to set a11y focus to a specific element, and

Changed accessibility bridge so that it looks through and sets the initial focus to an element that has the focus flag set (I know it's meant for input focus, and a11y focus is a different thing, but I didn't want to hack around too much)

Ended up using the second only and ditching the first one, though I'm unsure if even that was ever taken all the way to production (local engine is a pain)
Only did it for iOS and am not comfortable enough with the Engine codebase to make an actual pull request out of it, however if someone is to work on this it might be good to allow specifying the element where the focus initially goes when a screen is drawn. There are more uses for that than just fixing the dropdown issue.

Reassigning to @gaaclarke

I verified that focusing on the menu after a selection is made is the same behavior you get from android widgets via the Contacts app, Add Contact.

If I comment out this line of code in the engine I can reproduce the behavior on iOS:
- (void)accessibilityElementDidLoseFocus {
  if (![self isAccessibilityBridgeAlive])
    return;
//  if ([self node].HasAction(flutter::SemanticsAction::kDidLoseAccessibilityFocus)) {
//    [self bridge]->DispatchSemanticsAction([self uid],
//                                           flutter::SemanticsAction::kDidLoseAccessibilityFocus);
//  }
}

It seems the android engine is failing to send a kDidLoseAccessibilityFocus action.

This is the stacktrace for when the button gets focus again after an item is selected on iOS:
Thread 1 Queue : com.apple.main-thread (serial)
#0	0x0000000102b5c108 in -[SemanticsObject accessibilityElementDidBecomeFocused] at /Users/aaclarke/dev/engine/src/flutter/shell/platform/darwin/ios/framework/Source/SemanticsObject.mm:453
#1	0x00000001ab4b1024 in -[NSObject(AXPrivCategory) _accessibilityHandleATFocused:assistiveTech:] ()
#2	0x00000001ab4a05bc in -[NSObject(AXPrivCategory) _iosAccessibilitySetValue:forAttribute:] ()
#3	0x00000001ab47e1f4 in _setAttributeValueCallback ()
#4	0x00000001aa4e5150 in ___AXXMIGSetAttributeValue_block_invoke ()
#5	0x00000001aa4e47a4 in _handleNonMainThreadCallback ()
#6	0x00000001aa4e505c in _AXXMIGSetAttributeValue ()
#7	0x00000001aa4ddbbc in _XSetAttributeValue ()
#8	0x00000001aa4f4210 in mshMIGPerform ()
#9	0x000000019feac2b4 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ ()
#10	0x000000019feab9c0 in __CFRunLoopDoSource1 ()
#11	0x000000019fea6774 in __CFRunLoopRun ()
#12	0x000000019fea5ce8 in CFRunLoopRunSpecific ()
#13	0x00000001a9ff038c in GSEventRunModal ()
#14	0x00000001a3fd4444 in UIApplicationMain ()
#15	0x00000001026fcb54 in main at /Users/aaclarke/dev/flutter-examples/counter/ios/Runner/AppDelegate.swift:5
#16	0x000000019fd2d8f0 in start ()

If you place this print statement into the engine:
bool RuntimeController::DispatchSemanticsAction(int32_t id,
                                                SemanticsAction action,
                                                std::vector<uint8_t> args) {
  FML_LOG(ERROR) << "aaclarke DispatchSemanticsAction " << id << " " << static_cast<int>(action);

This is the only thing that is printed out:
[VERBOSE-2:runtime_controller.cc(265)] aaclarke DispatchSemanticsAction 4 1
[VERBOSE-2:runtime_controller.cc(265)] aaclarke DispatchSemanticsAction 11 1

1 corresponds to kTap so it doesn't seem to be a message being sent from the embedder to the engine that causes focus to change after the dropdown is dismissed.  So, it isn't semantic actions that are responsible for the refocus.

I verified that the FocusManager is behaving the same on Android and iOS.  It isn't some difference there that causes the different behaviors.
After dismissing the dropdown this is the log:
I/flutter (17378): FocusManager#0cbe9
I/flutter (17378):  │ primaryFocus: FocusScopeNode#4951c(_ModalScopeState<dynamic>
I/flutter (17378):  │   Focus Scope [PRIMARY FOCUS])
I/flutter (17378):  │ primaryFocusCreator: FocusScope ← _ActionsMarker ← Actions ←
I/flutter (17378):  │   PageStorage ← Offstage ← _ModalScopeStatus ←
I/flutter (17378):  │   _ModalScope<dynamic>-[LabeledGlobalKey<_ModalScopeState<dynamic>>#5883d]
I/flutter (17378):  │   ← Semantics ← _EffectiveTickerMode ← TickerMode ←
I/flutter (17378):  │   _OverlayEntryWidget-[LabeledGlobalKey<_OverlayEntryWidgetState>#85206]
I/flutter (17378):  │   ← _Theatre ← Overlay-[LabeledGlobalKey<OverlayState>#19cb2] ←
I/flutter (17378):  │   _FocusMarker ← Semantics ← FocusScope ← AbsorbPointer ←
I/flutter (17378):  │   _PointerListener ← Listener ← HeroControllerScope ← ⋯
I/flutter (17378):  │
I/flutter (17378):  └─rootScope: FocusScopeNode#347c4(Root Focus Scope [IN FOCUS PATH])
I/flutter (17378):    │ IN FOCUS PATH
I/flutter (17378):    │ focusedChildren: FocusScopeNode#48d46(Navigator Scope [IN FOCUS
I/flutter (17378):    │   PATH])
I/flutter (17378):    │
I/flutter (17378):    └─Child 1: FocusNode#c0e4e(Shortcuts [IN FOCUS PATH])
I/flutter (17378):      │ context: Focus
I/flutter (17378):      │ NOT FOCUSABLE
I/flutter (17378):      │ IN FOCUS PATH
I/flutter (17378):      │
I/flutter (17378):      └─Child 1: FocusNode#0f9bb(FocusTraversalGroup [IN FOCUS PATH])
I/flutter (17378):        │ context: Focus
I/flutter (17378):        │ NOT FOCUSABLE
I/flutter (17378):        │ IN FOCUS PATH
I/flutter (17378):        │
I/flutter (17378):        └─Child 1: FocusScopeNode#48d46(Navigator Scope [IN FOCUS PATH])
I/flutter (17378):          │ context: FocusScope
I/flutter (17378):          │ IN FOCUS PATH
I/flutter (17378):          │ focusedChildren: FocusScopeNode#4951c(_ModalScopeState<dynamic>
I/flutter (17378):          │   Focus Scope [PRIMARY FOCUS])
I/flutter (17378):          │
I/flutter (17378):          ├─Child 1: FocusScopeNode#4951c(_ModalScopeState<dynamic> Focus Scope [PRIMARY FOCUS])
I/flutter (17378):          │ │ context: FocusScope
I/flutter (17378):          │ │ PRIMARY FOCUS
I/flutter (17378):          │ │
I/flutter (17378):          │ ├─Child 1: FocusNode#1d7ff(DropdownButton<String>)
I/flutter (17378):          │ │   context: Focus
I/flutter (17378):          │ │
I/flutter (17378):          │ └─Child 2: FocusNode#7c625
I/flutter (17378):          │     context: Focus
I/flutter (17378):          │
I/flutter (17378):          └─Child 2: FocusScopeNode#17c13(_ModalScopeState<dynamic> Focus Scope)
I/flutter (17378):            │ context: FocusScope
I/flutter (17378):            │ NOT FOCUSABLE
I/flutter (17378):            │ focusedChildren: FocusNode#75637
I/flutter (17378):            │
I/flutter (17378):            ├─Child 1: FocusNode#75637
I/flutter (17378):            │   context: Focus
I/flutter (17378):            │   NOT FOCUSABLE
I/flutter (17378):            │
I/flutter (17378):            ├─Child 2: FocusNode#09526
I/flutter (17378):            │   context: Focus
I/flutter (17378):            │   NOT FOCUSABLE
I/flutter (17378):            │
I/flutter (17378):            └─Child 3: FocusNode#0c8a6
I/flutter (17378):                context: Focus
I/flutter (17378):                NOT FOCUSABLE

The _ModalScopeState is focused, not the DropdownButton which is a little surprising.
dropdownfocusandroid.txt
dropdownfocusios.txt

When following through the steps the following error is printed when the dropdown menu is presented and when it is dismissed which may be related:
E/AccessibilityBridge( 9738): Semantics node id does not equal ROOT_NODE_ID.
E/AccessibilityBridge( 9738): Semantics node id does not equal ROOT_NODE_ID.

It was a bit hard to reproduce with Talkback debugging activated since often the dropdown menu failed to popup, with the following error printing to the screen:
I/Adreno  ( 9738): DequeueBuffer: dequeueBuffer failed

Here is the semantic tree:
before interaction
06-25 13:38:39.219  8393  8393 V TreeDebug: Window: AccessibilityWindowInfo[title=nullid=93, type=TYPE_SYSTEM, layer=161000, bounds=Rect(0, 0 - 1440, 84), focused=false, active=false, hasParent=false, isAnchored=false, hasChildren=false]
06-25 13:38:39.223  8393  8393 V TreeDebug: (-2147452834)93.FrameLayout:(0, 0 - 1440, 84):A
06-25 13:38:39.223  8393  8393 V TreeDebug:   (-2147450912)93.ImageView:(33, 12 - 93, 72):CD{Family Room TV notification: A device on your Wi-Fi is casting}:A
06-25 13:38:39.223  8393  8393 V TreeDebug:   (-2147449951)93.ImageView:(117, 12 - 177, 72):CD{Tango Core notification: Tango User Segments Uploaded: 0 / 0}:A
06-25 13:38:39.223  8393  8393 V TreeDebug:   (-2147448990)93.ImageView:(201, 12 - 261, 72):CD{Splendid notification: Splendid}:A
06-25 13:38:39.224  8393  8393 V TreeDebug:   (-2147448029)93.ImageView:(285, 12 - 345, 72):CD{Android System notification: USB debugging connected}:A
06-25 13:38:39.224  8393  8393 V TreeDebug:   (-2147447068)93.FrameLayout:(939, 0 - 1023, 84):CD{Wifi signal full.}:A
06-25 13:38:39.224  8393  8393 V TreeDebug:   (-2147446107)93.FrameLayout:(1047, 0 - 1131, 84):CD{No SIM card.}:A
06-25 13:38:39.224  8393  8393 V TreeDebug:   (-2147445146)93.ImageView:(1138, 0 - 1222, 84):CD{Battery charging, 71 percent.}:A
06-25 13:38:39.225  8393  8393 V TreeDebug:   (-2147453795)93.TextView:(1222, 0 - 1412, 84):TEXT{1:38 PM}:CD{1:38 PM}:A
06-25 13:38:39.225  8393  8393 V TreeDebug: Window: AccessibilityWindowInfo[title=issue_42372id=100, type=TYPE_APPLICATION, layer=21035, bounds=Rect(0, 0 - 1440, 2560), focused=true, active=true, hasParent=false, isAnchored=false, hasChildren=false]
06-25 13:38:39.229  8393  8393 V TreeDebug: (-2147453788)100.FrameLayout:(0, 0 - 1440, 2560):A
06-25 13:38:39.229  8393  8393 V TreeDebug:   (-2147452827)100.FrameLayout:(0, 0 - 1440, 2560):A:focusable:focused
06-25 13:38:39.229  8393  8393 V TreeDebug:     (30852)100.View:(0, 0 - 1440, 2560):A
06-25 13:38:39.230  8393  8393 V TreeDebug:       (30883)100.View:(0, 0 - 1440, 2560):A
06-25 13:38:39.230  8393  8393 V TreeDebug:         (30914)100.View:(0, 0 - 1440, 2560):A
06-25 13:38:39.230  8393  8393 V TreeDebug:           (30945)100.View:(0, 0 - 1440, 2560):A
06-25 13:38:39.230  8393  8393 V TreeDebug:             (30976)100.Button:(477, 1238 - 655, 1406):CD{Foo}:AC:focusable:clickable
06-25 13:38:39.230  8393  8393 V TreeDebug:             (31007)100.Button:(655, 1238 - 963, 1406):CD{Reset}:AC:focusable:clickable

while dropdown is present
06-25 13:39:55.789  8393  8393 V TreeDebug: Window: AccessibilityWindowInfo[title=nullid=93, type=TYPE_SYSTEM, layer=161000, bounds=Rect(0, 0 - 1440, 84), focused=false, active=false, hasParent=false, isAnchored=false, hasChildren=false]
06-25 13:39:55.793  8393  8393 V TreeDebug: (-2147452834)93.FrameLayout:(0, 0 - 1440, 84):A
06-25 13:39:55.793  8393  8393 V TreeDebug:   (-2147450912)93.ImageView:(33, 12 - 93, 72):CD{Family Room TV notification: A device on your Wi-Fi is casting}:A
06-25 13:39:55.793  8393  8393 V TreeDebug:   (-2147449951)93.ImageView:(117, 12 - 177, 72):CD{Tango Core notification: Tango User Segments Uploaded: 0 / 0}:A
06-25 13:39:55.793  8393  8393 V TreeDebug:   (-2147448990)93.ImageView:(201, 12 - 261, 72):CD{Splendid notification: Splendid}:A
06-25 13:39:55.793  8393  8393 V TreeDebug:   (-2147448029)93.ImageView:(285, 12 - 345, 72):CD{Android System notification: USB debugging connected}:A
06-25 13:39:55.794  8393  8393 V TreeDebug:   (-2147447068)93.FrameLayout:(939, 0 - 1023, 84):CD{Wifi signal full.}:A
06-25 13:39:55.794  8393  8393 V TreeDebug:   (-2147446107)93.FrameLayout:(1047, 0 - 1131, 84):CD{No SIM card.}:A
06-25 13:39:55.794  8393  8393 V TreeDebug:   (-2147445146)93.ImageView:(1138, 0 - 1222, 84):CD{Battery charging, 71 percent.}:A
06-25 13:39:55.794  8393  8393 V TreeDebug:   (-2147453795)93.TextView:(1222, 0 - 1412, 84):TEXT{1:39 PM}:CD{1:39 PM}:A
06-25 13:39:55.794  8393  8393 V TreeDebug: Window: AccessibilityWindowInfo[title=issue_42372id=100, type=TYPE_APPLICATION, layer=21035, bounds=Rect(0, 0 - 1440, 2560), focused=true, active=true, hasParent=false, isAnchored=false, hasChildren=false]
06-25 13:39:55.797  8393  8393 V TreeDebug: (-2147453788)100.FrameLayout:(0, 0 - 1440, 2560):A
06-25 13:39:55.797  8393  8393 V TreeDebug:   (-2147452827)100.FrameLayout:(0, 0 - 1440, 2560):A:focusable:focused
06-25 13:39:55.798  8393  8393 V TreeDebug:     (30852)100.View:(0, 0 - 1440, 2560):A
06-25 13:39:55.798  8393  8393 V TreeDebug:       (30883)100.View:(0, 0 - 1440, 2560):A
06-25 13:39:55.798  8393  8393 V TreeDebug:         (31069)100.View:(0, 0 - 1440, 2560):A
06-25 13:39:55.798  8393  8393 V TreeDebug:           (31100)100.View:(421, 1210 - 739, 1770):A
06-25 13:39:55.798  8393  8393 V TreeDebug:             (31131)100.View:(421, 1210 - 739, 1770):A
06-25 13:39:55.798  8393  8393 V TreeDebug:               (31255)100.View:(421, 1210 - 739, 1770):A:focusable
06-25 13:39:55.798  8393  8393 V TreeDebug:                 (31162)100.View:(421, 1238 - 739, 1406):CD{Foo}:aC:focusable:focused:clickable:accessibilityFocused
06-25 13:39:55.799  8393  8393 V TreeDebug:                 (31193)100.View:(421, 1406 - 739, 1574):CD{Bar}:AC:focusable:clickable
06-25 13:39:55.799  8393  8393 V TreeDebug:                 (31224)100.View:(421, 1574 - 739, 1742):CD{Baz}:AC:focusable:clickable
06-25 13:39:55.799  8393  8393 V TreeDebug:         (31038)100.View:(0, 0 - 1440, 2560):A

after dismissal
06-25 13:40:21.711  8393  8393 V TreeDebug: Window: AccessibilityWindowInfo[title=nullid=93, type=TYPE_SYSTEM, layer=161000, bounds=Rect(0, 0 - 1440, 84), focused=false, active=false, hasParent=false, isAnchored=false, hasChildren=false]
06-25 13:40:21.718  8393  8393 V TreeDebug: (-2147452834)93.FrameLayout:(0, 0 - 1440, 84):A
06-25 13:40:21.718  8393  8393 V TreeDebug:   (-2147450912)93.ImageView:(33, 12 - 93, 72):CD{Family Room TV notification: A device on your Wi-Fi is casting}:A
06-25 13:40:21.718  8393  8393 V TreeDebug:   (-2147449951)93.ImageView:(117, 12 - 177, 72):CD{Tango Core notification: Tango User Segments Uploaded: 0 / 0}:A
06-25 13:40:21.719  8393  8393 V TreeDebug:   (-2147448990)93.ImageView:(201, 12 - 261, 72):CD{Splendid notification: Splendid}:A
06-25 13:40:21.719  8393  8393 V TreeDebug:   (-2147448029)93.ImageView:(285, 12 - 345, 72):CD{Android System notification: USB debugging connected}:A
06-25 13:40:21.719  8393  8393 V TreeDebug:   (-2147447068)93.FrameLayout:(939, 0 - 1023, 84):CD{Wifi signal full.}:A
06-25 13:40:21.719  8393  8393 V TreeDebug:   (-2147446107)93.FrameLayout:(1047, 0 - 1131, 84):CD{No SIM card.}:A
06-25 13:40:21.719  8393  8393 V TreeDebug:   (-2147445146)93.ImageView:(1138, 0 - 1222, 84):CD{Battery charging, 71 percent.}:A
06-25 13:40:21.719  8393  8393 V TreeDebug:   (-2147453795)93.TextView:(1222, 0 - 1412, 84):TEXT{1:40 PM}:CD{1:40 PM}:A
06-25 13:40:21.719  8393  8393 V TreeDebug: Window: AccessibilityWindowInfo[title=issue_42372id=100, type=TYPE_APPLICATION, layer=21035, bounds=Rect(0, 0 - 1440, 2560), focused=true, active=true, hasParent=false, isAnchored=false, hasChildren=false]
06-25 13:40:21.723  8393  8393 V TreeDebug: (-2147453788)100.FrameLayout:(0, 0 - 1440, 2560):A
06-25 13:40:21.724  8393  8393 V TreeDebug:   (-2147452827)100.FrameLayout:(0, 0 - 1440, 2560):A:focusable:focused
06-25 13:40:21.724  8393  8393 V TreeDebug:     (30852)100.View:(0, 0 - 1440, 2560):A
06-25 13:40:21.724  8393  8393 V TreeDebug:       (30883)100.View:(0, 0 - 1440, 2560):A
06-25 13:40:21.724  8393  8393 V TreeDebug:         (30914)100.View:(0, 0 - 1440, 2560):A
06-25 13:40:21.724  8393  8393 V TreeDebug:           (30945)100.View:(0, 0 - 1440, 2560):A
06-25 13:40:21.724  8393  8393 V TreeDebug:             (30976)100.Button:(477, 1238 - 655, 1406):CD{Baz}:AC:focusable:clickable
06-25 13:40:21.724  8393  8393 V TreeDebug:             (31007)100.Button:(655, 1238 - 963, 1406):CD{Reset}:AC:focusable:clickable

We can clearly see that the dropdown isn't focused after the dismissal.  It is a bit hard for me to debug this though because there is a bug where the rectangle is not getting rendered and the gesture for printing the semantic tree doesn't work 100% of the time.  We can at least see that the tree looks correct.

Here is the verbose log from talkback.  I've looked it over a couple of times for clues.  There isn't any focus event happening, so the dropdown button isn't explicitly not being selected.     One oddity is that there is a TYPE_VIEW_CLICKED event happening in the middle of the logs during TYPE_WINDOW_CONTENT_CHANGED messages:
...
06-25 13:55:42.371  8393  8393 D Pipeline: execute() feedback=Feedback{eventId=type:EVENT_TYPE_ACCESSIBILITY subtype:TYPE_WINDOW_CONTENT_CHANGED time:2320448, sequence=[Part= speech= Speech{action=SPEAK, text=, options=com.google.android.accessibility.utils.output.SpeechController$SpeakOptions@b83873e5} ]}
06-25 13:55:42.372  8393  8393 D Actors  : act() eventId=type:EVENT_TYPE_ACCESSIBILITY subtype:TYPE_WINDOW_CONTENT_CHANGED time:2320448 part=Part= speech= Speech{action=SPEAK, text=, options=com.google.android.accessibility.utils.output.SpeechController$SpeakOptions@b83873e5}
06-25 13:55:42.446  8393  8393 V A11yEventProcessor: Event listeners: Pipeline,ProcessorEventQueue,ProcessorPhoneticLetters,ProcessorAccessibilityHints
06-25 13:55:42.447  8393  8393 D Pipeline: execute() feedback=Feedback{eventId=type:EVENT_TYPE_ACCESSIBILITY subtype:TYPE_VIEW_CLICKED time:2320382, sequence=[Part= interruptGroup=0 interruptLevel=1 ]}
06-25 13:55:42.447  8393  8393 D Actors  : act() eventId=type:EVENT_TYPE_ACCESSIBILITY subtype:TYPE_VIEW_CLICKED time:2320382 part=Part= interruptGroup=0 interruptLevel=1
06-25 13:55:42.448  8393  8393 D ProcessorEventQueue: Processing event: EventType: TYPE_VIEW_CLICKED; EventTime: 2320382; PackageName: com.example.issue_42372; MovementGranularity: 0; Action: 0 [ ClassName: null; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: false; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
...

That event could potentially subvert selecting the dropdown button?
dropdownlog.txt

@dnfield I have a theory why this is happening and it's something you seem to have worked on in the past.  I think the error message "Semantics node id does not equal ROOT_NODE_ID." is apropos.  I edited the the print statement and now it prints out:
E/AccessibilityBridge(19038): Semantics node id does not equal ROOT_NODE_ID. {id:4, flags:200008, actions:1, label:Foo, value:null, hint:null}
E/AccessibilityBridge(19038): Semantics node id does not equal ROOT_NODE_ID. {id:4, flags:200008, actions:1, label:Foo, value:null, hint:null}

So, it is related to our problem.  If you look at the code this error happens when unexpectedly there is a null parent, so it just sets the parent to the view.  My theory is that that may be confusing the focus after the menu is dismissed (despite the tree seeming to look fine  when i print it out).  What do you think of that theory, any insight?

This seems like it's an issue in the framework.
If you have a small repro, it'd be helpful to add debugDumpSemantics before and after the problem happens. Something weird seems to be happening to the tree - it's probably not valid on the framework side though.

Here is another comparison between the output of a working widget and this widget.  I think the important thing to look for is FocusProcForScreenState
When the menu is dismissed we don't hear from FocusProcForScreenState in flutter, but we do with the android widget.  It reports a SUCCESS_RESTORED_LAST_FOCUS:
06-25 19:41:48.114  8393  8393 D ProcessorEventQueue: Processing event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 20869996; PackageName: com.asus.contacts; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.Spinner; Text: []; ContentDescription: null; ItemCount: 10; CurrentItemIndex: 7; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: true; BeforeText: null; FromIndex: 7; ToIndex: 7; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
06-25 19:41:48.655  8393  8393 V WindowEventInterpreter:    updateWindowRoles() interpretation=WindowA= ID=119 Title="Contacts" TitleForFeedback="Contacts" OldID=131  WindowB= TitleForFeedback="untitled"  A11yOverlay= TitleForFeedback="untitled"  PicInPic= TitleForFeedback="untitled"  WindowIdFromEvent=119 MainWindowsChanged WindowsStable allowAnnounce EventType=TYPE_WINDOW_STATE_CHANGED
06-25 19:41:48.656  8393  8393 V WindowEventInterpreter: END delayedInterpret() interpretation=WindowA= ID=119 Title="Contacts" TitleForFeedback="Contacts" OldID=131  WindowB= TitleForFeedback="untitled"  A11yOverlay= TitleForFeedback="untitled"  PicInPic= TitleForFeedback="untitled"  WindowIdFromEvent=119 MainWindowsChanged WindowsStable allowAnnounce EventType=TYPE_WINDOW_STATE_CHANGED
06-25 19:41:48.657  8393  8393 V ScreenFeedbackManager: composeFeedback() interpretation=WindowA= ID=119 Title="Contacts" TitleForFeedback="Contacts" OldID=131  WindowB= TitleForFeedback="untitled"  A11yOverlay= TitleForFeedback="untitled"  PicInPic= TitleForFeedback="untitled"  WindowIdFromEvent=119 MainWindowsChanged WindowsStable allowAnnounce EventType=TYPE_WINDOW_STATE_CHANGED
06-25 19:41:48.661  8393  8393 V SpeechControllerImpl: Speaking fragment text "Contacts" with spans null for event type:EVENT_TYPE_ACCESSIBILITY subtype:TYPE_WINDOW_STATE_CHANGED time:20869907
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: Screen state changed with result=SUCCESS_RESTORED_LAST_FOCUS :
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: Start time=20869891
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: Duration=724
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: From: Number of windows:3
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: System window: 93-null;
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: Application window: 131-null(active);
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: Accessibility window:
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: InputMethod window: 122-null;
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: PicInPic window:
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: isInSplitScreenMode:false
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: To: Number of windows:3
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: System window: 93-null;
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: Application window: 119-Contacts(active);
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: Accessibility window:
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: InputMethod window: 122-null;
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: PicInPic window:
06-25 19:41:48.719  8393  8393 D FocusProcForScreenState: isInSplitScreenMode:false

In flutter that message doesn't happen after its TYPE_WINDOW_CONTENT_CHANGED:
06-25 19:50:15.912  8393  8393 D ProcessorEventQueue: Processing event: EventType: TYPE_WINDOW_STATE_CHANGED; EventTime: 21377777; PackageName: com.example.issue_42372; MovementGranularity: 0; Action: 0 [ ClassName: null; Text: [null]; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: false; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
06-25 19:50:15.912  8393  8393 D ProcessorEventQueue: Processing event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 21377785; PackageName: com.example.issue_42372; MovementGranularity: 0; Action: 0 [ ClassName: null; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: false; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
06-25 19:50:16.054  8393  8393 V A11yEventProcessor: Event listeners: ScrollEventInterpreter,Pipeline,ProcessorEventQueue
06-25 19:50:16.055  8393  8393 D ProcessorEventQueue: Processing event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 21377945; PackageName: com.example.issue_42372; MovementGranularity: 0; Action: 0 [ ClassName: null; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: false; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0

worksdropdown.txt
pathological.txt

YAQS<PHONE_NUMBER>981824 filed.

Please see YAQs or b/142427914, won't fix, infeasible.

Oh, also for documentation sake I tried to delay the TYPE_WINDOW_CONTENT_CHANGED by one second to see if it is a timing issue.  That didn't fix the issue.  It is probably more likely because the popup menu is not a separate window.

@gaaclarke so this is not currently being worked on?

There is nothing to work on, this is the behavior of Android's Talkback and the Android team said it's the expected behavior.  You could take my information and file a bug with them if you think that's the incorrect conclusion.


Build Pyodide into workerd

This uses bazelisk to download the upstream Pyodide v0.25.0a1 release and compile it into workerd. We should soon make our own custom build of Pyodide because upstream Pyodide loads slowly. But this is a start.

Looks like the CI is failing because autogates are not initialised in the TestFixture. Here is a diff that fixes that:
diff --git a/src/workerd/tests/test-fixture.c++ b/src/workerd/tests/test-fixture.c++
index f5ab2e53..8b5a6677 100644
--- a/src/workerd/tests/test-fixture.c++
+++ b/src/workerd/tests/test-fixture.c++
@@ -189,6 +189,14 @@ inline server::config::Worker::Reader buildConfig(
   auto modules = config.initModules(1);
   modules[0].setName(mainModuleName);
   modules[0].setEsModule(params.mainModuleSource.orDefault(mainModuleSource));
+
+  // Initialise autogates with an empty config. TODO(later): allow TestFixture to accept autogate
+  // states and pass them in here.
+  //
+  // This needs to happen here because `buildConfig` is called early in the construction of
+  // `TestFixture`.
+  util::Autogate::initAutogate({});
+
   return config;
 }

I highly suggest separating dep+build change from actually making pyodide to work. This will make it easier to review and land considering it has downstream implication.

I'd appreciate it if we could move forward instead of splitting this into even more PRs (this is already nr. 3 in a series of PRs and we have other PRs depending on this). We need to get this in so we can iterate on it in our internal repo (right now I'm blocked on this).
Build changes should be very low risk, so I'd rather we merge in what we have and iterate on the build files in parallel rather than block this PR further.

Unconditionally installing unsafe eval for everyone is definitely a big security concern.

This is not unconditionally installing it for everyone. This exposes it only through an internal-only import that is only available to built-in modules.

Perhaps the UnsafeEval changes should also be split off then? Probably it would also be good to add a basic test for newWasmModule to unsafe-test.js.

Okay I added some basic tests for newWasmModule in unsafe-test.js and a separate test that we can't import UnsafeEval from user code directly to src/workerd/server/tests/unsafe-eval/.


feat: Enable read+write: Zendesk Support

#895  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @Cobalt0s and the rest of your teammates on  Graphite


WasmJs Support

Is there anything currently blocking wasmJs support?

@eygraber just moko resources. I want to just port to compose resources at this point when I get the chance.

I ported it to compose resources. So should be pretty easy to add wasmJs now when I get a chance

I'm having a build issue with mockative. But wasmJs runs on this branch
https://github.com/luca992/libphonenumber-kotlin/tree/wasmJs


Cleanup Travis and tox configs

Cleaned up the Travis and tox configs, including some bits missed in my PR to django-sudo:

Moved the environment list completely to tox, which should be easier to read given how we need to exclude quite a few Django/Python combinations
Added linting and doc building to tox and fixed a few issues
Linting and doc building are done as separate jobs on Travis
Kept all this compatible with the already defined invoke tasks
Removed dependencies from dev-requirements.txt that are no longer needed

I've added flake8 and pytest back into dev-requirements.txt. I've also added django to that file because pytest is pretty useless without it.

Looks good to me. Do you think any of these commits should be squashed? I'm happy to combine them into a single commit upon merge, of course... Just wasn't sure if you wanted to organize them in any particular semantic manner. ☺

I have no personal preference, so feel free to do a squash-merge ☺


Social Media: 03.10.22

Please add below any items you'd like to share on the OpenFeature social media channels (Linkedin, Twitter, YouTube) so I can schedule them for publishing this week.
Examples: project updates, new releases, blog posts, videos, future events, speaking engagements, etc.
Don't forget to add your social media handles if you want to be tagged!

OpenFeature will have a part-time booth with the AM schedule

Wednesday, October 26: 10:30 - 3:30 ET
Thursday, October 27: 10:30 - 2:30 ET
Friday, October 27: 10:30 - 1:00 ET


Documentation Typo Fix

Fixes typo in documentation:
Process managment --> Process management

@mginnard Merged it. Thanks for the pull :)


[WIP] /: add timeout for the k8s API

Currently, we don't set any timeouts.
This fixes it by setting the following timeouts:

Sets the timeout on each REST call, see
k8s.io/client-go/rest#Request.Timeout
Sets the global http timeout on k8s's http client, see
https://github.com/kubernetes/client-go/blob/8abb21031259350aad0799bb42ba213ee8bb3399/rest/config.go#L211-L216

Note: This needs some testing, before merging.
/cc @brancz @squat

Do we actually need to pass the timeout every single time? Isn't that something that can be done when creating the client once?

@metalmatze yes and no, the http client is initialized only once. But a per-request timeout is also being set, effectively setting its context, see https://github.com/kubernetes/kubernetes/pull/70998.

@s-urbaniak the per request timeout sets a timeout query parameter so that the server can can close long requests, but arguably that’s not what we really care about here, no? Our case is that the server is unable to close the timeouts because it’s not actually up and running yet so the query parameter doesn’t really help. Maybe we could instead simplify this PR to only set the timeout on the client config OR we could bump the client-go dep to something that includes that new commit and sets all automatically. Otherwise I think we may be complicating our codebase and adding debt for little or no benefit.

@squat agreed, this is starting to become a rabbit hole. I also discussed with @brancz adding client generation here. For now I'll open a separate PR on the http client timeout only and leave this PR open for client generation.

closing in favor of #2163


Implement default epinow2 config generation script

Implement a script that generates a set of configuration objects based on the default parameters in https://github.com/cdcent/cfa-nnh-pipelines/blob/33b4a55daba3479cddc85fe10dac4732b2f2c91b/NHSN/Rt/run_azure_batch/write_config.R
It should also accept arguments so that the workflow can be triggered manually with a different set of arguments than the default.
My thought was to adapt the YAML in that script to the sample config here.
This PR will just handle the JSON generation, but future work will validate and push timestamped versions to Blob Storage.
Some questions about the schema for @zsusswein @natemcintosh --

What are the values for geo_type? Is there a reason geo_value is an array (my understanding was that each config would just have one geography associated with it)?
Is the default report_date (today) the same as the default as_of_date and reference_date?
Each set of configs will have the same job_id (I think I will use UUID7 as @natemcintosh suggested), but can the task_id be any random string?

geo_type is almost always "state". However, at some point we'd also like to be able to run model at e.g. the county level, in which case geo_type would be "county". (Also when we do a model run for the whole US, it is not "state". I'm not sure of the exact name we use there, but it's probably something like "nation".
The report_date is usually just today, though sometimes we'd like to do retrospective testing, and then we'll change it. I believe as_of_date would be a timestamp, where report_date would just be a date. E.g. in python, as_of_date would be a datetime.datetime, and report_date is a datetime.date. @zsusswein is that correct?. reference_date is the day of the "event", e.g. hospitalization, ED visit date, etc. So a given report_date would probably have a bunch of reference_dates, where all(reference_date <= report_date) == True.
I think UUID7 would be good for jobs, but after looking around, I've really only found one python library that can generate them, and it looks slightly half-baked, and has not been updated in a while. I think if we can find a good way to generate them, great, but otherwise we should consider some other UUID. In terms of task IDs, it would be useful to the person running the model to be able to see what particular pairings made up that task, e.g. Covid and the state of California => COVID-CA. That said, there may be other considerations I've forgotten about.

Thanks @natemcintosh ! This is helpful; I'll take a stab at a PR and we can iterate from there.


[CSL-145] [Research] Scripting

@neongreen

@neongreen
Status update:
We know what language we are going to have and there's no turning back. We know that we (likely) won't have built-in MAST, and we know how scripts would be used in the context of CSL.
This week @augur has fixed some critical bugs in Plutus. There was no progress made on two more-or-less big questions (specifically, primitive types and the imports problem). We'll get to them after Plutus is more fleshed out (e.g. right now @augur is working on a better parser for the language).

@neongreen
Status update:
@augur is doing research:

“resource-aware Plutus”: https://input-output-rnd.slack.com/archives/sl-plutus/p1480703923000355
https://input-output-rnd.slack.com/archives/sl-plutus/p1480704130000360

This week I'm going to nag @augur about adding primitives, see his comments in #cardano-sl-script:

i can add primitives, i've just slacked a bit on that u.u;
ill add a bunch of primitives today, and i'll add some core functionality
regarding a standard library, we should figure out what the basic functionality
is that we want to have, at least at the What level, and I can write it quickly enough


Interest in wrapping drawerlayout

Is there any interest in wrapping drawerlayout on android? http://facebook.github.io/react-native/docs/drawerlayoutandroid.html#content

On the other hand - we should provide maybe at least very limited bridge/proxy so that it also works on Android without any extra switches / separate components.

On the other hand - we should provide maybe at least very limited bridge/proxy so that it also works on Android without any extra switches / separate components.