App does not route by default

Hi,
I had trouble with figuring out how to set up a separate /api middleware pipe.
My system set up:

A root middleware pipe (Zend\Expressive\Application) with a home route / serving the layout + assets for a SPA.
A API middleware pipe (also Zend\Expressive\Application) with a programmatic created router following the docs

This is how my index.php initializes the pipes:
$services = new \Zend\ServiceManager\ServiceManager(new \Zend\ServiceManager\Config(require 'config/services.php'));

$app = $services->get('Zend\Expressive\Application');

$cargoBackend = $services->get('cargo.backend');

$app->pipe('/api', $cargoBackend);

$app->run();

And this is the factory responsible for the cargo.backend middleware pipe:
    public function __invoke(ContainerInterface $container)
    {
        $router = $container->get('cargo.backend.router');

        $app = AppFactory::create($container, $router);

        //This is the problem a want to address with the issue!!!
        $app->pipe("/", [$app, 'routeMiddleware']);

        return $app;
    }

I was not aware that I have to pipe the router manually when using the AppFactory. When passing a router to it I expect that the router gets activated automatically by the factory.
I would like to submit a PR that fixes the problem, but want to ask first if this would have side effects
and if such a change is wanted/needed? Otherwise we should document it. The cookbook would be a good place.
I have a second problem but currently don't know what causes it:
Before I added the line $app->pipe("/", [$app, 'routeMiddleware']) to the factory my route was not invoked (/api/locations) but I got an empty response with a 200 ok status code instead of a 404 not found. Maybe this happens because the api app is set up with its own emitter and final handler by default, so the final handler of the root app is not invoked?

@codeliner did you try this cookbook? https://github.com/zendframework/zend-expressive/blob/master/doc/book/cookbook.md#how-can-i-prepend-a-common-path-to-all-my-routes
You don't need to inject the router programmatically, you can just consume the API as follow:
$config = new Zend\ServiceManager\Config(require 'config/services.php');
$sm = new Zend\ServiceManager\ServiceManager($config);

$api = Zend\Expressive\AppFactory::create();
$api->pipe($sm->get('cargo.backend'));

$app =  $sm->get('Zend\Expressive\Application');
$app->pipe('/api', $api);
$app->pipe('/', $sm->get('home')); // where 'home' is the middleware for layout + SPA
$app->run();

@ezimuel thanks for fast response. yea my main routes work fine: '/' and '/api'
but my cargo.backend defines it own routes like /locations which should be become /api/locations when piping the backend to /api.
This mechanism works fine. The only problem I have is that when I pass a RouterInterface to AppFactory routing is still not enabled until I do this: $app->pipe("/", [$app, 'routeMiddleware']) manually. Why is this not included in the AppFactory?

Some more background to what I want to achieve:
I want to define - well let's call them module routes. So my cargo.backend is not only a middleware doing some stuff with the request and response but an independent sub system with its own routing logic. The main system only knows that all routes starting with /api should be routed to cargo.backend but not more. The cargo.backend then performs the sub routing to the action middlewares based on the uri sub path /locations.
So just doing this:
$api = Zend\Expressive\AppFactory::create();
$api->pipe($sm->get('cargo.backend'));

does not solve my problem, because cargo.backend is invoked on every /api request which is fine but still has the problem that routing inside cargo.backend gets not activated by default.

@codeliner I see now your point, you need to use $app->route() instead of $app->pipe(). In the route() method we pipe the routeMiddleware if not already added. The pipe() method is implemented by Zend\Stratigility\MiddlewarePipe that offers a basic dispatch system, without routing.

@ezimuel Yes and no. When debugging the code I also recognized that $app->route() adds routeMiddleware. That's the reason why I've then tried it with $app->pipe("/", [$app, 'routeMiddleware']) or to make it more explicit: $cargoBackend->pipe("/", [$cargoBackend, 'routeMiddleware']) in my cargo.backend factory. Maybe this is confusing in my original question. I don't set up the root app in the factory code shown above. I set up the cargo.backend which is also a Zend\Expressive\Application but used as a sub system invoked whenever a route starts with /api. It works. Problem solved. BUT the next developer trying to do same thing will have the same problems.
I cannot use $cargoBackend->route() because of this issue #40 so I followed the docs to set up the aura router for cargo.backend the same way as described in the docs: https://github.com/zendframework/zend-expressive/blob/master/doc/book/router.aura.md#programmatic-creation
See the last line there. The router is added to the app via AppFactory. But piping such a app to /api won't work because AppFactory does not perform: $app->pipe("/", [$app, 'routeMiddleware'])
Seems to be hard to explain. Would it help if I prepare an example that you can try out yourself?

Give me some minutes. I'll create a gist. A failing script will illustrate the problem better.

@ezimuel here is the gist: https://gist.github.com/codeliner/ced591bedcca422c7e42
Can you please have a look.

Invoke the home route first: /
Then try to request locations: /api/locations
Note: you will get a blank page but a 200 ok response?
Then comment out the line 34.
You should now get the locations as json when hitting refresh in the browser.

I'm not quite sure I understand the problem, and I'm currently on my
tablet. However, I've had success with nested middleware that uses routing
on my own site:

https://github.com/weierophinney/mwop.net

The auth, blog, and jobs middleware are each separate Application
instances, piped under sub paths. While it's a very simple site, it does
illustrate the concept.
One note: don't share router instances between them; that could lead to
route collisions. In my case, I simply didn't pass a router to AppFactory,
and used the default (aura).
On Aug 13, 2015 8:24 AM, "Alexander Miertsch"<EMAIL_ADDRESS>wrote:

Invoke the home route first: /
Then try to request locations: /api/locations
Note: you will get a blank page but a 200 ok response?
Then comment out the line 34.
You should now get the locations as json when hitting refresh in the
browser.
—
Reply to this email directly or view it on GitHub
https://github.com/zendframework/zend-expressive/issues/64#issuecomment-130669161
.

@weierophinney Thanks for the help and the link. Very good example project for zend-expressive. However, you don't hit the problem because you use Zend\Expressive\Application::route to set up sub routing. For example in the Blog\Middleware factory: https://github.com/weierophinney/mwop.net/blob/master/src/Blog/MiddlewareFactory.php
As @ezimuel said earlier, the first call to Zend\Expressive\Application::route enables routing. But that is magic.
My example script passes a ready to use aura router to Zend\Expressive\AppFactory but routing is not enabled after doing so. Do you have the chance to run my example scirpt? You will see the problem. Or the two problems, because getting a blank page but a 200 ok status code seems to be wrong too.

@codeliner I rewritten your gist here: https://gist.github.com/codeliner/ced591bedcca422c7e42#gistcomment-1550155.
As I commented in the gist the main point is the difference between pipe() and route(). If you use route($path, $middleware) you are saying if URL match $path the execute $middleware. If you use pipe($route, $middelware) you are just saying if URL === $route then execute the $middelware.
There is no magic here, the route() method needs a router for the matching, we initialize one by default if you don't pass it to the application.

@ezimuel my own router is gone in your rewrite! Now it is no longer the use case I want to tackle.
I want to use a programmatic approach to set up a Aura.Router like described here:
https://github.com/zendframework/zend-expressive/blob/master/doc/book/router.aura.md#programmatic-creation
Currently, because I need to workaround this issue here #40.
How would I now be able to set up a POST /locations route to add a new location? But anyway, it does not matter if I want to use my own Aura.Router because of the workaround or because other reasons. The fact is that passing a manually set up router to AppFactory does not enable it. And that is all what my example tries to illustrate. I'm aware of the difference between $app->pipe and $app->route. But I don't want to use the latter to set up routing. Why do the docs suggest programmatic router set up if this system does not support it?

@codeliner has a point

@codeliner I'm working on issue #40 and I'll fix very soon so please don't write code against it. Regarding the usage of a router instance, passed to AppFactory, we need to pipe the routeMiddleware into the Application to activate the routing part, that's the point.
To facilitate your use case, I'm thinking that we can pipe the routeMiddleare by default if we pass a valid $router object in the AppFactory. What do you think?

I agree, piping routeMiddleare by default will do :+1:

@ezimuel :+1: exactly, how my PR would look like. Should I submit one with a test covering it?

@ezimuel Ah and what about the blank page with 200 ok response? When no route matches shouldn't the final handler return a 404? $api has its own emitter even when used within $app which also has an emitter and a final handler. Can this be the problem?

@codeliner it should return a 404 but we are still working on error handling, we need to double check here. The emitter should not be a problem here, but again I need to check.
Anyway, regarding the pipe of routeMiddleware if $router is not empty in AppFactory I'm not 100% convinced. We should also investigate the possibility to move the logic of  routeMiddlewareIsRegistered directly in Application::__invoke()... @weierophinney what do you think?

@ezimuel good catch. The first approach adds routeMiddleware as first element to the pipe and the latter approach adds routeMiddleware as last element to the pipe. Question is: What is better? Both can cause trouble or unexpected behavior.
What about an event system with priorities? Oh wait that's zend-mvc .. ;)
What I want to say is: Let's keep it simple and don't support programmatic router set up at all. Remove it from the docs and replace it with: YOU MUST DEFINE YOUR ROUTES BY USING $app->route. Of course #40 needs to be solved first, but then the rule should be ok.
If some crazy developer like me want to use a programmatic set up router, well then $app->pipe([$app, 'routeMiddleware']) is the only way to achieve this. This can be documented in the cookbook as an edge-case. What do you think?

@codeliner I agree, the easy way for us is to document this "advanced" behaviour in the cookbook. Anyway the idea to pipe routeMiddleware during __invoke is intriguing, I'm playing with it :) I'm leaving open this issue to collect more comments by @weierophinney and others.

@ezimuel sounds good :+1:

I'll reply when I'm back from vacation. I strongly disagree with having
pipe() delegate to route(), but it's too hard to explain in a mobile
keyboard.
On Aug 13, 2015 12:01 PM, "Enrico Zimuel"<EMAIL_ADDRESS>wrote:

@codeliner https://github.com/codeliner I agree, the easy way for us is
to document this "advanced" behaviour in the cookbook. Anyway the idea to
pipe routeMiddleware during __invoke is intriguing, I'm playing with it
:) I'm leaving open this issue to collect more comments by @weierophinney
https://github.com/weierophinney and others.
—
Reply to this email directly or view it on GitHub
https://github.com/zendframework/zend-expressive/issues/64#issuecomment-130761254
.

Okay, reading this now on a screen larger than my phone, I finally understand the issue. I'll boil it down:
Primary issue is this:

You have a router that you are programmatically creating.
You pass the configured router to AppFactory.
By default, no routing occurs, as no routes were added using the route() method, forcing you to pipe the routing middleware manually.

Secondarily, before you started piping the routing middleware, you'd get a 200 OK, not a 404 Not Found status.

Now with that analysis, we have a three obvious options:

First, as @codeliner noted, we can simply say we don't support routes not added through route(). The problem with this approach is that we then cannot support any advanced routing features (e.g., SSL only, matching on query strings, etc.).
Second, we can pipe the routing middleware at instantiation. This, however, means that we cannot have middleware piped before the routing middleware, making it impossible to have middleware intercept before routing occurs.
Third, we could pipe the routing middleware at invocation. This, however, means that we cannot have middleware piped after the routing middleware, which makes piping error middleware impossible.

Let's look at some non-obvious options.

We could introduce a pipeRoutingMiddleware() method to do the work (which makes understanding how to register the routing middleware simpler).
We could retain the logic for piping the routing middleware in addRoute(), which allows keeping the current workflow. (This would be done with options 1 and 3 above now.)
The container factory would call on it immediately prior to attempting to add routes, which would retain that workflow.
We could add the call to pipe the routing middleware during invocation; that way, if nothing has registered it previously, it will now be registered.

The above would mean that your example could work without adding the call to pipe the routing middleware, but if you wanted to shape a custom pipeline, you could do so:
$app = AppFactory::create($container, $router);
$app->pipe($interceptingMiddleware);
$app->pipeRoutingMiddleware();
$app->pipe($errorMiddleware);

Does my summary of the problems and the proposed solution sound reasonable, @codeliner ?

Oh, also: does the secondary problem still occur with current master? We've now added error handling, and fixed a number of issues discovered with routing that may affect that situation.

@weierophinney Tried my gist with the latest master (0eafe91) and the second problem still occurs. I'm sorry that I can not tell you why this happens. Currently, I don't have time for a debugging session (maybe at the weekend if you want to wait). However, you can try it yourself. Just copy & paste my gist into an empty project. Composer install zend-expressive and run the gist with the php built-in server.
Regarding your approach with $app->pipeRoutingMiddleware(); I think that could do the trick. At least if it is documented and explained why I have to call the method.
But in this case we should remove the auto piping from $app->addRoute() and always require the user to call $app->pipeRoutingMiddleware();. Otherwise it feels like it is to much "magic" involved.
Piping middleware in the correct order is a very important task. Force the user to do this manually would underline the importance.
The only fallback I would add would be this:

We could add the call to pipe the routing middleware during invocation; that way, if nothing has registered it previously, it will now be registered.

But only with a big, fat warning in the documentation that the routing middleware is in this case triggered as the last middleware in the pipe which may lead to unexpected behavior.

But in this case we should remove the auto piping from $app->addRoute() and always require the user to call $app->pipeRoutingMiddleware();. Otherwise it feels like it is to much "magic" involved.
Piping middleware in the correct order is a very important task. Force the user to do this manually would underline the importance.

Except that routing is the default and expected path; piping other middleware, while a feature, is not the primary feature of zend-expressive; routing is its value-add over zend-stratigility. If you don't need routing, use zend-stratigility by itself; if you do, zend-expressive is what you want.

Regarding your approach with $app->pipeRoutingMiddleware(); I think that could do the trick. At least if it is documented and explained why I have to call the method.

Planned! I've just finished the code changes necessary, and am now moving on to documentation. Once that's done, I'll tackle the secondary issue you outlined.

Except that routing is the default and expected path; piping other middleware, while a feature, is not the primary feature of zend-expressive; routing is its value-add over zend-stratigility. If you don't need routing, use zend-stratigility by itself; if you do, zend-expressive is what you want.

This is a very important information and should be added to the README.md. However, in this case I would say the route middleware is ALWAYS piped by the factory and if you want to pipe another middleware before routing you have to call something like $app->pipePreRoutingMiddleware().

The problem I have with $app->route() is that you have to know that the router is piped automatically when calling $app-route() the first time. So it is very important that you first pipe pre routing middleware then add routes with $app->route() and as a last step pipe post routing middleware. So two times you have to pipe middleware but in between you should add your routes.
Or do I have a wrong understanding of set up process?

Addressing:

However, in this case I would say the route middleware is ALWAYS piped by the factory and if you want to pipe another middleware before routing you have to call something like $app->pipePreRoutingMiddleware().

and:

The problem I have with $app->route() is that you have to know that the router is piped automatically when calling $app-route() the first time.

You definitely understand the workflow, and it's deliberately done that way.
To re-iterate:

If creating your Application instance programmatically, pipe any middleware you want to execute before routing before you route() anything and/or before calling pipeRoutingMiddleware()
If using the Container\ApplicationFactory, pre/post middleware are added in order (see https://github.com/zendframework/zend-expressive/blob/master/src/Container/ApplicationFactory.php#L173-L175 — the changes in this PR are actually in the injectRoutes() method, so it doesn't affect that workflow).

I do not plan on adding pre/post pipe registration hooks, for one very big reason: Stratigility uses SplQueue internally, and its enqueue/dequeue methods. Since this is an internal detail used by both MiddlewarePipe and Next, it's not something we can easily modify — and, to be honest, knowing the difficulties developers have had understanding priority in ZF2's EventManager, using a priority queue is not something I want to introduce here. Since the same effect can be achieved programmatically in a trivial fashion, I'm comfortable leaving the workflow as-is.
The main point is to consider where the two different mechanisms originate:

pipe() belongs to zend-stratigility, which is a middleware foundation.
route() belongs to zend-expressive, which builds on zend-stratigility to specifically address:

using a container for lazy-loading middleware
dynamic routing of middleware (vs the static, literal-only routing in zend-stratigility)
error handling
templating

The fact that pipe() exists in zend-expressive is because we build on zend-stratigility. pipe()'s use-cases within zend-expressive should then typically be limited to:

registering error handlers
registering middleware to execute on each request
segregating middleware by sub-path

For many, these may or may not be of interest; the basic features of routing and templating may be enough (and even templating may not be of much interest in some cases; consider APIs, for example). For those where they are of interest: we support it, but in a very specific workflow. Considering routing is the central value-add of zend-expressive, then making the default use case of routing an out-of-the-box experience is the primary goal. If we force users to register the route pipeline manually when they've already added routes, then we've missed that goal.
I think the proposed solution is a reasonable trade-off; you can do advanced routing configuration if desired, and still benefit from the integrated routing in zend-expressive.

To facilitate your use case, I'm thinking that we can pipe the routeMiddleare by default if we pass a valid $router object in the AppFactory

By the way, the above is not a viable option; it means there is zero way to have middleware that will execute prior to routing.

@codeliner I think I found the source of your second report, and it's in your own code:

https://gist.github.com/codeliner/ced591bedcca422c7e42#file-index-php-L26
https://gist.github.com/codeliner/ced591bedcca422c7e42#file-index-php-L44

You're not returning!
In the first instance, you're writing to the response, but not returning it. This particular instance works because the stream in the body is mutable, and the FinalHandler detects that the body size has changed since execution began; as such, it will correctly return a 200 with the content.
However, in the second, because you do not call return $next($request, $response);, but instead $next($request, $response);, the original response passed to the middleware is what is passed to the final handler on completion of the pipeline, meaning you get a 200 with no content. Add the return statement, and it will work. (This is documented in Stratigility: https://github.com/zendframework/zend-stratigility/blob/master/doc/book/api.md#next)

@weierophinney Thank you for the detailed explanation. I still would prefer pre and post pipe hooks because IMO these are more explicit, but I can live with the trade-off ;)

You're not returning!
Oh no, shame on me :tired_face: Sorry for that.

At the moment I have the feeling that understanding EventManager priorities was easier than all this middleware piping :smile:
Maybe this will change when playing a bit more with the concept but currently I can not imagine how all this will work when adding more middleware to the pipe: oauth2, content-negotiation, handling CORS requests, serving assets, ... Sure, the easy answer is: Just pipe the middlewares in the correct order but I can imagine that this can become a complex task when I have to pipe > 10 middleware which all do something with the request and/or response. but we'll see.

@codeliner We did a prototype of Apigility in node.js last year, using middleware. Interestingly, it was far easier to:

understand the order in which operations occurred
modify that order
slip in alternate implementations

Keep playing! :smile:

@weierophinney :+1: I'll do that. Refactoring of my cargo sample from zend-mvc to zend-expressive is nearly finished. I guess people will ask me about middleware in the future so I want to be prepared :smile:
One more propblem and one more question:
The problem:
With Relay a second PSR-7 middleware dispatcher was added to the game by pmjones. So fare no problem. But now we have zend-stratigility which requires  an arity of 4 for error middlewares like sencha connect does and we have Relay which does not work with a special arity for error middlewares. If I now want to provide an error middleware (maybe something like AssetManager, listening on 404 responses and serving assets instead) I have a problem because I need to provide to types of middleware one for stratigiligty and one for relay. I think that is bad and misses the goal of interoperable middlewares. Would be nice if all PSR-7 middleware dispatcher use the same best practices.
The question
Do you think middleware can replace event-driven approaches in other areas (not request/response related areas) too? For example Tactician is a command bus which also uses middleware but of course with a totally different middleware signature. prooph/service-bus on the other hand is inspired by the event-driven zend-mvc dispatch. What I really like with the event-driven approach is that plugins can be ListenerAggregates which know the correct position in the dispatch process for themselves. So a user don't need to worry about it. I'm a bit lost with this question because on the one hand middleware seems to be easier to understand but on the other hand if every plugin system is based on middleware then users need to pipe everything and have to deal with a lot of different middleware.

@codeliner —
re:

now we have zend-stratigility which requires an arity of 4 for error middleware like sencha connect does and we have Relay which does not work with a special arity for error middleware. If I now want to provide an error middleware I have a problem because I need to provide to types of middleware one for stratigiligty and one for relay.

This is one place where I wish people would do some homework before inventing something! The reason Stratigility, and by extension Expressive, uses the "arity of 4 === error middleware" paradigm is because it's already been invented and battle-tested elsewhere (i.e., in Connect and ExpressJS).
Interestingly, Relay does not offer a concept of error middleware. If you wanted to invoke a later middleware that can handle errors, you'd have to do something like adding the error to the request attributes, which becomes a convention-based approach, vs a contract-based approach.
Slim3 offers error handling, but it's separate from normal middleware; essentially, if an exception is raised within the application, Slim passes the request, response, and exception to the registered "error_handler" service; there is no stack.
As such, likely the safest, cross-project way to handle errors is to simply... not, and let the application handle them. In the case of Stratigility/Expressive, if an exception is raised, we pass it on to error middleware, if any, and then the final handler will process it. Thus, if you don't register any error middleware, but use our final handler capabilities as detailed in the manual, you should be fine.
A group of us have been focused on identifying and categorizing the various middleware patterns, but, interestingly, error handling and error middleware has not been part of that discussion.
re:

Do you think middleware can replace event-driven approaches in other areas (not request/response related areas) too?

Yes and no. I think events still have a place — for instance, anywhere you might use a command bus, or anywhere you might be tempted to use the observer pattern. Most of the time, these will be limited to your domain layer, however — not the request/response lifecycle. (That said, Guzzle uses middleware, too, for the client-lifecycle. This allows for things like response filtering and transformation.)
Essentially I've answered your question with, "it depends." I think middleware makes a ton of sense in creating a workflow engine, which is what you need for the request/response lifecycle. That's what we're addressing with this library.

@weierophinney

This is one place where I wish people would do some homework before inventing something!

:+1:

As such, likely the safest, cross-project way to handle errors is to simply... not, and let the application handle them.

yeah, seems to be the best idea.

interestingly, error handling and error middleware has not been part of that discussion.

That is really interesting! Because good error handling should be part of every system.

Essentially I've answered your question with, "it depends." I think middleware makes a ton of sense in creating a workflow engine, which is what you need for the request/response lifecycle.

Hhm, dispatching a message with a message bus has some similarity with dispatching a request and it is definitely some kind of workflow engine. However, the event-driven approach we use currently works great. But I will keep an eye on the middleware trend. You are absolutely right. "It depends".
Thank you for answering my off topic question.


Counting sort

Write the code to perform counting sort in python.
Guidelines:

Write the codes in python only.
Refrain from any unfair means or plagiarism.
The code should display authenticity.
Use proper formatting and comments.

Hi...I would like to work on this problem. Can you please assign it to me.
Thanks :-)

@sumanta02 GO for it. I'll assign this issue to you.


Is there way to get the url from a Video('asdasd')

I have a Playlist()
But I want to get the URL out of each video...
Is there a way to do so?

I have a Playlist()
But I want to get the URL out of each video...
Is there a way to do so?

Playlist().videos will give you list of video objects ... so iterate though the list items and take item.url which will give you url for each video

Thanks.


how can I bypass wandb?

Wanted to train smoke and my setup went fine.. I tried the following and I am getting the following...
Is there a way to skip creating W&B account?
python train.py --config data_configs/smoke.yaml --epochs 100 --model fasterrcnn_resnet50_fpn --project-name smoke_training --batch-size 16
Not using distributed mode
wandb: (1) Create a W&B account
wandb: (2) Use an existing W&B account
wandb: (3) Don't visualize my results
wandb: Enter your choice:

@dev2y
Type 3 and press the Enter (return) key

Thanks.. I went ahead and created an account.. But I am now getting this error.
Is this because I am trying to train using CPU but it says device cuda below?
Not using distributed mode
wandb: Currently logged in as: dev26 (dbo). Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.13.10
wandb: Run data is saved locally in /home//rcnn/fastercnn-pytorch-training-pipeline/wandb/run-20230214_000245-fgan9pge
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run smoke_training
wandb: ⭐ View project at https://wandb.ai/dbo/fastercnn-pytorch-training-pipeline
wandb: 🚀 View run at https://wandb.ai/dbo/fastercnn-pytorch-training-pipeline/runs/fgan9pge
device cuda
Creating data loaders
Traceback (most recent call last):
  File "/home//rcnn/fastercnn-pytorch-training-pipeline/train.py", line 532, in <module>
    main(args)
  File "/fastercnn-pytorch-training-pipeline/train.py", line 248, in main
    train_sampler = RandomSampler(train_dataset)
  File "/home/rcnn/pyenv/lib/python3.10/site-packages/torch/utils/data/sampler.py", line 107, in __init__
    raise ValueError("num_samples should be a positive integer "
ValueError: num_samples should be a positive integer value, but got num_samples=0
wandb: Waiting for W&B process to finish... (failed 1). Press Control-C to abort syncing.
wandb: 🚀 View run smoke_training at: https://wandb.ai/dbo/fastercnn-pytorch-training-pipeline/runs/fgan9pge
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20230214_000245-fgan9pge/logs

@dev2y
Can you try the --disable-wandb with the train.py command and let me know?
The above looks like a version issue.

Hi @sovit-123, thank you. Paths to data in data_confg were incorrect.
Looks like I am good to train now but it is looking for graphics driver. I don't have a GPU and wondering how I can fall back to CPU.
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

@dev2y
CUDA warning will be there, but you won't face any errors. It will train as it should in the CPU.

Thank you. Will close this.


HADOOP-17229. No updation of bytes received counter value after response failure occurs in ABFS

Backport for HADOOP-17229(#2264)
Tested by: mvn -T 1C -Dparallel-tests=abfs clean verify
[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0

[INFO] Results:
[INFO]
[WARNING] Tests run: 451, Failures: 0, Errors: 0, Skipped: 71

[INFO] Results:
[INFO]
[WARNING] Tests run: 206, Failures: 0, Errors: 0, Skipped: 29

merged; closing. Thanks!

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
30m 23s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ branch-3.3 Compile Tests _

+1 :green_heart:
mvninstall
34m 54s
branch-3.3 passed

+1 :green_heart:
compile
0m 36s
branch-3.3 passed

+1 :green_heart:
checkstyle
0m 25s
branch-3.3 passed

+1 :green_heart:
mvnsite
0m 42s
branch-3.3 passed

+1 :green_heart:
shadedclient
19m 14s
branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 28s
branch-3.3 passed

+0 :ok:
spotbugs
1m  6s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
1m  3s
branch-3.3 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 36s
the patch passed

+1 :green_heart:
compile
0m 31s
the patch passed

+1 :green_heart:
javac
0m 31s
the patch passed

+1 :green_heart:
checkstyle
0m 19s
the patch passed

+1 :green_heart:
mvnsite
0m 32s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedclient
18m 14s
patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 23s
the patch passed

+1 :green_heart:
findbugs
1m  3s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
1m 24s
hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 37s
The patch does not generate ASF License warnings.

113m 10s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2283/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2283

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 123ec666bbb8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.3 / 5710005

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~16.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2283/1/testReport/

Max. process+thread count
422 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2283/1/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


Be free in target repository and path

Good idea to improve the way of publishing build artefacts in GitHub.
Would be nice to be free in configuring the target repository and the destination path (maybe also target branch).

Hi Thomas, thanks for your Feedback.
Could you elaborate what you mean by "destination path"? Do you mean where the artefacts will be stored on the destination repository?
Branches are currently mirrored from the source to the destination… I guess it could be overridden with a target branch, but then you run the risk of having code from different source branches mixed in the destination.


Separated GitHub Pages deployment workflows by branches

Production workflow deployment for main branch
Staging workflow deployment for staging branch

Checking if GitHub Pages deployment workflow separation works


code question

Hi:
In spatial_transformer.py line 131
if (self.drop_connect and self.training):
mask = torch.bernoulli((0.5) * torch.ones(B * self.Nh * V, device))
mask = mask.reshape(B, self.Nh, V).unsqueeze(2).expand(B, self.Nh, V, V)
weights = weights * mask
Why dose weight multiply mask will drop connect and avoid overfitting?

Hi!
You can give a look to this paper https://arxiv.org/abs/1907.11065. The idea is to drop attention weights which you obtained from self-attention, as standard dropout on features does, and thus regularizing the training.
Chiara


Colorize debug marker entries from VK_EXT_debug_marker

VkDebugMarkerMarkerInfoEXT structure contains a field for passing debug marker colors to a debugging application.
It would be nice if these colors would also be used in the Event browser to make debug markers a bit easier to distinguish from actual API calls :wink:

Yeh colors have been there since the original d3d9/d3d11 markers, but I've not applied them. It probably shouldn't be too hard but there's a question of how to display them - should the whole section underneath a marker (until the next one, I guess) be coloured? or just the root node?
I suspect just the root node. I'll have to experiment though :).

The "how" part is tricky I guess, especially as colors can be set to any value (including white). And white text on white background is hard to read.
Maybe it's possible to do something like the productivity power tools for Visual Studio :

And color the lines with the colors set by the debug marker?

I was thinking that the node would get its background colour from the marker colour, and the text would automatically pick a contrasting colour for visibility.
Hastily drawn poor mockup:

But this could be a configurable option depending on how many different permutations I'm willing to implement :).
To some degree I'm willing to give people enough rope to hang themselves with - if you pick alternating black and white then it will look horrible... but that's what you asked for!

Looks fine, and developers are responsible for the colors anyway :wink:
But one thing that I'm finding a bit hard (which is mostly the fault of how a tree view works) is spotting the actual scope of the debug region, and especially where it ends.
Having some kind of visual queue in there would surely help. Not sure how to implement that (or if it's actually possible with the tree view you use) but this is something I threw together in PS :

Yes that's a fair point, there should be something to show the extent of the section. I'll need to play around with it a bit, because e.g. in that example the green is slightly visible on render scene but basically invisible on the line because it's so thin.

I've done a first pass at this, here's what it looks like roughly (garish colours I chose not withstanding):

There are options to control whether colouring is enabled at all, and to control whether or not the row on the event browser or just keep the vertical bar on the side.

Perfect. Looks even better than I imagined :+1:


CLOUD FLARE JAVASCRIPT PROTECTED WEB SERVER'S

Hello Kuba, I was Testing Evilginx2, which is supposed to help me bypass 2FA on web servers by using the MITM proxy phishing attack.
I run the test on couple of web servers
I decided to try this project out with web servers using Cloud flare based JavaScript security to mitigate MITM proxy phishing techniques such Evilginx2.
Been facing problems for a while now.
This leaves my phishing domain hanging on cloud flare's DDOS protection page with the text below.
Checking your browser before accessing cloudflare-protected.site.
This process is automatic. Your browser will redirect to your requested content shortly.
Please allow up to 5 seconds…
DDoS protection by Cloud flare
Ray ID:
Anyway I can address this issue? Thank you.

Just add it as phishing domain. If you provide a demo URL, we could provide you more specific code.

Related to this issue.

@JamesCullum can you help me with this
`min_ver: '2.3.0'
proxy_hosts:

{phish_sub: '', orig_sub: '', domain: 'paxful.com', session: true, is_landing: true}
sub_filters:
{triggers_on: 'www.paxful.com', orig_sub: 'www', domain: 'paxful.com', search: 'action="https://{hostname}', replace: 'action="https://{hostname}', mimes: ['text/html', 'application/json']}
{triggers_on: 'www.paxful.com', orig_sub: 'www', domain: 'paxful.com', search: 'href="https://{hostname}', replace: 'href="https://{hostname}', mimes: ['text/html', 'application/json']}
{triggers_on: 'www.paxful.com', orig_sub: 'www', domain: 'paxful.com', search: 'src="https://{hostname}', replace: 'src="https://{hostname}', mimes: ['text/html', 'application/json']}`


Restructured level definition

I've modified the way of adding new levels into a JSON object definition.
I think a next step would be probably better the separation of levels into different files and then a live screen for level creation would be easy to implement

Looks fine to me, thanks for your additions. I'll test this out once and merge if everything works fine. Also, give me a reminder in a week or two incase I forget to merge it.

Hi @ketanhwr I wonder if you could check the refactor out?
Regards

@ketanhwr is anything going on with this? i wrote up a level class constructor that i would like to add to it thanks

Hey, I'll try to check this out in a week. Let's get this done.

It would be great :+1:


Error applying new CRD version from v0.1.0 helm chart

Bug description
On cluster where we already have old version of capsule CRD installed, when we are trying to apply new CRD version, it is throwing below error.
The CustomResourceDefinition "tenants.capsule.clastix.io" is invalid:
* spec.conversion.webhookClientConfig: Forbidden: should not be set when strategy is not set to Webhook
* spec.conversion.conversionReviewVersions: Forbidden: should not be set when strategy is not set to Webhook
Failed: NonZeroResultCode: Result code was 1

A clear and concise description of what the bug is.
How to reproduce
Steps to reproduce the behavior:

Installed older version of CRDs from v0.0.5 on a cluster
Download a new version of CRD from v0.1.0
Then run kubectl apply -f  from crds directory of helm chart

Provide the Capsule Tenant YAML definitions
Provide all managed Kubernetes resources

Expected behavior
It should create the CRDs
Logs
If applicable, please provide logs of capsule.
The CustomResourceDefinition "tenants.capsule.clastix.io" is invalid:
* spec.conversion.webhookClientConfig: Forbidden: should not be set when strategy is not set to Webhook
* spec.conversion.conversionReviewVersions: Forbidden: should not be set when strategy is not set to Webhook
Failed: NonZeroResultCode: Result code was 1

In a standard stand-alone installation of Capsule,
you'd get this by running kubectl -n capsule-system logs deploy/capsule-controller-manager.
Additional context

Capsule version: (capsule --version) v0.1.0
Helm Chart version: (helm list -n capsule-system) v0.1.0
Kubernetes version: (kubectl version)

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.11", GitCommit:"c6a2f08fc4378c5381dd948d9ad9d1080e3e6b33", GitTreeState:"clean", BuildDate:"2021-05-12T12:27:07Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.12", GitCommit:"2cb4598553d0a2ca5f3eb7980c99cd8236cc5011", GitTreeState:"clean", BuildDate:"2021-06-16T12:52:44Z", GoVersion:"go1.15.13", Compiler:"gc", Platform:"linux/amd64"}

Thanks for the bug report @viveksyngh, we're going to provide some docs on how to update Capsule in a seamless way!

Thanks @prometherion !! I don't see any spec.conversion configuration and in the helm charts CRDs. spec.conversion.strategy is also set None.  I was curious to know where this is coming from. Do you have any idea ?

Sure, the reason is how Helm manages the handling of CRDs.
Dealing with multiple versions of a CR requires a conversion webhook that is specified in the CustomResourceDefinition and, as with other webhooks, it must be backed by a CA that is managed by Capsule itself. Said so, we're dealing with an egg/chicken problem since the CA is going to be created by the Capsule manager, and this needs the installation of the CRD manifest.
We wanted to solve this with the same approach we had with patching the mutating and validating webhooks.
https://github.com/clastix/capsule/blob/7db263b2b607d47b06c5609c39b72fcf590b255f/controllers/secret/ca.go#L42-L75
Said so, the error you're reporting is odd.
https://github.com/clastix/capsule/blob/7db263b2b607d47b06c5609c39b72fcf590b255f/charts/capsule/crds/tenant-crd.yaml#L8-L10
We're providing a None conversion since the Capsule manager is going to set up it properly, whilst your Helm is saying it's a different value. Maybe we could ask for help from @MaxFedotov since he's dealing daily with Capsule in production.

@prometherion I think I know what is the problem and this approach may be going to have this problem when you upgrade the CRD. Just to clarify here, we are not using helm and helm also does not updates CRDs if they are already installed.
This is happening when we are running kubectl apply for the CRDs. This is not issue with migration from v0.0.5 to v0.1.0 but this is problem when you do kubectl apply on CRD even for the same version on v0.1.0.
The reason is when we install the CRD for the first time the ca reconciler will update the CRD with conversion section and information around the conversion webhook server.  When we do kubectl apply again and since it uses patch API, it updates the spec.conversion.strategy to None. Then this goes to api extension server validation and returns the error.
https://github.com/kubernetes/apiextensions-apiserver/blob/b0555e7aaceb203bea5409d786cb14e26be32a4e/pkg/apis/apiextensions/validation/validation.go#L437

is there a way to use cert manager to inject the CA bundle on on the conversion webhook configuration as an alternative solution to this ?

@prometherion I was using helm for update and our flow was like:

uninstall old capsule release
patch crds
install new capsule release

But I think the same can be done without helm at all.

uninstall old capsule release

@MaxFedotov you get ride off of all the tenants in this way. I think the case here is about migration.

@bsctl nope, helm doesn't uninstall crds (https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations) so it is safe

@MaxFedotov you’re right, my fault

@bsctl, i forgot one step. You have to update ownerReference to new capsule controller for each of the namespaces

is there a way to use cert manager to inject the CA bundle on on the conversion webhook configuration as an alternative solution to this ?

OperatorSDK is supporting certificates backed by CertManager: although a viable option, we wanted to manage this on our side to avoid additional dependencies for end-users.
Some hack on the manifests is required, mostly regarding annotations for the CustomResourceDefinition, Mutating and Validating Webhooks, and the Deployment.
Unluckily, I have no idea how to achieve this. We're working to solve this issue without the need of external components.

@viveksyngh I'm trying to reproduce the issue and I'm ending with a different error.

kind create cluster
helm upgrade --install capsule ./charts/capsule -n capsule-system --create-namespace (release 0.0.5)
wait for Capsule to get up and running
helm upgrade --install capsule ./charts/capsule -n capsule-system --create-namespace (release v0.1.0`

I'm not able to install Capsule from Helm due to the following issue:
Error: UPGRADE FAILED: unable to recognize "": no matches for kind "CapsuleConfiguration" in version "capsule.clastix.io/v1alpha1"

Solving this is possible by updating the CRDs as follows:
$: kubectl apply -f charts/capsule/crds
customresourcedefinition.apiextensions.k8s.io/capsuleconfigurations.capsule.clastix.io created
customresourcedefinition.apiextensions.k8s.io/tenants.capsule.clastix.io configured

Now, issuing the helm upgrade I'm able to update Capsule to the latest version without encountering errors.
{"level":"info","ts":"2021-09-30T19:52:48.072Z","logger":"setup","msg":"Capsule Version v0.1.0 7979c25"}
{"level":"info","ts":"2021-09-30T19:52:48.072Z","logger":"setup","msg":"Build from: https://github.com/clastix/capsule"}
{"level":"info","ts":"2021-09-30T19:52:48.072Z","logger":"setup","msg":"Build date: 2021-08-23T15:09:36"}
{"level":"info","ts":"2021-09-30T19:52:48.072Z","logger":"setup","msg":"Go Version: go1.16.7"}
{"level":"info","ts":"2021-09-30T19:52:48.072Z","logger":"setup","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":"2021-09-30T19:52:48.730Z","logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":"2021-09-30T19:52:48.733Z","logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, admission.Defaulter interface is not implemented","GVK":"capsule.clastix.io/v1alpha1, Kind=Tenant"}
{"level":"info","ts":"2021-09-30T19:52:48.733Z","logger":"controller-runtime.builder","msg":"skip registering a validating webhook, admission.Validator interface is not implemented","GVK":"capsule.clastix.io/v1alpha1, Kind=Tenant"}
{"level":"info","ts":"2021-09-30T19:52:48.734Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/convert"}
{"level":"info","ts":"2021-09-30T19:52:48.734Z","logger":"controller-runtime.builder","msg":"conversion webhook enabled","object":{"name":""}}
{"level":"info","ts":"2021-09-30T19:52:48.734Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/pods"}
{"level":"info","ts":"2021-09-30T19:52:48.736Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/namespaces"}
{"level":"info","ts":"2021-09-30T19:52:48.736Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/ingresses"}
{"level":"info","ts":"2021-09-30T19:52:48.736Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/persistentvolumeclaims"}
{"level":"info","ts":"2021-09-30T19:52:48.737Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/services"}
{"level":"info","ts":"2021-09-30T19:52:48.737Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/networkpolicies"}
{"level":"info","ts":"2021-09-30T19:52:48.737Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/tenants"}
{"level":"info","ts":"2021-09-30T19:52:48.737Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/namespace-owner-reference"}
{"level":"info","ts":"2021-09-30T19:52:48.738Z","logger":"controller-runtime.webhook","msg":"registering webhook","path":"/cordoning"}
{"level":"info","ts":"2021-09-30T19:52:48.740Z","logger":"setup","msg":"starting manager"}
I0930 19:53:04.867128       1 leaderelection.go:258] successfully acquired lease capsule-system/42c733ea.clastix.capsule.io
{"level":"debug","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.events","msg":"Normal","object":{"kind":"ConfigMap","namespace":"capsule-system","name":"42c733ea.clastix.capsule.io","uid":"43d8d5ff-6d2c-42e1-9251-8615d4a42424","apiVersion":"v1","resourceVersion":"2656"},"reason":"LeaderElection","message":"capsule-controller-manager-7f8997cc96-zg2p8_75868268-6f93-4521-b308-7355f7f783a2 became leader"}
{"level":"debug","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.events","msg":"Normal","object":{"kind":"Lease","namespace":"capsule-system","name":"42c733ea.clastix.capsule.io","uid":"7fa8b4c4-0427-44f6-bb7c-ce6daa2edb5f","apiVersion":"coordination.k8s.io/v1","resourceVersion":"2657"},"reason":"LeaderElection","message":"capsule-controller-manager-7f8997cc96-zg2p8_75868268-6f93-4521-b308-7355f7f783a2 became leader"}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.capsuleconfiguration","msg":"Starting EventSource","reconciler group":"capsule.clastix.io","reconciler kind":"CapsuleConfiguration","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.capsuleconfiguration","msg":"Starting Controller","reconciler group":"capsule.clastix.io","reconciler kind":"CapsuleConfiguration"}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.tenant","msg":"Starting EventSource","reconciler group":"capsule.clastix.io","reconciler kind":"Tenant","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.tenant","msg":"Starting EventSource","reconciler group":"capsule.clastix.io","reconciler kind":"Tenant","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.tenant","msg":"Starting EventSource","reconciler group":"capsule.clastix.io","reconciler kind":"Tenant","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.tenant","msg":"Starting EventSource","reconciler group":"capsule.clastix.io","reconciler kind":"Tenant","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.tenant","msg":"Starting EventSource","reconciler group":"capsule.clastix.io","reconciler kind":"Tenant","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.tenant","msg":"Starting EventSource","reconciler group":"capsule.clastix.io","reconciler kind":"Tenant","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controller-runtime.manager.controller.tenant","msg":"Starting Controller","reconciler group":"capsule.clastix.io","reconciler kind":"Tenant"}
{"level":"info","ts":"2021-09-30T19:53:04.867Z","logger":"controllers.Rbac","msg":"setting up ClusterRoles","ClusterRole":"capsule-namespace-provisioner"}
{"level":"info","ts":"2021-09-30T19:53:04.868Z","logger":"controller-runtime.manager.controller.clusterrole","msg":"Starting EventSource","reconciler group":"rbac.authorization.k8s.io","reconciler kind":"ClusterRole","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.868Z","logger":"controller-runtime.manager.controller.clusterrole","msg":"Starting Controller","reconciler group":"rbac.authorization.k8s.io","reconciler kind":"ClusterRole"}
{"level":"info","ts":"2021-09-30T19:53:04.869Z","logger":"controller-runtime.manager.controller.clusterrolebinding","msg":"Starting EventSource","reconciler group":"rbac.authorization.k8s.io","reconciler kind":"ClusterRoleBinding","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.869Z","logger":"controller-runtime.manager.controller.clusterrolebinding","msg":"Starting EventSource","reconciler group":"rbac.authorization.k8s.io","reconciler kind":"ClusterRoleBinding","source":{}}
{"level":"info","ts":"2021-09-30T19:53:04.869Z","logger":"controller-runtime.manager.controller.clusterrolebinding","msg":"Starting Controller","reconciler group":"rbac.authorization.k8s.io","reconciler kind":"ClusterRoleBinding"}
{"level":"info","ts":"2021-09-30T19:53:04.869Z","logger":"controller-runtime.manager.controller.secret","msg":"Starting EventSource","reconciler group":"","reconciler kind":"Secret","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.870Z","logger":"controller-runtime.manager.controller.secret","msg":"Starting Controller","reconciler group":"","reconciler kind":"Secret"}
{"level":"info","ts":"2021-09-30T19:53:04.870Z","logger":"controller-runtime.manager.controller.secret","msg":"Starting EventSource","reconciler group":"","reconciler kind":"Secret","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.870Z","logger":"controller-runtime.manager.controller.secret","msg":"Starting Controller","reconciler group":"","reconciler kind":"Secret"}
{"level":"info","ts":"2021-09-30T19:53:04.870Z","logger":"controller-runtime.manager.controller.service","msg":"Starting EventSource","reconciler group":"","reconciler kind":"Service","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.870Z","logger":"controller-runtime.manager.controller.service","msg":"Starting Controller","reconciler group":"","reconciler kind":"Service"}
{"level":"info","ts":"2021-09-30T19:53:04.870Z","logger":"controller-runtime.manager.controller.endpoints","msg":"Starting EventSource","reconciler group":"","reconciler kind":"Endpoints","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.870Z","logger":"controller-runtime.manager.controller.endpoints","msg":"Starting Controller","reconciler group":"","reconciler kind":"Endpoints"}
{"level":"info","ts":"2021-09-30T19:53:04.871Z","logger":"controller-runtime.manager.controller.endpointslice","msg":"Starting EventSource","reconciler group":"discovery.k8s.io","reconciler kind":"EndpointSlice","source":"kind source: /, Kind="}
{"level":"info","ts":"2021-09-30T19:53:04.871Z","logger":"controller-runtime.manager.controller.endpointslice","msg":"Starting Controller","reconciler group":"discovery.k8s.io","reconciler kind":"EndpointSlice"}
{"level":"info","ts":"2021-09-30T19:53:04.973Z","logger":"controller-runtime.manager.controller.endpointslice","msg":"Starting workers","reconciler group":"discovery.k8s.io","reconciler kind":"EndpointSlice","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:04.973Z","logger":"controllers.Rbac","msg":"setting up ClusterRoles","ClusterRole":"capsule-namespace-deleter"}
{"level":"info","ts":"2021-09-30T19:53:04.973Z","logger":"controllers.Rbac","msg":"setting up ClusterRoleBindings"}
{"level":"info","ts":"2021-09-30T19:53:04.973Z","logger":"controller-runtime.manager.controller.endpoints","msg":"Starting workers","reconciler group":"","reconciler kind":"Endpoints","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:04.974Z","logger":"controller-runtime.manager.controller.service","msg":"Starting workers","reconciler group":"","reconciler kind":"Service","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:04.974Z","logger":"controller-runtime.manager.controller.secret","msg":"Starting workers","reconciler group":"","reconciler kind":"Secret","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:04.974Z","logger":"controllers.Tls","msg":"Reconciling TLS Secret","Request.Namespace":"capsule-system","Request.Name":"capsule-tls"}
{"level":"info","ts":"2021-09-30T19:53:05.063Z","logger":"controller-runtime.manager.controller.secret","msg":"Starting workers","reconciler group":"","reconciler kind":"Secret","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:05.063Z","logger":"controllers.CA","msg":"Reconciling CA Secret","Request.Namespace":"capsule-system","Request.Name":"capsule-ca"}
{"level":"info","ts":"2021-09-30T19:53:05.064Z","logger":"controllers.CA","msg":"Handling CA Secret","Request.Namespace":"capsule-system","Request.Name":"capsule-ca"}
{"level":"info","ts":"2021-09-30T19:53:05.064Z","logger":"controllers.CA","msg":"Updating CA secret with new PEM and RSA","Request.Namespace":"capsule-system","Request.Name":"capsule-ca"}
{"level":"info","ts":"2021-09-30T19:53:05.065Z","logger":"controllers.Tls","msg":"Reconciliation completed, processing back in 4319h59m25.02297535s","Request.Namespace":"capsule-system","Request.Name":"capsule-tls"}
{"level":"info","ts":"2021-09-30T19:53:05.069Z","logger":"controller-runtime.manager.controller.capsuleconfiguration","msg":"Starting workers","reconciler group":"capsule.clastix.io","reconciler kind":"CapsuleConfiguration","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:05.073Z","logger":"controllers.CapsuleConfiguration","msg":"CapsuleConfiguration reconciliation started","request.name":"default"}
{"level":"info","ts":"2021-09-30T19:53:05.073Z","logger":"controller-runtime.manager.controller.tenant","msg":"Starting workers","reconciler group":"capsule.clastix.io","reconciler kind":"Tenant","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:05.073Z","logger":"controller-runtime.manager.controller.clusterrole","msg":"Starting workers","reconciler group":"rbac.authorization.k8s.io","reconciler kind":"ClusterRole","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:05.073Z","logger":"controller-runtime.manager.controller.clusterrolebinding","msg":"Starting workers","reconciler group":"rbac.authorization.k8s.io","reconciler kind":"ClusterRoleBinding","worker count":1}
{"level":"info","ts":"2021-09-30T19:53:05.075Z","logger":"controllers.CapsuleConfiguration","msg":"CapsuleConfiguration reconciliation finished","request.name":"default"}
{"level":"info","ts":"2021-09-30T19:53:05.380Z","logger":"controllers.CA","msg":"Reconciliation completed, processing back in 87647h59m0s","Request.Namespace":"capsule-system","Request.Name":"capsule-ca"}

I double-checked also the content of the CA bundle required in the CRD for the API conversion and it's the correct one:
$: k get customresourcedefinitions.apiextensions.k8s.io tenants.capsule.clastix.io -o jsonpath="{.spec.conversion.webhook.clientConfig.caBundle}" | base64 -d
-----BEGIN CERTIFICATE-----
MIIFwTCCA6mgAwIBAgICB+MwDQYJKoZIhvcNAQELBQAwcjELMAkGA1UEBhMCVUsx
CTAHBgNVBAgTADEPMA0GA1UEBxMGTG9uZG9uMSIwIAYDVQQJExkyNywgT2xkIEds
b3VjZXN0ZXIgU3RyZWV0MREwDwYDVQQREwhXQzFOIDNBWDEQMA4GA1UEChMHQ2xh
c3RpeDAeFw0yMTA5MzAxOTUyMDVaFw0zMTA5MzAxOTUyMDVaMHIxCzAJBgNVBAYT
AlVLMQkwBwYDVQQIEwAxDzANBgNVBAcTBkxvbmRvbjEiMCAGA1UECRMZMjcsIE9s
ZCBHbG91Y2VzdGVyIFN0cmVldDERMA8GA1UEERMIV0MxTiAzQVgxEDAOBgNVBAoT
B0NsYXN0aXgwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDbovLbFwCI
m5+Xiv+PrMclqLtMRudD5wsybu5TT/M0Y0hi97WAcEkz3eMgDnilYYKEhsAnr3h9
c/76ZoQVx/HEbPUBCOFyPyFuE053n3qdN1Mp5kZly/Bb5uEaKsvF0CaLH1xSddxL
wMi+bj2m52Vz4TFjIXJvJbR80ltttpNM4dbUg2RiFqk3swp37vvcVJmWG1llbzM0
E9xAO6PryUcF/1WvDxe28sjseVTTkrhpt00PQvupJ9EqKMzAAsPlgrGaF4PVw4ws
fvkiptiFOeCk2YvSWQLEkRiOJR2LbG/Wgh1iFpO6m+0bQeay1nZC176s2s+xV+Ho
z+Fq1DPO9sMgNG7Et5QQQ1fhSEb7/Ha/FHemHMhoHxkXCfqVPS9eVn8e8WEtgGPJ
btkHXuQh+14bnKUBX9dE/beTi5b9CdSiFw/U7h86d+R33mhwKz+uXbfEPFBGxRH4
7LooToxiezrFbKfYtCLcXxh4gwh5sEy4ZGEvL2amzRVOvxttmdqvFX3tXbK+a9L2
WHKX7GTIrRlp0/Q+w8XKXzN4jKwxp8ddrzBdkr+irriDUo5aMXJz6Ga8vpGlCptz
RM0vgNPhlCrTxPHZbMXseqcBlM3erfDBXb/+aDEtrGJvJirV7bDEEAeELSHxrlRU
9dbu2CbZ1zlapy9xXXbawAK2tszbtYsGdwIDAQABo2EwXzAOBgNVHQ8BAf8EBAMC
AoQwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMA8GA1UdEwEB/wQFMAMB
Af8wHQYDVR0OBBYEFAEStxEz0/d/NX7rDL5mWqo6NOu+MA0GCSqGSIb3DQEBCwUA
A4ICAQBoREip9dEstQKWwXz+FcumvHlGT2UMZRfqtLRJKzv/Cf/tGKcxEv0sKU2F
8Ia3Y9y3loOvC/lvb4qXLjQt8bSH5z6+x+HBo8lPxxGte/jbrtYpEpkJYw/ETlX4
FdryHbm0Kj63kbSo03+cbEg9nDve1ANDDogxrMTCdW4cVWNDwsocM6S/dg3L06Bv
pA7XEsvkCOO71sehrZO5abQHT2NXr9KodnRCm6C2iVFX1IHFcTUBN02dXsItt5Qn
DuY5sUJpWkbSV+I535+qEWg8lxxQKhevYJ+mf0uXhFFtL9uX5Gn6FXnQ08Hdf5KA
xjYAjEN7EAr5xlmWx2HBqva5vx4/S5jN6OtS2ghEe6JndIi5U6j2Yoe1SXIPLU6W
QiJhlixBO3N4JXDLe22KaK0foVne3WbgrGwN25YIyAyJrs5Fj68XJLSf0XYEEG+J
JcaFJ6uptXlk/a//VK8R8mpOI/Q9Y7QzGm0oJbzSuV5LR/eazQEKoLecuMEEX6Sq
orFyrV+dPCxKw0irRG8rnmCiYL5D07eYSwUISaWK9q19tL+ZWNMReD4Iwrt31lS7
IrRkVPvXbIgEuOE/fUkYRSMzi+JOwDuIsxImuKMwMp/T3yTDzcMV23kq3+6BZrVG
JLFW0R3XLiZfeuSVHbxWePa4tPI0Vas7ISyZcqIi2uDV0x0K5g==
-----END CERTIFICATE-----

All suggestions provided by @MaxFedotov and @bsctl on upgrading Capsule (like scaling down the deployment to avoid writing corrupted files on etcd during the update procedure) are still valid: would be great having a step-by-step to reproduce your error to write the best documentation as possible.

Closing due to inactivity, feel free to open it back.

@prometherion i think we had to reopen this issue. When migrating capsule under argocd, we start getting the following errors during sync:

The problem is related to https://github.com/clastix/capsule/blob/master/charts/capsule/crds/tenant-crd.yaml#L10 this line. ArgoCD tries to run kubectl apply with this CRD, and it fails. Moreover, ignoring these fields in the application spec doesn't work:
  - group: apiextensions.k8s.io
    kind: CustomResourceDefinition
    jsonPointers:
      - /spec/conversion
      - /spec/conversion/strategy
      - /spec/conversion/webhook
      - /spec/conversion/webhook/conversionReviewVersions

So my proposal is to change tenants crd spec to the following:
spec:
  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        service:
          name: capsule-webhook-service
          namespace: capsule-system
          path: /convert
          port: 443
      conversionReviewVersions:
        - v1alpha1
        - v1beta1

so Capsule controller will update only caBundle for clientConfig.
This will allow to ignore it in argocd using:
  - group: apiextensions.k8s.io
    kind: CustomResourceDefinition
    jsonPointers:
      - /spec/conversion/webhook

We cannot address this with release v0.1.1 since this should be considered as a breaking change, since this would require a manual intervention by the cluster administrator to update the CRD specification.
WDYT?

I think that won't be a breaking change, because the actual crd would be the same (as capsule controller already patched it :)). This change will affect only new users or argocd users.

I think you're right.
Could you take care of this @MaxFedotov by submitting a PR? I was planning to release the v0.1.1 release for the first days of 2022, would be great to get this landing into the upcoming version, too.

Yes, no problem. We already built a helm chart with this fix and verified that everything is working in argoCD


pcsc-cyberjack: minor cleanups

Previously, pulling in libusb meant that you would get libusb1 as well as it was
propagated by libusb which meant that derivations incorrectly depending on
libusb when they should be depending on libusb1 would work.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/update-error-usbdev-c3-error-error-neither-hal-nor-usb1-found/6638/4

@aszlig already fixed this, so this PR is now only the minor cleanups that were included as part of my fix.

@aszlig - just some minor cleanups left. Instead of trying to be clever with the version, just be specific. It's cleaner and easier to read.


Procedural Macro - Add placeholder lifetime for std::fmt::Formatter

To resolve #36 the placeholder lifetime for std::fmt::Formatter has been added in markup-proc-macro/src/generate.rs.
This PR also removes a unnecessary path prefix (syn) in the attr function, which is also located in markup-proc-macro/src/generate.rs.

Thanks!


Remove dependency on boost histogram package?

The issue with boost histograms is worrying.
Since weeks we cannot generate a M1 docker image because of issues in this package (see PR and failing tests). I don't think we should have to deal with compilation errors of packages.
The dependency on boost_histograms is light and my suggestion is to replace the corresponding code in simtools/corsika/corsika_histograms.py by numpy.histograms.

Which functionality is it which we cannot do with e.g. numpy histograms? It seems we are not using a lot of functions from boost.


pdfsak fails with Ghostscript >= 9.56.1

Describe the bug
When invoked on Linux, pdfsak fails altogether with an error about page numbering failure. The error message is not well-formatted.
To Reproduce
pdfsak -t '$page' br 1 1 -if input.pdf -o output.pdf

Expected behavior
Expected pdfsak to page number my file.

This is due to the new PDF interpreter in Ghostscript 9.56.1.
It doesn't work for me even with your PR:
$ ../../../pdfsak --input-file ../../input/article1.pdf --output ../../output/addtext.pdf -t "$page" tl 0 0 --overwrite
ERROR: Cannot get page count.
 cmd: ['gs', '-q', '-dNOSAFER', '-dNODISPLAY', '-c', '(/mnt/dataint/data/progetti_miei/pdfsak/examples/input/article1.pdf) (r) file runpdfbegin pdfpagecount = quit']
stdout: 32

stderr: free(): double free detected in tcache 2

As a temporary workaround downgrade Ghostscript to 9.55

Yes, your PR allows PDFsak to work with Ghoscript >= 9.56.
The additional error is due to -dNODISPLAY, and is most likely a bug of Ghostscript 9.56.1. I opened a bug report, I'm gonna remove that option for the time being.
Thank you very much for this PR!
I will release a new version soon.


Convert all existing models to Literate

After this PR, all files in models contain \literate{/scripts/...}.

Difficult to review in this format. I prefer to check it using the website.

As far as I know, there are a few ways to make that possible:

I push directly to master and we fix from there. This might be a bit unwieldy when, at a later point in time, people start sending in PRs which we have to check.
In the GitHub Action, push pull requests to a different folder (like PR), so that https://statisticalrethinkingjulia.github.io/TuringModels.jl/PR/ would show the site for the latest pull-request. This option is quite easy to achieve but multiple pull requests will then overlap.
Like the previous one but send the PRs to different folders in the gh-pages branch.
Usually, Netlify is the best option for showing pull request web pages. However, I think we would run out of build minutes there and I'm not sure whether the build image includes Julia. I've been messing with their build image before and noticed that you don't have sudo permissions.

Difficult to review in this format. I prefer to check it using the website.

As far as I know, there are a few ways to make that possible:

I push directly to master and we fix from there. This might be a bit unwieldy when, at a later point in time, people start sending in PRs which we have to check.
In the GitHub Action, push pull requests to a different folder (like PR), so that https://statisticalrethinkingjulia.github.io/TuringModels.jl/PR/ would show the site for the latest pull-request. This option is quite easy to achieve but multiple pull requests will then overlap.
Like the previous one but send the PRs to different folders in the gh-pages branch.
Usually, Netlify is the best option for showing pull request web pages. However, I think we would run out of build minutes there and I'm not sure whether the build image includes Julia. I've been messing with their build image before and noticed that you don't have sudo permissions.

The model is unfinished by the way, which I try to indicate with the Draft label. I thought it would be easier this way to see what work is in progress.

The model is unfinished by the way, which I try to indicate with the Draft label. I thought it would be easier this way to see what work is in progress.

Related discussions at https://github.com/travis-ci/dpl/issues/991, https://github.com/bids-standard/bids-website/issues/76 and https://github.com/bids-standard/bids-website/issues/81.

Related discussions at https://github.com/travis-ci/dpl/issues/991, https://github.com/bids-standard/bids-website/issues/76 and https://github.com/bids-standard/bids-website/issues/81.


Adjust ContentLoader color

Motivation
RN I think it's looking too dark in light theme, and too light in dark theme. we should make it more discrete
https://user-images.githubusercontent.com/8636507/235757611-11fac62c-28b1-4059-a969-2fdc32717411.mov
Usage example
No response
Possible implementations
No response

To add on to this the generic intentsBaseX variables do not work when passed in.  Only the darkColors or lightColors variables work here, but it would be nice if the generic colors worked as well.


simplify dynamic container setup

instead of explicitly finding the next available dynamic port,
and then forcing the dynamic container to use that one, just
make that container instance relax its requirements.  This
should end up allocating the same port as before in the normal
case, but works better when reusing previous port allocations.

@bratseth please review
@gjoranv @hmusum FYI

@bratseth please take a look - this is really three different things, but the second commit depends on the first.


ZXingScannerViewRenderer was causing the app to crash if the view was inside a ContentView

If the ZXingScannerView was inside of a ContentView instead of a ContentPage, the application would crash by throwing a Java.Lang.NullPointerException. The Context passed to ZXingScannerViewRenderer via the constructor was actually a ContextThemeWrapper which does not inherit from Activity. Therefore when it was cast in the call of the constructor of ZXingScannerView in the renderer on Android, a null would be passed for a context instead.
Fixes: #943

I experience a similar issue. The ZXingScannerView is inside a ScrollView in a ContentPage. The application crashes for the reason described above.
The issue does not occur, when the ScrollView is removed.
The bug fix proposed above fixes the issue, although I would recommend zxingSurface = new ZXingSurfaceView(Context.GetActivity(), formsView.Options); rather than the proposed one.


ScrollAreas can overrun their container's margins

Describe the bug
When displaying a ScrollArea inside a container, and it only scrolls along one axis, it will be a few pixels too big and will overflow into that container's margins along that axis.
This is easiest to see when we set the container's margins to 0. Here's an example with a ScrollArea containing the grey rectangle in the central panel, surrounded by 4 panels:

I haven't delved into the code much, but it seems like the scroll area overruns the container margin by clip_rect_margin pixels. Setting clip_rect_margin to 0.0 before rendering the scroll area fixes the issue.
To Reproduce
Steps to reproduce the behavior:
See the following demo code, which renders the UI shown above:

Demo code
```rust
use eframe::egui;
fn main() -> Result<(), eframe::Error> {
let options = eframe::NativeOptions {
initial_window_size: Some(egui::vec2(480.0, 360.0)),
..Default::default()
};
eframe::run_native(
"My egui App",
options,
Box::new(|_cc| Box::::default()),
)
}
#[derive(Default)]
struct MyApp {}
impl eframe::App for MyApp {
fn update(&mut self, ctx: &egui::Context, frame: &mut eframe::Frame) {
let no_margins = egui::Frame::side_top_panel(&ctx.style()).inner_margin(0.0);
egui::TopBottomPanel::top("top").show(ctx, || {});
egui::TopBottomPanel::bottom("bottom").show(ctx, || {});
egui::SidePanel::left("left").show(ctx, || {});
egui::SidePanel::right("right").show(ctx, |_| {});
egui::CentralPanel::default()
.frame(no_margins)
.show(ctx, |ui| {
egui::ScrollArea::new([true; 2]).show(ui, |ui| {
ui.add(
egui::Image::from_texture((
egui::TextureId::default(),
egui::vec2(360.0, 240.0),
))
.uv(egui::Rect::from_min_max(
egui::pos2(0.0, 0.0),
egui::pos2(0.0, 0.0),
))
.tint(egui::Color32::GRAY),
);
});
});
}
}
</details>

**Expected behavior**
The ScrollArea should properly resize to fit within its container.

**Screenshots**
See GIF above.

I came up with a tentative fix that just doesn't expand the inner clip rect by clip_rect_margin but it has some side effects. For instance, the current codebase just so happens to almost cancel out the inner frame's margin (in the default style) with clip_rect_margin. This means that scroll areas' contents are mostly visible even around the margins. My fix would change that behavior to just clip the scroll area:

Current
Proposed fix

This matches how e.g. the CSS padding property works.
It seems like clip_rect_margin has snaked its way around the codebase, and its default value of 3.0 is close enough to many default-style spacing values to conceal many visual hiccups. I would try to remove it, but I have no idea what the intended visuals are.

clip_rect_margin is an ugly hack, and I'd like to get rid of it.
The reason I added it is because a lot of widgets draw their border outline at their margin, and so if the border is e.g. 2 points wide, half of the outline is outside the widget rect. This means that if you put such a widget first thing inside e.g. a ScrollArea, you would see a clipped button, unless there was some extra margin somewhere.
I think there are a few ways to fix this mess:

Adjust the border logic of widgets and Frame to always paint within the lines instead
Add an margin to the contents of a ScrollArea, so that the first widget does not start at the top-left (0,0) coordinate, but e.g. at (3,3)

Painting the borders inside the lines seems like the best solution to me as well.
It seems like egui doesn't currently count borders as taking up any space. This would make them more similar to the CSS outline property than a CSS border. This is good because changing the borders to paint inside the lines won't have a bunch of subtle layout implications like I originally thought.
It seems like most of the current use cases for clip_rect_margin outside of ScrollArea are for elements like Area and Resize which clip their contents then immediately create a Frame as their only child element. I haven't looked into whether those two can have a size which isn't known until they render its contents, but:

If they have a size that isn't fully determined before adding their contents, then it seems like clipping doesn't really do much? You can clip the top and left sides, but the bottom and right could be significantly further out than expected. In this case, could we just stop clipping entirely?
If they have a size that is fully determined, then we may be able to separate the styling of Frame from the sizing. The elements that previously used Frame could just paint the Rect themselves since they know the size. This would allow them to precisely control clipping.

Hm, it seems like a lot of widgets draw outside their rectangle. Resize, for instance, expands its bounds by a hardcoded 2 units to give the content some "breathing room". And basically every widget draws a little bit outside a Frame's inner_margin.
For example, the Window element uses one big frame for both its title bar and contents (which does not clip), then places a Resize inside that frame for its contents only (which does clip).
I tried removing clip_rect_margin from Resize, but this leads to window contents being clipped (see the slider being hovered):

On the other hand, if I just stop Resize from clipping entirely, then windows' contents can overflow (see the central panel on the left window):

It seems like the right place for clipping to happen is Frame, but as the commented-out Frame clipping code from 3 years ago says, it doesn't know how big its contents will be until after they're drawn.
Maybe instead of having one hardcoded clip_rect_margin, a Frame could propagate its inner margin downwards to its children, which could then expand their clip rectangles by that margin. Alternatively, as you suggested, container elements like ScrollArea and Resize which clip their contents could have their own inner margin. This would require rewriting a lot of Window's layout code, however.

I may tackle this when I start working on https://github.com/emilk/egui/issues/3284

I'm thinking about this once again, and I think I've convinced myself that the real issue has been ScrollArea the entire time.
In the HTML/CSS world, you have one primitive: the box. It has a border, margin, and padding, and you can make it scrollable.
In egui, if you want a scrollable container, you have to put a ScrollArea inside an existing container (like a Panel). There is no way for that Panel to "tell" the ScrollArea what inner margin it has (and transfer it over)--what if that ScrollArea isn't the only child of the Panel? So instead, there's clip_rect_margin, which approximates that functionality if you don't look too closely.
There's also resizability--as far as I can tell, the Resize container has two disparate functionalities. It can be a widget that you can resize by dragging it with a mouse, or it can be a primitive for building your own resizable layouts by setting resizable to false and controlling it manually, as Window does. In fact, it seems like Window overrides a lot of Resize's behavior--what purpose does Resize serve there?
I think that it's better to conceptualize "scrollable" and "resizable" as attributes of a box/layout, rather than their own distinct widgets. Window already does this--it has the resizable and scroll options.
From there, you would need to rework the container widgets' APIs with this in mind. I think the best way would be to create a struct that contains both the Frame and scroll/resize state (something like Box). Containers like Panel can then expose the functionality that makes sense to support.
What are your thoughts on this API? I may try to implement it--I will probably fail, but I think it's worth exploring.


Creating a Way to Pause Workers from Grabbing New Jobs

I'm currently using django-rq on Heroku.  On Heroku, workers don't finish their current tasks when a shut down signal is received.   Instead, they run for 10 seconds, and then terminate mid process.
We're currently working around this by waiting until all workers are idle before deploying new code (deploying new code causes workers to reboot).
However, one thing that I think would be immensely helpful would be to have a way to pause workers from grabbing new jobs.  Even though we currently wait for queues to clear out before deploying, if a long-running task happens to be enqeued WHILE deploying, it gets killed.

Hello @boxbeatsy, the behavior you describe in Heroku is similar with the Docker Engine.
First the container gets a SIGSTOP, if the container don't terminate in 10 sec, a SIGTERM is sent to the process.
The rqworker works a little different, your first gets a SIGSTOP/SIGQUIT(ctrl+c) that triggers a worm shutdown, which will wait the task finishes. If a new SIGSTOP/SIGQUIT is sent a cold shutdown begins, killing the process and failing the task.
I do not have experience with Heroku, but I think it fits in the first case.
And about a way t pause the workers, it's not entire necessarily once you can use the worm shutdown. Maybe there is a way to send Linux signals to your processes on Heroku and fix the behavior.

Hi @mbodock, thanks for the input.  I believe that Heroku handles the SIGTERM signal differently which doesn't give the worker the ability to complete the task at hand: https://github.com/nvie/rq/issues/584#issuecomment-217116188
That is, I don't believe the worm shutdown is honored on Heroku.  Please let me know if I'm missing something though!

RQ actually already has an "rq suspend" command which is not documented. This command pauses job execution so you can pause workers before shutting it down in Heroku. We should make a management command to wrap this command in Django-RQ.
PR welcome 😃


Data for the Abort API is wrong

https://platform-status.mozilla.org/#abort-api marks the feature as landed in firefox, and no signals in any other browsers. But according to https://www.chromestatus.com/features/5631483679080448 and https://developer.mozilla.org/en-US/docs/Web/API/AbortController#Browser_compatibility it is already shipped in all browsers.

We've updated the platform-status app, so at this point you should see a change in the data. It still says no signals for Edge, but the MDN page contradicts this. I can't find an entry about it in dev.windows.com, but I will keep investigating this.
Thanks for reporting!

Yeah, I was not able to find the ie_ref for it when updating the info.
Caniuse and BCD are both marking it as shipped in Edge, but the Caniuse info is not used to mark it as landed, and BCD is not used at all (that's #534)


[docs] Fixed React.forwardRef missing display name ESLint error

Fixes #25975

[x] I have followed (at least) the PR section of the contributing guide.

@arpitBhalla you need to run
yarn prettier
yarn docs:typescript:formatted

for solving the CI errors. Otherwise the changes look good 👍

@mnajdova Still failing 😟

@mnajdova Still failing

Generally, you should work on issues in a new branch not the default branch. That way we can easier determine what files changed. Since you just committed the changes to your next branch, we can't determine the changed files which is why yarn prettier did not format every file.
Our contributing guide describes the  pull request workflow in more detail.
I went ahead and fixed the formatting  and typing issues

@eps1lon I am sorry

@eps1lon I am sorry

No need to apologize. That's why we have all these automated checks so that we can work on these issues before they become a problem.

@arpitBhalla  It's a great first pull request on Material-UI 👌🏻. Thank you for working on it!

Thank you @eps1lon  @oliviertassinari


fix(es/modules): Fix re-assignment of the export variable in export default function

Description:

The swc_ecma_transforms_modules crate do not transform the following code as expected：
Input
export var format;
export function defaultLocale(definition) {
  locale = formatLocale(definition);
  format = locale.format;
  return locale;
}
format = '123';

The expected generated code as below, which is the same as the output of babel:
Expected Output
"use strict";
Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.default = defaultLocale;
exports.format = void 0;
function defaultLocale(definition) {
    locale = formatLocale(definition);
    exports.format = format = locale.format;
    return locale;
}
var format;
exports.format = format;
exports.format = format = '123';

the re-assignment of format in defaultLocale should be transformed to exports.format = format = local.format. current output:
Current Unexpected Output
"use strict";
Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.default = defaultLocale;
exports.format = void 0;
function defaultLocale(definition) {
    locale = formatLocale(definition);
    format = locale.format;
    return locale;
}
var format;
exports.format = format;
exports.format = format = '123';

A test case is added for this case.
BREAKING CHANGE:

Related issue (if exists):

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.wumingliang.0113 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

@kdy1 updated

The Performance regression check failed due to Network issue. Could you rerun it, thank you


Ui dialog callkit incoming

Text mobile (123456789), avatar, button decline and accept is partly hidden below. I use device oppo and google pixel. Please check image bellow and fix it. Thanks you very much.

I met the same issue also. Please fix it asap.
My device: Google Pixel 4 - Android 13.
Version SDK: flutter_callkit_incoming: ^2.0.0+2.


Document units

It would be very helpful if we could document the units of P/I/D and also what the defaults are (P of 0 appears to be ∞, not 0 for example)
https://www.youtube.com/watch?v=IB1Ir4oCP5k

In PID there are no defaults! The values are the bases of the PID system, and it's value is the part that they take on the system.
I'm working on a auto runner, that should be OK for determination of the bases of the values. It's still a work in progress.

Are the i and d seconds as in the example or measurement points?

Hi,
I think @hannut has a legitimate question here, I was asking myself. As I want to control my central heating, the P and Ds are more in the range of an hour or two. So I tried to set them as seconds here and did not connect the output to my target flow temperature (I hope that's the correct english word for the German "Vorlauftemperatur"). So the controller does not actually set the value. Then this happened:

Those are my settings:
platform: pid_controller
name: ETA PID Vorlauf Controller

set_point: "{{state_attr('climate.wohnzimmer_master', 'temperature') | float}}"
entity_id: sensor.ts_wohnzimmer_temperature
precision: 1
sample_time: 60
maximum: 70
unit_of_measurement: °C
p: 2
i: 10800 // Seconds
d: 2700 // Seconds

So I will now convert them to multiples of the ´sample_time´ value and see what happens.

The P, I and D, are fractions of the output value! They are the parts that are incremented or decreased when the value is processed! P is incremented based on the difference between the input value and the reference value, I and D are related with the delta of the error. All 3 summed give the output!

Hi, thanks for the fast response. My knowledge is very limited on that regard, as it is about ten years I had cybernetics.
So I took your manual readme and tried to reproduce everything. This is what I understood:

Get the values from the step response of the system (so the value from the sensor):

So I figured the fist chart is the head (25 °C) So in the case the target value was 25, the overshoot (difference between peak (30) and target value) 5.
Then I took the time from the peak until the system goes back down to the target value (I called it $t_0$).
Calculate P:
$P = 2 x Overshoot$
Next to i:
From the documentation:

To set the i value we will double the time of the overshoot of the first phase, in our example 22 (11*2)

So to me: $i = 2 * t_0 = 2 * 11 = 22$
Then to d:

... add some Derivate, to calculate this we can set half the overshoot time measured in phase 1, in our case 5 (11/2)

$d = t_0 / 2 = 11 / 2 = 5.5  \approx 5$
So the x units in the chart are seconds. So when reading my chart I calculate from hours to seconds.
Is that correct?

I get that the p, i, and d coefficients are not unitless because they act on a specific set of units, for example
p (process_output_units / process_input_units) * error (process_input_units) + i (process_output_units / (process_input_units * time_units) * error_integral (process_input_units * time_units) + d (process_output_units / (process_input_units / time_units)) * error_derivative (process_input_units/time_units)
To use a temperate set point and a duty cycle PWM output example it might be
p (% / f) * error (f) + I (% / f-sec) * error_integral (f-sec) + d (% / f/sec) * error_derivative (f/sec)
Since these coefficients are unit mirrors of each error component they are often left off because they can be deduced from the following form
p(f) * i(f-sec) * d(f/sec) = %

Short answer from the sources:
it is in the time domain using:
https://www.geeksforgeeks.org/python-time-monotonic-method/
So basically in seconds with a millisecond part, like 15.23 s


An error pops up after clicking "Ignore these local items".

Environment

Steps to Reproduce

In VS, select File > New > Project, search for "Django", and select the Blank Django Web Project template.
Enter the Name and location and check the option "Add to source Control", create the project and wait for project loaded in SE
Create the virtual environment with install requirements.txt and exclude it from source control
a. Go to team explorer click "Changes"
b. Right-click the virtual environment folder and select "Ignore these local items"

Actual result and screenshots
An error pops up.

This is not PTVS bug since it appears in other languages as well. Filed an internal ticket to track this bug. https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1582769

Will be fixed once this PR is in. https://devdiv.visualstudio.com/DevDiv/_git/VSTeamExplorer/pullrequest/413871


Fixes #488

select2_from_ajax was losing selected item if redirect with input

Thanks @MarcosBL ! Merged, will be available to everyone in the next patch release.
Cheers, keep 'em coming!


When using parameters on initialRoute, the widget crashes

When I initialized my GetMaterialApp like this:
  Widget build(BuildContext context) {
    Get.config(enableLog: false);
    return GetMaterialApp(
      initialRoute: '/qr/84226ff6-6ded-4b9d-88c9-95a210896d3a',
      namedRoutes: { '/qr/:code': GetRoute(
        page: QRScreen(),
      ),
    },
      routingCallback: MiddleWare.observer,
    );
  }

And I start the app, I get this error:
flutter: ══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
flutter: The following NoSuchMethodError was thrown building IconTheme(color: Color(0xdd000000)):
flutter: The getter 'route' was called on null.
flutter: Receiver: null
flutter: Tried calling: route
flutter:
flutter: The relevant error-causing widget was:
flutter:   MaterialApp
flutter:   file:///Users/jan/.pub-cache/hosted/pub.dartlang.org/get-2.0.10/lib/src/root/root_widget.dart:172:12
flutter:
flutter: When the exception was thrown, this was the stack:
flutter: #0      Object.noSuchMethod (dart:core-patch/object_patch.dart:53:5)
flutter: #1      _GetMaterialAppState.namedRoutesGenerate (package:get/src/root/root_widget.dart:132:40)
flutter: #2      _WidgetsAppState._onGenerateRoute (package:flutter/src/widgets/app.dart:999:36)
flutter: #3      NavigatorState._routeNamed (package:flutter/src/widgets/navigator.dart:3138:44)
flutter: #4      Navigator.defaultGenerateInitialRoutes (package:flutter/src/widgets/navigator.dart:2162:32)
flutter: #5      NavigatorState.initState (package:flutter/src/widgets/navigator.dart:2582:39)
flutter: #6      StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4640:58)
flutter: #7      ComponentElement.mount (package:flutter/src/widgets/framework.dart:4476:5)
flutter: ...     Normal element mounting (216 frames)
flutter: #223    Element.inflateWidget (package:flutter/src/widgets/framework.dart:3446:14)
flutter: #224    Element.updateChild (package:flutter/src/widgets/framework.dart:3214:18)
flutter: #225    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4527:16)
flutter: #226    _InheritedProviderScopeMixin.performRebuild (package:provider/src/inherited_provider.dart:221:11)
flutter: #227    Element.rebuild (package:flutter/src/widgets/framework.dart:4218:5)
flutter: #228    ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4481:5)
flutter: #229    ComponentElement.mount (package:flutter/src/widgets/framework.dart:4476:5)
flutter: ...     Normal element mounting (7 frames)
flutter: #236    SingleChildWidgetElementMixin.mount (package:nested/nested.dart:223:11)
flutter: #237    Element.inflateWidget (package:flutter/src/widgets/framework.dart:3446:14)
flutter: #238    Element.updateChild (package:flutter/src/widgets/framework.dart:3214:18)
flutter: #239    RenderObjectToWidgetElement._rebuild (package:flutter/src/widgets/binding.dart:1148:16)
flutter: #240    RenderObjectToWidgetElement.mount (package:flutter/src/widgets/binding.dart:1119:5)
flutter: #241    RenderObjectToWidgetAdapter.attachToRenderTree.<anonymous closure> (package:flutter/src/widgets/binding.dart:1061:17)
flutter: #242    BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2607:19)
flutter: #243    RenderObjectToWidgetAdapter.attachToRenderTree (package:flutter/src/widgets/binding.dart:1060:13)
flutter: #244    WidgetsBinding.attachRootWidget (package:flutter/src/widgets/binding.dart:941:7)
flutter: #245    WidgetsBinding.scheduleAttachRootWidget.<anonymous closure> (package:flutter/src/widgets/binding.dart:922:7)
flutter: (elided 11 frames from class _RawReceivePortImpl, class _Timer, dart:async, and dart:async-patch)
flutter:
flutter: ════════════════════════════════════════════════════════════════════════════════════════════════════

I was able to fix it locally by changing this on the file root_widget.dart
before:
    /// onGenerateRoute to FlutterWeb is Broken on Dev/Master. This is a temporary
    /// workaround until they fix it, because the problem is with the 'Flutter engine',
    /// which changes the initial route for an empty String, not the main Flutter,
    /// so only Team can fix it.
    final parsedString = parse.split(
        settings.name == '' ? (widget.initialRoute ?? '/') : settings.name);

    String settingsname = parsedString.route;

after:
    /// onGenerateRoute to FlutterWeb is Broken on Dev/Master. This is a temporary
    /// workaround until they fix it, because the problem is with the 'Flutter engine',
    /// which changes the initial route for an empty String, not the main Flutter,
    /// so only Team can fix it.
    var parsedString = parse.split(
        settings.name == '' ? (widget.initialRoute ?? '/') : settings.name);

    if(parsedString == null) {
      parsedString = AppRouteMatch();
      parsedString.route = settings.name;
    }

    String settingsname = parsedString.route;

Not making a pull request as I'm not sure this break something else, or what. But I figured I would give you what seems to work for me.
Thank you

They just made updates to the master with changes to onGenerateRoute that may have an impact on Get, I will take advantage and include this in the tests, thanks for the suggestion.

Nice, I'm on beta by the way, not master

Flutter (Channel beta, v1.17.0-3.4.pre, on Mac OS X 10.15.4 19E287, locale
en-US)

Yes, I just commented because I have the Get code open doing tests for the master, and I will include that in the tests as well.

Apparently, at least on the master, the Flutter forces the initial route to be '/'.
When using any other named route, such as '/ home'. is displayed in the log [GOING TO ROUTE] /home and then [GOING TO ROUTE] /.
Can you confirm that the same is true in the beta? Because if this is a standard behavior, your solution will cause the initial routes to be duplicated.
Right after navigating to your route, I also notice that it's current route is replaced by null.
If the first behavior is observed in the beta, I believe that there is nothing that can be done besides opening an issue in Flutter (I would have to reconstruct all materialApp to fix this error, and changing the MaterialApp would make the library unstable).
In case you have not noticed this, I will downgrade to beta version to apply the patch.

Fix on 2.1.0. Thank you so much for this.
I still have things to investigate, but about the Flutter part, and why it clears the initial route on the master. And it forces the use of '/', but this is already about the Flutter.