Performance issue on outgoing requests

We have an API developed using DRF but recently we switched to FastAPI. We are loving it a lot. This is one of the greatest software that we have ever come across.
Now getting back to the point, our upcoming version of API is built using FastAPI. When we deploy it to a Docker Swarm cluster, regular endpoints (that just return JSON responses without making any computation) perform really great. In our small cluster, we achieved 40K+ requests per second in wrk benchmarks.
But when we call the endpoints where we are making an outgoing request, the performance dramatically decreases. We barely achieve 100 requests per second in our benchmarking tests.
The endpoints where the performance issue is present make outgoing requests and we return data in real-time. We are using a proxy service too so latency issues are there. But the performance difference between the normal endpoints and the ones that get external data and return to the user is really really huge.
We have tried using Gunicorn + Uvicorn worker and Uvicorn alone too. We also tried applying an API gateway like Kong but the performance issue remains there. On the nodes where the API is running CPU usage rises while the API gateway doesn't go under any load.
Is this a CPU-intensive case or threads are locked when external requests response is awaited? We are using httpx and each API call results in a single external request. Any help will be appreciated greatly.

Could you provide examples/more information?
a. Are you sure the service ur calling isn't the bottleneck, how many services are you calling in each request to FastAPI?
b. How many gunicorn/uvicorn workers?
c. Have you tried a different outgoing service / only calling once?
d. Could you give more info / data about the CPU usage you're getting?
e. Did you run other benchmarks with other endpoints that require slightly less computation?
f. Have you tried other benchmarking tools?
With the information I have right now I can't really see how the benchmark would be that wrong. Your comparing the computation endpoint to a simple json response which will always be insanely fast, you need to test other endpoints that do different actions.

Hi @ArcLightSlavik, thank you for replying.
a. The services that I am calling take up to 5 seconds to respond
b. (2 * 2) + 1 per core (And I tested with 6 nodes each with 4 cores, so 120 Gunicorn workers)
c. Yep, same poor performance with all. And the service is called once per request
d. CPU usage across the cluster reaches ~100% before connections start timing out
e. I have not tested other endpoints yet. What I have tested do no intense computation and they just return a predefined JSON response
f. Not yet, I'll try and I'll post the output here

@rehmatworks Some thoughts with your performance:

The "(2 * 2) + 1 per core" is a general thought.  It really might matter based on if you are using a cloud server that is sharing with others and the CPU/core layout.  It might depend if you have alot of JSON transformations, business logic....
Try to adjust the different parameters and perform a load testing with something like: https://locust.io/. I use this personally and it really helps tune things correctly.
Are you hitting an HTTPS endpoint with your external call?   You may find that you client is spending all of your CPU establishing TLS connections.  Look at a pooling option so you don't pay for that setup with every connection.
What HTTP client are you using?  I use AIOHttp and have to tune our clients to get optimal performance: https://docs.aiohttp.org/en/stable/client_advanced.html#connectors
Lastly, try to add some telemetry with an APM tool: New Relic, AppDynamics, Dynatrace, or add some low level logging so you can understand which functions and layers are your bottlenecks.

It sounds like you are getting good performance with a simple use case.

@STeveShary Thank you so much for replying. Looks like you have pointed me in the right direction. I am modifying my code and I'll run some benchmarking tests again.

I'll give locust.io a try
Yes, the endpoints are HTTPS. I am now using HTTP endpoints as well as I am trying pooling
I am using HTTPX
I am using Sentry but haven't looked in the logs for this particular issue yet

Thank you for all your help. I'll be running benchmarking tests again and I'll share the results.

@STeveShary Thank you so much for guiding me in the right direction and helping me in solving this serious performance issue.


About resume training

if args.restore_ckpt is not None:
    strStep = os.path.split(args.restore_ckpt)[-1].split('_')[0]
    total_steps = int(strStep) if strStep.isdigit() else 0
else:
    total_steps = 0

Hi,
I found this code in train.py. The class OneCycleLR need a paramter 'last_epoch' to resume the lr, so it needs save the optimizer's state_dict, because we need a key called 'initial_lr'. However, I don't find the code to save optimizer's dict. Why?
Do you have to rename the model file when you load it? This would allow this code to be skipped.

Hey,
We didn't save optimizer's state_dict. The total_steps is attained from the file name (usually it ends with the number of iterations). So when resume training, you may expect some differences from your original training process.

Thank you for replying，
if 'initial_lr' not in group:
raise KeyError("param 'initial_lr' is not specified "
"in param_groups[{}] when resuming an optimizer".format(i))
Look at this code in OneCycle line 32. My question is there are no codes to add the parameter 'initial_lr' to optimizer in train.py, so if the 'last_epoch' in OneCycle is set, wouldn't an error be thrown at runtime?

Hey, why not take a try. I think initial_lr may be set somewhere in RAFT's codebase.

How do you compare the results of your experiments with the 2-view ones when they are warm-start?

Hey, we only report the 2-view results. You may check the caption of Table 2 in the paper or run the experiments on your own.


Updated cvarlist.md to version <IP_ADDRESS>

Thanks for this - I've committed b709b96d1c7ebb1529bcac8b477ff91fb2654205 with the latest changes. In future, please can you run cvarlist_all on the client rather than a dedicated server? Otherwise all the client-side cvars disappear.


spec: signing the headers

It's not super clear whether the headers of the HTTP message are also signed under webhook-signature. This should generally be the case so that they too are authenticated. This probably requires adding a section about canonicalizing the headers and including them in the signature computation.

So the signature would sign all the headers except the webhook-signature one right?
That sounds a legitimate but i have a question:
On the consumer side what would happen then when the request goes to N proxies before landing to the webhook handler? If any proxy mutates the http headers it would fail no? There's many doing so. eg:
In Kong you can setup keys for a route via a concept of consumer and this route will add headers related to the identication of this key. ref: https://docs.konghq.com/hub/kong-inc/key-auth/#upstream-headers

Under the current spec (we probably should clarify that), they would indeed not be signed. There are two reasons for that: (1) what @zekth said about all the intermediary that could potentially modify the headers, and (2) they shouldn't hold any information of significance. Per the per spec the data should only be sent as part of the payload, so as long as that's verified, everything that matters is verified. Verifying the headers should therefore not add any security.
 I think it would be immensely difficult to maintain a consistent behavior across different implementations. So even if we wanted it, I think it's going to add a tonne of complexity.

So the signature would sign all the headers except the webhook-signature one right? That sounds a legitimate but i have a question:
On the consumer side what would happen then when the request goes to N proxies before landing to the webhook handler? If any proxy mutates the http headers it would fail no? There's many doing so. eg: In Kong you can setup keys for a route via a concept of consumer and this route will add headers related to the identication of this key. ref: https://docs.konghq.com/hub/kong-inc/key-auth/#upstream-headers

That's fine. Not all headers should participate in the signature. And in any case, if the intermediary is trusted, it can have access to the private key and recompute the signature if necessary.

Under the current spec (we probably should clarify that), they would indeed not be signed. There are two reasons for that: (1) what @zekth said about all the intermediary that could potentially modify the headers, and (2) they shouldn't hold any information of significance. Per the per spec the data should only be sent as part of the payload, so as long as that's verified, everything that matters is verified. Verifying the headers should therefore not add any security.

Ah but this is the problem. For example, the spec says check webhook-id and webhook-timestamp but me as an attacker, I can choose any value for those. So I can replay the same message with its valid signature, and vary those two headers to my choosing.
Had they been included in the signature, the whole payload would be immediately rejected if I tried to do anything like that.

 I think it would be immensely difficult to maintain a consistent behavior across different implementations. So even if we wanted it, I think it's going to add a tonne of complexity.

We can say something like:

All webhook- prefixed headers are sorted alphabetically.
Concatenated in their original form, i.e. webhook-xyz: abc\n\r.
Prepended to the body in its original form.
Finally signature is computed.

My bad i forgot this part.
For ref:
https://github.com/standard-webhooks/standard-webhooks/blob/d6b7156598536e02a5ff898984e55f63dd208298/libraries/go/webhook.go#L94-L118

I'm not sure if this might be of interest to the project, but there is a "HTTP message signatures" RFC standard draft which might work for this use case.

Hey @teliosdev, thanks for the link! We are familiar with the RFC (it's even mentioned in the README!). There were a few reasons, but one of the main ones is: one of the design goals of this spec is to meet people where they are. So following but cementing industry best practices, rather than trying to do something very different. It's much better to have a great spec that's widely adopted than a perfect one that's used by no one. With that being said, as HTTP message signatures become more widespread/widely supported, this recommendation will probably change (future version of the spec).

I'm closing this ticket, as the original issue is resolved (headers are signed).


add note about running in terminal, not iTerm

... to avoid https://github.com/sveinbjornt/hear/issues/15
When running in iTerm, hear gives Abort trap: 6, but that doesn't get trapped, so the resulting error is an attempted fork loop:
Starting ./hear
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Starting ./hear
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Starting ./hear

etc.

Case it helps anyone reading, this applies to running Teleprompter in VS Code integrated terminal - same error:


[BEAM-11709] Disable Gradle module metadata generation

(cherry picked from commit 506e5cf88121c69cb1d94df33f773a8d223c4dce)
Cherry pick of PR #13836

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Java PreCommit

Run Java PreCommit

R: @chamikaramj

R: @chamikaramj


Add more information to useful-not-found-page

Description
Renamed files turn into 404s, but we add a link to the commit history:

What we if were more explicit about what happened to the file and link to the last known version?
Example:

the file was deleted by bab4ba 3 months ago
the file was renamed to a.jsx
the branch doesn't exist
see the last version

Arguably these links are more useful than the breadcrumbs resolution and can be appended to the page as the API calls resolve
Example URL
https://github.com/gorhill/uBlock/blob/master/assets/ublock/resources.txt

Hi there!
I would like to take up this issue, but I noticed that it has the under discussion label. Is there something I should know before taking it up or could I take a crack at it?
Thanks

under discussion means that the details are not exact and further suggestions are welcome. You could either propose an improved UI to include all of that information or just add one or 2 more links.
I think the core code change here is fetching the last commit that affected the file and then find out what happened to it (deleted or renamed) and then point to the previous commit.
But again I haven't looked into the details, so alternative/better implementations are welcome.

Hi!
Thanks for the reply. I couldn't think of an improved UI to display this info, so I took a crack at adding it to the page. I opened a draft PR for this (#4646), it would be great if you could take a look at the changes I made.
Thanks a lot!


Gke

controller support on GKE cluster
Only provide a sample prometheus_service.yaml as stakeholder now
https://github.com/open-cluster-management/backlog/issues/14797
https://github.com/open-cluster-management/backlog/issues/14795

/lgtm


Stop serializing entire rows/beans on update forms

Rationale
JSON serialization is unnecessary and can be problematic. Serialize/deserialize just the version and PK. Leave experimental feature for now in case there's still code relying on previous behavior.

In addition to BVT, BVT-EHR, and Study Postres (above), Daily Postgres ran completely green on this FB: https://teamcity.labkey.org/buildConfiguration/LabkeyTrunk_DailyPostgres_2/2769509?buildTab=dependencies&mode=list&type=snapshot


Transition element removes class that existed on the element before transition was triggered

Version
3.0.4
Reproduction link
https://codepen.io/HJBDev/pen/abmBYjx
If you look at the HTML box, you can see that the element has opacity-75 on it to make the element translucent. When the animation triggers, transitioning from 0 to 75 opacity, the class is removed.
Steps to reproduce
Put a class on an element, create a transition that uses that same class, trigger the transition. The initial class gets removed.
What is expected?
The original class is reinstated after the transition has completed.
What is actually happening?
The class stays removed.

This is really only an issue for utility CSS frameworks. I was using the translate classes in tailwind to offset some notifications that transition in so it's obvious there are multiple notifications on screen.
As you can see one of the notifications looks wonky, because the transform class I use for the entrance transition was also meant to be used to offset that particular element (if that makes sense)
https://i.imgur.com/KLECaAs.png
I'm not entirely sure the best way of remedying the problem. Maybe storing the original classList and then comparing them afterwards?

The problem is that you shouldn't have the opacity-75 always set because it is defined after opacity-0 and therefore has a higher priority. That's why there is no initial animation because the element has both classes and only opacity-75 is applied.
What you need to do is create another CSS selector that applies the starting and ending state (opacity-0) and that has a higher css specificity than .opacity-75.
Classes should not be repeated on an element anyway (that's why classList is a Set that ignores duplicates). In this case, even if the class was preserved, you would still see the same problem.


[BUG]: NTT fails in certain special cases

Description
Mixed-radix coset NTT seems to work incorrectly when ordering is MN (switching to other orderings, non-coset or radix2 makes the error disappear). Specifically, non fast twiddle version fails on sizes 9, 11, 19, 21 and 23. Fast twiddle version additionally fails for 13, 14 and 16.
Reproduce
I customised the coset ntt test to showcase the issue, pls see https://github.com/ingonyama-zk/icicle/blob/a9a2f07c61bca76fe7245b9a59ef767ec99c2099/wrappers/rust/icicle-core/src/ntt/tests.rs#L104
To reproduce, checkout the branch I linked and run cargo test test_ntt_coset_from_subgroup from any of the curve crates.
Additional context
It's pretty clear that the issue is in reordering and not ntt itself, since all orderings except one work fine. Sizes I mentioned seem to be related to stage sizes in mixed radix ntt: failures in non fast twiddle version happen exactly where stage sizes are not symmetrical (i.e. "mixing" permutation has order bigger than 2). And additional failures for fast twiddles correspond to differences between fast twiddle and regular stage sizes. Maybe the problem is that the code assumes that dig_rev function must bring permutation to identity after two iterations, and in places non-fast twiddles stages are used instead of fast twiddles stages but I couldn't figure out where exactly.

@DmytroTym please review https://github.com/ingonyama-zk/icicle/pull/497


return ExtesionProvider clusters when PILOT_FILTER_GATEWAY_CLUSTER_CO…

Please provide a description of this PR:
fixed: https://github.com/istio/istio/issues/37997
To help us figure out who should review this PR, please put an X in all the areas that this PR affects.

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[x] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Please check any characteristics that apply to this pull request.

[x] Does not have any user-facing changes. This may include CLI changes, API changes, behavior changes, performance improvements, etc.

@zirain: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
72b525571a868d1f8851022f8194b452e9a32ee0
link
true
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Don't start the Chrome Debugger for a file.

This probably isn't great (ideally we would be able to debug everything). But the use-case I have for this is the following:

I have a hybrid app (both RN and native code)
I have 'common' RN code included in the project as a .jsbundle file
I also have app-specific logic running via npm start which I want to debug

Currently when you try to run any of the code in the jsbundle, it will attempt to connect to the Chrome Debugger (if it's running) and then fail (because the Chrome Debugger doesn't currently seem to support bundled files).
This change basically stops any bundled files from connecting to the Chrome Debugger, allowing people to debug non-bundled code along side non-debugging bundled coded.

c

Didn't get it, how do you have both bundled and non-bundled code?

@satya164 In our case we have a couple of hybrid apps (both native and RN UI), both of those apps share an Auth bundle which only contains the logic & UI required for loging in (this way it is smaller and loaded faster than if we attempted to load the full app bundle on launch).
This also means we could do over-the-air updates for just the Auth module rather than the entire RN app.
It basically gives the freedom to have a common RN library between multiple apps whilst continuing development independently of the library. It also opens the door for consistent 3rd-party integration via CocoaPods or Maven regardless of wether or not the 3rd-party app is being built in RN or native (one consistent install method makes supporting an SDK much easier).

If there is a better approach for something like this then I'm open to it though (it would make proper debugging easier) - I haven't had the chance to look in to the HMR stuff yet so don't know if that would suffice in this case.

@bottledwalter would you mind taking a look at this pull request? It's been a while since the last commit was reviewed.

This commit looks very niche and isn't the right approach to the problem. I'm not sure what the right approach is, but I think you should keep this in your own fork of RN instead of committing it upstream.

@ide: Fair enough, will close the PR.
I still do think failing to load a bundle just because you forgot to disable chrome debugging is not an obvious error; especially considering it only actually appears in the chrome debugger (not XCode or in the simulator). I wouldn't mind the the error was a little more obvious (maybe turn the top 'loading' bar to red with an error?).


standardize xml_declaration

some templates have the <?xml> declaration and some don't. standardize.
probably requires a test suite to validate regression

i remember doing much of this a few weeks ago, so there might not be much left to do.


SpatialObject Refactoring for ITKv5

High level description of changes made:

Eliminate IndexToX transforms. SpatialObjects exist purely in physical space coordinates
Eliminate vnl TreeNode usage. SpatialObjects now track their transforms, parents, and children directly.
Eliminate AffineGeometryFrame
Eliminate OffsetTable computation
Dimension renamed to ObjectDimension - for consistency
AddSpatialObject() renamed to AddChild() - for consistency
RemoveSpatialObject() renamed to RemoveChild() - for consistency
Clear() renamed to ClearChildren() - for clarity and consistency
AddChildrenToList added to speed parsing of children in tree
GetSpatialObjectTypeAsString() is now GetClassNameAndDimension()
PropertyType is templated over ScalarType, not float
Converted API from using char * to using std::string
SetParent and AddChild now create consistent trees and do not cause a spatial object to move in physical space (i.e., transforms are updated appropriately).
RemoveChild and RemoveAllChildren fixed to remove all pointers to / from those children to / from the tree
Helper functions simplify the specification of IsInside, ValueAt, and other computations that potentially traverse an SO tree.
Derived classes typically only need to implement IsInside and ComputeObjectBoundingBox member functions. Logic for ValueAt and such is improved.

This is a squash and rebase of #585.

This doesn't build on my computer, so don't merge this yet.

Looks good to me.


reCAPTCHA exists

Issue description:
reCAPTCHA is a service or an option that is available for webmasters - however, it is poorly designed, bad for user experience, and has no regard for one's privacy. It is bad.
Environment:
TCP/IP WWW
Steps to Reproduce:
Go onto a site which requires the use of reCAPTCHA (many such cases available).
Suggested Changes:
It would be objectively best to remove the entire repository, either through an emptying PR or complete annihilation of the project. It would then convince webmasters and large organizations to start using the much-better hCaptcha so that we no longer have to coexist in a reality where the issues reCAPTCHA sows are prominent.
Thank you for looking into this and I hope a suitable change will be made.

By your logic both take long to solve, therefore neither would be better in that regard — except that hcaptcha is anything but Google, therefore winning anyway.
I'm skeptical by your claim anyway because h is very friendly especially when testing with proxies, never have I ever needed to do a retry when using that service.


refactor(dispatch-api): switch to @nestjs/axios, seperate open api

:tada: This PR is included in version 3.6.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Not all resources are removed upon appsub removal

When adding an appsub, all its resources are given an annotation that specifies their relation  with that appsub, so they will also be removed when removing the appsub.
This is fine in the general case but might fail when the application itself spawns other resources like pods without propagating that annotation to the spawned resources.
One may argue that when removing the appsub the appsub should only be responsible to remove its direct resources and the application is responsible to remove the spawned resources; or that the application is responsible to propagate the aforementioned annotation into its spawned resources. But this is hard to enforce.
A possible solution for this may be to remove all entities within the appsub's namespace, no matter if they have that annotation or not. Need to be careful not to remove things that will be part of the admin-namespace (for multi-namespace appsubs) though.

I can't see us removing resources we were not directly responsible for. Like if we deployed an operator, we would expect the operator to own cleanup of the operand when we attempted to remove it.
We open to offering the other direction, where we provide a switch to leave all artifacts created by the subscription behind. Plz reach out if you see a benefit from that one.


feat: add Nat.mul_lt_mul₀

This is a Nat variant of
https://leanprover-community.github.io/mathlib4_docs/Mathlib/Algebra/Order/WithZero.html#mul_lt_mul₀

Let me know which naming convention I should follow. There are:

Nat.mul_lt_mul
Nat.mul_lt_mul'
mul_lt_mul₀

and they all use different conventions. Right now I am following Nat.mul_lt_mul'.

It appears that particular theorem derives from https://github.com/leanprover-community/lean/commit/e865d103751f771e8d2836be4e311fc18c3794a0 : it did have the hypothesis but lost it later.

Here is a proposal to fix these issues:
-- formerly: mul_le_mul_of_nonneg_left
protected theorem mul_le_mul_left' {a b c : Nat} : a ≤ b → c * a ≤ c * b := Nat.mul_le_mul_left c

-- formerly: mul_le_mul_of_nonneg_right
protected theorem mul_le_mul_right' {a b c : Nat} : a ≤ b → a * c ≤ b * c := Nat.mul_le_mul_right c

-- formerly: mul_lt_mul, with hypothesis 0 < b weakened to 0 < d
protected theorem mul_lt_mul_of_lt_of_le (hac : a < c) (hbd : b ≤ d) (hd : 0 < d) : a * b < c * d :=
  Nat.lt_of_le_of_lt (Nat.mul_le_mul_left' hbd) (Nat.mul_lt_mul_of_pos_right hac hd)

-- formerly: mul_lt_mul'
protected theorem mul_lt_mul_of_le_of_lt (hac : a ≤ c) (hbd : b < d) (hc : 0 < c) : a * b < c * d :=
  Nat.lt_of_le_of_lt (Nat.mul_le_mul_right' hac) (Nat.mul_lt_mul_of_pos_left hbd hc)

-- formerly: mul_lt_mul₀
protected theorem mul_lt_mul {a b c d : Nat} (hac : a < c) (hbd : b < d) : a * b < c * d :=
  Nat.mul_lt_mul_of_le_of_lt (Nat.le_of_lt hac) hbd (Nat.zero_lt_of_lt hac)

These changes don't break anything in Std. I did a rudimentary search through Mathlib and only found a couple of instances of each of the former lemmas.

The proposal looks good to me. Feel free to close this PR if that approach is taken instead.

Now incorporated in #177 under the name Nat.mul_lt_mul.

Nice, and thank you!


[Spec Update] Move default_agg_time to be in default struct within SemanticModel

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward change to existing dbt-semantic-interfaces functionality, rather than a Big Idea better suited to a discussion

Describe the feature
This is a follow up after #49 . In reality this is more of a semantic model property than it is a dimension property.
Describe alternatives you've considered
No response
Who will this benefit?
No response
Are you interested in contributing this feature?
No response
Anything else?
No response

Resolved by https://github.com/dbt-labs/dbt-semantic-interfaces/pull/79


Updated requirements

v0.6 introduced a new requirement on pandas 0.20.1 and numexpr... was it 2.26?

[ ] update requirements so installs automatically update pandas and numexpr
[ ] add new requirement to changelog.

Pandas bound on 0.20.2 is indeed the numexpr culprit https://pandas.pydata.org/pandas-docs/stable/install.html#install-recommended-dependencies

Want me to just include this on the RC2 branch?

This is solved in #135.


Loss becomes NaN

When I run the unknown initialization experiments on MNIST:
python main.py --mode distill_basic --dataset MNIST --arch LeNet
I get the following after a few dozen epochs:
Traceback (most recent call last):   File "main.py", line 402, in <module>     main(options.get_state())   File "main.py", line 130, in main     steps = train_distilled_image.distill(state, state.models)   File "/home/isucholu/original/dataset-distillation/train_distilled_image.py", line 296, in distill     return Trainer(state, models).train()   File "/home/isucholu/original/dataset-distillation/train_distilled_image.py", line 283, in train     raise RuntimeError('loss became NaN') RuntimeError: loss became NaN
Was gradient fairly stable when you were running this for the paper? Do I just need to make some more attempts?

That is weird. In my experience it has been quite stable. Maybe try with a lower learning rate?

Yeah looks like lower learning rate leads to more stable gradients. It might be the updated default learning rate you set the other day is too high for this mode?

@ilia10000 Ah yeah it could be. I reset it back to 0.01 in https://github.com/SsnL/dataset-distillation/commit/00652bb94596addf8ddda2f89cae8aaac6b21747 just to be safe. Let me know if it still is unstable for you.

Question: how do I interpret the nan on the standard deviation?  can this be related to @ilia10000  learning rate problem?
2020-11-29 07:15:24 [INFO ]  distilled with 10 steps and 3 epochs  (1 unknown_init nets) test results:
2020-11-29 07:15:24 [INFO ]  	          STEP                   ACCURACY                   LOSS
2020-11-29 07:15:24 [INFO ]  	            before steps          28.7854 ±  nan%            1.1057 ±  nan
2020-11-29 07:15:24 [INFO ]  	     step 30 (lr=0.0121)          38.1032 ±  nan%            1.0879 ±  nan

@Rubiel1 You're using a single unknown init net, so there is only a single result (i.e. just the one accuracy value). If you use more nets you'll have standard deviations.


Multi cursor deselects when typing brackets in multiple positions on same line

VSCode Version:

Version 1.8.1
Commit ee428b0eead68bf0fb99ab5fdc4439be227b6281
Date 2016-12-19T14:41:20.664Z
Shell 1.4.6
Renderer 53.0.2785.143
Node 6.5.0

OS Version:
Ubuntu 16.04  Kernel: 4.4.0-47-generic

Steps to Reproduce:

Open a file - I have reproduced with .less, .js and .html
Select somewhere
Hold alt and select somewhere else further along the same line
Open and close a pair of brackets
Notice that the second pair (and all other subsequent pairs) of brackets did not close and the cursor is now only selecting the left-most place on the line.

The issue does not occur when typing only a closing bracket (unless that closing bracket is immediately after an opening one)
I've tried this with extensions both enabled and disabled - the issue is present in both situations
There are no errors in the dev tools console

Update:

It's not the left-most cursor that remains, it is the first/primary cursor.
This does not happen when both cursors are inside a string in a .js file
This did not happen in 1.6.0, but does happen in 1.8.1

This seems to be fixed in v1.9, no ?

Confirmed: I no longer see the issue after upgrading to v1.9.0.


feat: set queue properties

Closes #92

Go tests passed on my machine + passed on the checkov check.

Might need to implement something similar to #100 to prevent trying to set queue properties when queue storage is disabled.
We should also implement a few new tests (or cases for an existing test) to test the different storage kinds (storagev2, blobstorage, filestorage ...).

#103 should be merged before this, as it might change the implementation in this PR.

Can we merge this now?

Terraform docs states that queue_properties cannot be set when the account_kind is set to BlobStorage. Wouldn't this, being hard-coded, affect the test examples with account_kind = BlobStorage if merged? I'll run a test to confirm.

Working on it 😄

Tests pass 🎉

@sebastianlolv Merge away 🏄


connection param: timeout  do not work

so i have a connection with timeout:

redis = await aioredis.create_redis(('<IP_ADDRESS>',6379), password='xxx',  timeout=3)

and use this command to block redis:

debug sleep 30

i expect get an  timeout error when query redis liket this:

await redis.info()

but this connection just wait 30S and return normal data.
i think this  ‘timeout' param doesn't work correctly。

aioredis version: 1.3.1

I am seeing the same issue. running with uvloop, gunicorn. Timeout is ignored. Attempting to wrap the redis operation in a task and use asyncio.wait_for with a timeout seems to solve the problem but I have a suspicion that such timed out operations leave open sockets and are not acceptable in a production environment

This timeout param is only used to establish connection, not to command execution

Hey @opt-m - as @ramonz mentioned, the timeout param refers to acquiring a valid connection, which the client is still able to do, regardless of whether command execution is blocked.
If you wish to enforce a command timeout, you can use a library like async-timeout or make use of asyncio's asyncio.wait_for().
@ognacy regarding your concern about leaving an open socket, that should not be the case, but if you observe such behavior, I'd appreciate a ticket with possible reproduction so that we can triage and prioritize a fix appropriately.
For now, I'm going to close out this ticket.


⚠ Bofrost.de has degraded performance

In d130c29, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1634 ms

Resolved: Bofrost.de performance has improved in 7fe097e.


!discord bridge not working

that is because of disableContext: true, - why did we add that, what did that fix?

oh that question was @Half-Shot btw

I think this got fixed.


make new assert/assume/cover statements public

Contributor Checklist

[ ] Did you add Scaladoc to every public function/method?
[ ] Did you add at least one test demonstrating the PR?
[ ] Did you delete any extraneous printlns/debugging code?
[ ] Did you specify the type of improvement?
[ ] Did you add appropriate documentation in docs/src?
[ ] Did you state the API impact?
[ ] Did you specify the code generation impact?
[ ] Did you request a desired merge strategy?
[ ] Did you add text to be included in the Release Notes for this change?

Type of Improvement

API Impact

Backend Code Generation Impact

Desired Merge Strategy

Release Notes

Reviewer Checklist (only modified by reviewer)

[ ] Did you add the appropriate labels?
[ ] Did you mark the proper milestone (Bug fix: 3.3.x, [small] API extension: 3.4.x, API modification or big change: 3.5.0)?
[ ] Did you review?
[ ] Did you check whether all relevant Contributor checkboxes have been checked?
[ ] Did you mark as Please Merge?

@azidar: I think I need your help to fix the SelectSpec. Not sure, why it is failing now. I think the problem is that assert now uses an assert instead of a stop(1).

Can you clarify a bit on how this assert/assume/cover is related (or not) to the experimental verif stuff?

Can you clarify a bit on how this assert/assume/cover is related (or not) to the experimental verif stuff?

It is essentially subsuming it and adding new capabilities to assert and assume to have the same printf mechanism that we used to have with the stop(1) based chisel3.asset. I hope that this way of exposing the new verification statements will be mostly source compatible with Chisel 3.4 code.

LGTM, again. I guess the last question is if the cover message isn't being used at all, should it be added, or should we wait until there is a use case to avoid dead code?

I am working on a followup PR which will automatically attach a VerificationInfoAnnotation to every assert, assume, cover and stop. That annotation will have a message field.
My current prototype looks like:
/** meta-data of assert, assume, cover and stop statements which can be used by downstream tools to analyze verification results */
case class VerificationInfoAnnotation(statement: ReferenceTarget, print: Option[ReferenceTarget], info: VerificationInfo) extends MultiTargetAnnotation {

  override def targets = Seq(Seq(statement), print.toSeq)
  override def duplicate(n: Seq[Seq[Target]]) = n match {
    case Seq(Seq(s: ReferenceTarget), Seq(p: ReferenceTarget)) => copy(statement=s, print=Some(p))
    case Seq(Seq(s: ReferenceTarget), Seq()) => copy(statement=s, print=None)
    case other => throw new RuntimeException(s"Unexpected argument to duplicate: $other")
  }
}

case class VerificationInfo(message: String, filename: String, line: Int, cond: String, kind: String) {
  require(line >= 0)
  require(Set("assert", "assume", "cover", "stop").contains(kind))
}

I am thinking about also adding some stack trace information on how the assertion was created. That way, I might be able to throw an exception in chiseltest that contains the stack trace of when the assertion was created.


Restructure the route and ride models

There is redundancy in a Ride model that needs to be taken care of. Please modify the route model table to save following attributes as well:

Destination
Purpose (Title of ride card)

Adding these in route model can help us eliminate redundant data from Ride.

PR from branch 'edit_func' restructured the code.


Update formatting of contact forms

Splits contact form inputs into multiple columns on larger screens
Enforces a max form width on larger screens
Shortens the length of question titles for contact form inputs
Resolves a couple of typos

https://i.imgur.com/IknLSJZ.png
https://i.imgur.com/8ENZ4kd.png


README addition

Compiled a README from content on the site with links to site and licensing.

Merged, though I am skeptical of all this duplication.


Enforce newline on cert body/chain before destination uploads?

👋 I noticed that some https servers expect a newline at the end of the cert body and chain when terminating TLS (e.g., golang; older example with kubernetes nginx ingress). The relevant RFC seems to indicate parsers need to handle different newline conventions:

Furthermore, parsers SHOULD ignore whitespace and other non-
base64 characters and MUST handle different newline conventions.

One server failed for me after retrieving a cert from a Lemur destination upload because it thought our PEM block was malformed due to concatenating the domain cert and chain without newlines, and skipped over it. We realized we had introduced the bug ourselves in parsing, however, so not sure how big of an issue this is.
I added a patch in https://github.com/havron/lemur/commit/76b1bb7eb0b99f1fb2fd9ab6153238b886977586; is there interest in a PR to enforce this for all certs? I wasn't sure what an appropriate way to test this would be, since we'd maybe want to mock a server loading the certs with+without newlines.

Hey Sam, thanks for sharing this with us. Honestly, we haven't experienced any issues with the current destinations we use, and I am not sure what would break, if we change the newline. We can keep an eye out, if this should become needed.

Sounds good to me, thanks! I'll close this for now :)


Basic value_and_gradient examples not working

Forgive me if this is dumb, but I can't get basic usage of Mooncake to work. For example:
using DifferentiationInterface
import Mooncake

f(x) = sum(x)

x = [1, 2]

value_and_gradient(f, AutoMooncake(config=nothing), x)

produces
ERROR: MethodError: no method matching copyto!(::Mooncake.NoTangent, ::Bool)

How do I differentiate simple functions?

julia> versioninfo()
Julia Version 1.11.2
Commit 5e9a32e7af2 (2024-12-01 20:02 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 8 × Apple M2
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m2)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

(@v1.11) pkg> st
  ...
  [a0c0ee7d] DifferentiationInterface v0.6.27
  [da2b9cff] Mooncake v0.4.60
  ...

I realised my mistake — Ints are not differentiable.
Perhaps consider this issue a request to produce kinder error messages when trying to take the tangent of a non-differentiable type?

Ah, right, yes. The basic problem here is that we don't have enough constraints on the input types. @gdalle is not keen to impose input type constraints at an interface-wide level in DifferentiationInterface.
@gdalle would it be permissible to impose constraints on the functions passed for Mooncake to differentiate? Specifically that

if they return a scalar, it must be an instance of Union{Float16, Float32, Float64}, and
that the input that the element type of the arg w.r.t. which the adjoint is computed be a subtype of Union{Float16, Float32, Float64}?

I think this would probably prevent this kind of problem in the future, and it would be quite straightforward to document.

As to whether this kind of restriction would constitute a breaking change for DifferentiationInterface, it turns out that "is Base.copy defined for this gradient type" is an implicit part of the interface currently -- see here and here.
Mooncake doesn't currently define Base.copy for most things, including Mooncake.Tangent, Mooncake.MutableTangent, and Mooncake.NoTangent, and Base.copy is not defined for Tuples or NamedTuples. This means that the vast majority of types currently won't work if you try to return them as the result of differentiating stuff using DI + Mooncake -- like the user will get some unintelligible error at present. In fact, one of the very few tangent types that copy will work for is Array.
This is actually a win for this proposal, because it gives us wide latitude to make the interface for Mooncake quite explicit (if we want to) in a non-breaking fashion.

If we do this, we should also enforce a no-aliasing policy on the arguments (this is functionality I'm going to add to Mooncake soon anyway, because if you can exclude the possibility of aliasing and circular referencing in the arguments to a function, a bunch of things become quite a bit cheaper to do).

The basic problem here is that we don't have enough constraints on the input types

Rather than restricting the input tangent types (what if I have a custom array type?), wouldn't it be more prudent to explicitly assert that dy_righttype is not NoTangent() where appropriate and raise an informative error message otherwise?

Just popping by to say that I'm currently busy with moving from Switzerland to France, but I'll catch up on the discussion in a few days. Feel free to also open issues in DI if necessary


docs: clarify logical ID's and testMatch checks

What is the improvement or update you wish to see?
For Browser Checks automatically created from *.spec.ts files with testMatch, the logical ID is automatically generated and is based on the path of the check. One important limitation of this is that moving a *.spec.ts file can cause the check deleted then recreated when running npx checkly deploy, rather than simply updating the check. This can cause users to loose the history of the check.
This is a potential footgun that might be worth a small call-out in the docs. For users that which to avoid this, they may be better off creating checks directly using the BrowserCheck construct to have more control over the logical ID.
Is there any context that might help us understand?
no additional context
Does the docs page already exist? Please link to it.
No response

Additional information: recreating a check not only happens when a check is moved but also when it's simply added to a group (provided the check was created using testMatch.

This is documented now ("Using checkMatch and testMatch"). I'll close this issue.

When you rename a file that was previously deployed, the logicalId will change. The effect is that once you deploy again the new logicalId will trigger a deletion of the “old” Check and a creation of this “new” Check and you will lose any historical metrics.


V12 Migration: Node type imported from @xyflow/react is missing positionAbsolute field

What platform were you using when you found the bug?

React Flow / Svelte Flow version: 12.1.1
Browser and version:
OS and version:

Live code example
No response
Describe the Bug
Before V12 we were using positionAbsolute to work with parent nodes, and many other staff related to positioning.
Now after upgrade to V12 positionAbsolute is missing in Node type imported from '@xyflow/react', but documentation saying that it is here.
https://reactflow.dev/api-reference/types/node#position-absolute
Can this field be used or not? Where the problem: in docs or in type definitions?
And if this field is deprecated, what can we use instead?
Steps to reproduce the bug or issue

Expected behavior

Screenshots or Videos
No response
Additional context
No response

The documentation is wrong about this and needs to be updated.
Since v12 user nodes (which are of type Node) do not have a field called positionAbsolute anymore.
These fields are only stored in internal nodes (which are of type InternalNode) as internalNode.internals.positionAbsolute.
If you need access to the internals like positionAbsolute you can use getInternalNode and access the property like this
const { getInternalNode } = useReactFlow()

const internalNode = getInternalNode(nodeId)

const positionAbsolute = internalNode.internals.positionAbsolute

Thanks! We updated the docs.


Map type not working on OSS vs Enterprise

What happened:
Querying map using Clickhouse plugin v3.3.0 fails with the latest docker image of Grafana OSS:
Status: 500. Message: sql: Scan error on column index 8, name "botData": unsupported Scan, storing driver.Value type map[string]*string into type *string: Could not process SQL results
Using the following Grafana:
docker run -d --name=grafana -p 3443:3000 grafana/grafana-oss:10.2.2-ubuntu
What you expected to happen:
Able to query datatype map(string,string).
Trying with docker run -d --name=grafana -p 3443:3000 grafana/grafana-enterprise:10.2.2-ubuntu using the same clickhouse plugin v3.3.0 and the same datasource works.
How to reproduce it (as minimally and precisely as possible):
Launch Grafana OSS with Docker, install clickhouse plugin 3.3.0, try to query a map(string, string) type.
Environment:

Grafana version: grafana-oss:10.2.2-ubuntu
Plugin version: v3.3.0
OS Grafana is installed on: docker
User OS & Browser: unrelated
Others: works well with Grafana Enterprise

Technically it's happening when I'm using the log panel and I do an ugly query like:
SELECT * FROM mytable LIMIT 10 to display example of "logs".
I have playing a little bit and I have examples:
This works:
SELECT botData['botScore'] as score FROM mytable WHERE timestamp < now() and timestamp > now() - INTERVAL 10 MINUTE AND score = '10' LIMIT 100

But this doesn't:
SELECT botData FROM mytable WHERE timestamp < now() and timestamp > now() - INTERVAL 10 MINUTE LIMIT 100

I'm guessing if you try SELECT a FROM table_map; it'll fail on OSS but works on enterprise.

@dsztykman Hmmm, this doesn't seem to be reproducible for me in the logs view either, on my local version based on the main branch or Docker images of OSS 10.2.2 or Enterprise 10.2.2.

A sample dataset would help here if possible, as the exception seems to be thrown by the database driver, so might it be something that the driver, or the abstraction layer we have around it, is parsing incorrectly. It could also be me missing something here, so if you have any other helpful context, please feel free to add it.

Closing this due to inactivity, please feel free to re-open if this is still an issue 😊

Sorry for my lack of response, it is indeed still an issue.
grafana-5b46cbf846-zs7g2 logger=datasources t=2024-04-04T18:56:24.827593858Z level=debug msg="Querying for data source via SQL store" uid=a6835544-2bfe-4f3a-98da-524301ae2280 orgId=2
grafana-5b46cbf846-zs7g2 logger=query_data t=2024-04-04T18:56:24.828285307Z level=debug msg="Processing metrics query" query="unsupported value type"
grafana-5b46cbf846-zs7g2 logger=query_data t=2024-04-04T18:56:24.832854513Z level=debug msg="Processing metrics query" query="unsupported value type"
grafana-5b46cbf846-zs7g2 logger=secrets.kvstore t=2024-04-04T18:56:24.833698127Z level=debug msg="got secret value" orgId=2 type=datasource namespace="demo cluster"
grafana-5b46cbf846-zs7g2 logger=secrets.kvstore t=2024-04-04T18:56:24.833746668Z level=debug msg="got secret value" orgId=2 type=datasource namespace="demo cluster"
grafana-5b46cbf846-zs7g2 logger=plugin.grafana-clickhouse-datasource t=2024-04-04T18:56:24.972726311Z level=error msg="sql: Scan error on column index 3, name \"leaf_cert_all_domains\": unsupport
ed Scan, storing driver.Value type []*string into type *string: Could not process SQL results"
grafana-5b46cbf846-zs7g2 logger=context userId=2 orgId=2 uname=xxxxxxx t=2024-04-04T18:56:24.97344002Z level=info msg="Request Completed" method=POST path=/api/ds/query status=400 remote
_addr=<IP_ADDRESS> time_ms=149 duration=149.282978ms size=242 referer="https://dashboards.trafficpeak.live/d/c13cbad3-20b7-42ba-afbd-67a70be92e46/certificate-transparency?from=now-5m&orgId=2&refresh=
30s&to=now" handler=/api/ds/query status_source=downstream

We are using Grafana in HA via k8s and we are using the following env variable to load it:
- name: GF_INSTALL_PLUGINS
  value: https://github.com/grafana/clickhouse-datasource/releases/download/v4.0.4/grafana-clickhouse-datasource-4.0.4.zip;clickhouse-datasource

I can ping you a set of credentials to our deployment if you want to try to reproduce it?

@aangelisc it feels like very similar to this issue:
https://github.com/grafana/clickhouse-datasource/issues/783

Reopening this, I'll share with our current triage to review 😊

Hi @dsztykman - I cannot replicate this either. If you could share a set of credentials and the dataset you're using so we can further investigate that would be great. Thanks!

Wanted to chime in and ask/suggest a few things:
Questions:

This was originally reported with plugin v3.x but it seems like it's still an issue with 4.x? (according to your GF_INSTALL_PLUGINS env var this seems to be the case)
Is this an issue with Grafana, the plugin, or the Go driver the plugin backend uses? It seems like this might be a driver level issue.
The query_type (Table, Logs, Time Series,  Trace) does apply some mutations to the data, but only in the case of Trace.
Lastly, and most important can you provide some DDL as well as a few INSERT rows to replicate this as minimally as possible? I see you provided some example queries but if you could also include some queries with the test DDL that would be helpful. Without the data there's no way to step through the error.

I am suspicious with how this error is appearing based on OSS/Enterprise versions. To my knowledge there shouldn't be any differences at that level of the application.
Hope to find the solution to this soon. Thank you for submitting this issue and providing an update recently

Yes it is still happening right now, the only solution we have is to cast into string so instead of selecting the map we actually do something like:
SELECT toString(requestHeaders) as RequestHeaders EXCEPT(requestHeaders)

We are using the query_type Logs as we want users to see the requestHeaders.
ddl_dump.zip
You can find the zip file which contains some rows and the DDL.
The "weird" thing is it doesn't happens all the time, for example we deploy grafana in HA and the query works for one of the container but not the other.

by the way we changed our schema to avoid that issue so on the dump the botData is not a map anymore but the requestHeaders still is and generate problem

Thanks for the info! I have loaded the table+data into my local ClickHouse server.
I've run a few different queries, SELECT *, SELECT requestHeaders, SELECT requestHeaders['Host'], etc. and none of them have errored yet. This is on OSS v10.3.3 (although I will try with the originally reported versions too).

The "weird" thing is it doesn't happens all the time, for example we deploy grafana in HA and the query works for one of the container but not the other.

Not sure what the query was in this case, but assuming Grafana+plugin are on the same version this could be something to do with the time ranges applied on the queries perhaps? For example, the first query has a different set of results from the second query.
I will try simplifying the table to just one column. I also want to note that the original issue said Map(String, String) but the DDL has Map(String, Nullable(String)).
Could you provide some more example queries, and confirm your most recently used Grafana + plugin version?
Thank you for your patience, this is issue is difficult to reproduce.

No worries I know it's difficult to reproduce because it doesn't happen on all our instances either...
This is the dashboard we use:
Akamai SIEM-1712941876761.json
You can try to start and stop grafana service and restart to see if it happens on it

Thanks! I loaded the dashboard and it works well with your sample data. I also added a few rows where Map(String, Nullable(String) would have a NULL map value.
Unfortunately, still no errors...
But I have reviewed the error logs you've pasted:
Scan error on column index 8, name "botData": unsupported Scan, storing driver.Value type map[string]*string into type *string: Could not process SQL results

Scan error on column index 3, name \"leaf_cert_all_domains\": unsupported Scan, storing driver.Value type []*string into type *string: Could not process SQL results"

It seems like this error could be happening with multiple types? Also, if you have a recent occurence of this error, you could find the original query in the system.query_log table and perhaps provide that information. That will show what the exact query was that triggered the error. Along with the DDL and time information this should be good information for reproducing the error, at least at the driver level.

Interestingly this error is on ruleActions which is an array of string...

@SpencerTorres I have upgraded the plugin to 4.0.6 which includes https://github.com/grafana/clickhouse-datasource/issues/783 and it fixes my issue.
I'm going to close this


add a drain timeout

same as https://github.com/weaveworks/kured/pull/79 but with latest changes
Closes: #78

@bboreham I added you in the reviewer list, since you already reviewed the PR 79 here https://github.com/weaveworks/kured/pull/79

@flbla Can you mention in your github PR: "Closes: #78" ? This will auto-close the related issue upon merge. Thank you.

Closes: #78

@evrardjp : I rebased my branch and added the close comment

Sorry. Can you please retarget your branch to be merged into main please?


chore: update overrides helper package reference in skeleton samples

Make the skeleton package.json reference to the versions of the override helpers to be released.

Codecov Report

Merging #8866 (c28c000) into master (1f55c6b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #8866   +/-   ##
=======================================
  Coverage   55.60%   55.60%
=======================================
  Files         868      868
  Lines       48594    48594
  Branches     9643     9643
=======================================
  Hits        27019    27019
  Misses      20705    20705
  Partials      870      870

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1f55c6b...c28c000. Read the comment docs.


is possible to have @include in filters

is possible to have @include in filters
example:
query searchPabl($search: String,
$judet: String, $lot: [Int!] = [1,2,3,4,5,6,7]) {
allRonetSirutas(filter: { or:[
{pabl:
{containsInsensitive: $search}
},
{pi:
{containsInsensitive: $search}
},
]
and:[{judet:
{containsInsensitive: $judet}
},
{lot:
{in: $lot}
} @include(lot: $lot) ]
})

Not currently.  I believe you would need to fork the graphql-js reference implementation (which PostGraphile depends on) to implement this.  Also, I can understand wanting to conditionally include a filter element with something like @include(if: $includeLot), but can you clarify what @include(lot: $lot) is intended to do?

Yes, I was asking if is possible to include a filter element in and: || or: like @include but this can be handled at an upper level logic

👍


NIFI-11685 Update zstd-jni to 1.5.5-4

Summary
NIFI-11685
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[X] Apache NiFi Jira issue created

Pull Request Tracking

[X] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[X] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[X] Pull Request based on current revision of the main branch
[X] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[X] JDK 11
[ ] JDK 17

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

The checks ran successfully for this in my own branch


fix getAuthCodeLink not a function

fix getAuthCodeLink or authCodeLink

It is not supposed to be a function. Refer to the documentation


Error 400 ???

Hello, I tried the bot like a business developper. It's ok when it seachs in Google but it blocks when it has to do actions with "Request failed with status code 400". What do you think about it ?

Having the same issue, did you find a solution?

No. It's always the same !


How can colorschemes support this plugin?

Perhaps highlight groups can be provided for the colors? E.g.: MarkidArgument1, MarkidArgument2, etc.

Hi @jssteinberg ,
That's an interesting idea. Currently, the group names are markid1, ..., markid#colors.length and set via
vim.api.nvim_set_hl(0, group_name, { fg = config.colors[idx] })

this function completely replaces the definition

But this can be overwritten:
                              • default: Don't override existing definition
                               |:hi-default|

So I could set default = true and allow plugins to set the group names (before or after).
Would that make sense?
Thanks and best regards,
David

I still would need to check if the highlight group already exists.

Perhaps the user would also need an option to use the plugin's colors if they want -- if a colorscheme does provide highlights for this plugin, but the user don't like them. (Dont know if that option should be in this plugin or the colorscheme, but guess it would be more intuitive to have that option in this plugin.)

So now, plugins can overwrite markid1 etc. but the color is always overwritten.
The problem is that the amount of colors is dynamic (depending how many colors the user provides).
So it's unclear how many highlight groups the plugins would need to provide.
I have to think about it.

Ok. I guess defining a Lua table would also work.

On a side note: Are you a plugin author willing to support the highlight groups? If yes, then I will increase the priority of this issue.

Hi @jssteinberg ,
I've implemented a small change. Now consumers can specify colors = false and give control to theme designers.
Theme designers can now define the highlight groups markid1, markid2, ... (as many as they want).
Best regards,
David

Hi, sounds good! Will give it a whirl soon.
I’m working on a couple of colorschemes, but nothing released yet.

Thanks @jssteinberg , I've now also added it to the README.

Awesome, thanks @jssteinberg!

I like the concept of the plugin but how can I retain the color scheme of my current color scheme?

I like the concept of the plugin but how can I retain the color scheme of my current color scheme?

I, for example, configured the markid colors based on the coloscheme palette I use and lightened them by 15%

@okuma10

I like the concept of the plugin but how can I retain the color scheme of my current color scheme?

At the moment, you need to provide your own colors in the colors setting.
@crivotz

I, for example, configured the markid colors based on the coloscheme palette I use and lightened them by 15%

That's an interesting approach!


[stable/datadog] enable custom logs in datadog

What this PR does / why we need it:
Adds the ability to mount host files to DataDog pods for consumption with DataDog's Agent Log Collection
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

/ok-to-test

@isaaguilar
Could you resolve the conflicting file please?
stable/datadog/Chart.yaml

+1

Hi, @xlucas , can you help review this and merge? This will enable shipping custom logs from the host node.

/assign @celenechang


Allow to skip searchable class check

As requested on the issue allow to set via a env setting (ALGOLIA_SETTINGS_SKIP_CHECK) if isClassSearchable check is be skipped

I think the issue stated in #26 is from something else, I'd rather find out before moving on with this issue.


A0-1546: Include benchmarks in unit test suite

Pretty self-explanatory

Aren't benchmarks a long running process? If so then maybe having it ran as part of unit test suite (on every commit) is not the best idea. How about we run it just before the merge (on merge commit)? We don't act on the results of benchmarks and having it run anyway on every PR will give us the traceability we desire.

@deuszx benchmarking is extensive due to repeating; the cases themselves go very fast:
running 27 tests
test tests::setup::__construct_runtime_integrity_test::runtime_integrity_tests ... ok
test tests::suite::stores_vk_with_fresh_identifier ... ok
test tests::suite::does_not_overwrite_registered_key ... ok
test tests::suite::does_not_store_too_long_key ... ok
test tests::suite::verify_shouts_when_no_key_was_registered ... ok
test tests::suite::verify_shouts_when_key_is_not_decodable ... ok
test benchmarking::suite::bench_store_key ... ok
test tests::suite::verify_shouts_when_proof_is_not_decodable ... ok
test tests::suite::verify_shouts_when_input_is_not_decodable ... ok
test benchmarking::suite::bench_verify_groth16_xor ... ok
test benchmarking::suite::bench_verify_groth16_linear_equation ... ok
test benchmarking::suite::bench_verify_groth16_merkle_tree_1024 ... ok
test benchmarking::suite::bench_verify_gm17_linear_equation ... ok
test tests::suite::verifies_proof ... ok
test benchmarking::suite::bench_verify_groth16_merkle_tree_64 ... ok
test benchmarking::suite::bench_verify_groth16_merkle_tree_8 ... ok
test tests::suite::verify_shouts_when_proof_is_incorrect ... ok
test benchmarking::suite::bench_verify_gm17_merkle_tree_1024 ... ok
test benchmarking::suite::bench_verify_gm17_merkle_tree_8 ... ok
test tests::suite::verify_shouts_when_verification_fails ... ok
test benchmarking::suite::bench_verify_gm17_xor ... ok
test benchmarking::suite::bench_verify_gm17_merkle_tree_64 ... ok
test benchmarking::suite::bench_verify_marlin_linear_equation ... ok
test benchmarking::suite::bench_verify_marlin_xor ... ok
test benchmarking::suite::bench_verify_marlin_merkle_tree_64 ... ok
test benchmarking::suite::bench_verify_marlin_merkle_tree_8 ... ok
test benchmarking::suite::bench_verify_marlin_merkle_tree_1024 ... ok

test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s

or actually 1.46s on lame GH runner: https://github.com/Cardinal-Cryptography/aleph-node/actions/runs/3463676349/jobs/5784243035#step:9:811


Inconsistency between f32 and f64 in convertability to RealField

The following function compiles:
fn my_convert<T: simba::scalar::RealField>(value: f64) -> T {
    T::from_subset(&value)
}

However, the following function does not compile (replacing f64 with f32):
fn my_convert<T: simba::scalar::RealField>(value: f32) -> T {
    T::from_subset(&value)
}

The non-compiling function results in error[E0308]: mismatched types with expected `&f64`, found `&f32`  as the error message.
When looking for a root cause, I noticed that complex.rs's trait definition for ComplexField has SupersetOf<f64> as one of its trait bounds but not SupersetOf<f32>. If I modify this code myself, I am able to get the f32 version of my_convert to work.
I was able to reproduce this issue in the master branch as of 2024-02-24 (commit 9d95d6df788386d602b058938068a16ca866a0f1)

Fixed by #59


Channels option in training.

I want to train 256 pixels images, and trained with this options:
!python run_training.py --channels [32, 32, 64, 128, 256, 256, 256, 256, 256] --data_dir images
but got error message bellow.
... some messages ...   run_training.py: error: argument --channels: invalid int value: '[32,'
Thanks!

Hey!
Command line arguments where there are multiple values for an option should be specified as
--channels 32 32 64 128 256 256 256 256 256 --data_dir ...
Also if you want to train with images of size 256x256 you need to reduce the number of channel values. The size of the output image will be 2 ^ (n_channels + 1). If the argument --base_shape is not changed that is.
--channels 64 128 256 256 256 256 256 should produce images of size 256x256.
Hope this helps!

Thank you very much.
But I'm not sure about this option.
If the argument --base_shape is not changed

base_shape=(4,4)
I got it. This is constant layer, isn't it!
It works!  Thanks a lot.

Yea, that is the constant layer. You can change the size of it the same way as you pick channels, i.e. --base_shape 4 4!


Add a link to Dockerfile for dependabot monitoring

The link allows Dependabot to monitor the Dockefile inside a dot directory. There is an unfortunate downside. The PR created by Dependabot will replace the link with the actual file with a new version. We should not merge this PR (produced by Dependabot), but bump versions in the Dockefile manually.

Is this really the best way to do this?  I feel like others have had to encounter this sort of thing before.


License

Reading between the lines of the readme it seems that this is licensed for other people to use, but there is no license file to make that explicit or to explain under what terms it is licensed. Any chance you could add a LICENSE file? I’d suggest a permissive open source license like the Apache or MIT licenses if your goal is to make it as easy as possible to use, which sounds like it was your goal.

Thanks for the comment, this was something that slipped my mind haha.
I've added the MIT license in 89f72e8dc13a90aecc075fdafb61202a6606c368 to follow similar projects, such as the schwab-api, robin_stocks, and webull repositories.
Thanks for bringing this to my attention!


[Event Change Request] Table 37 Sales Line - procedure CopyFromItem - event OnAfterAssignItemValues

We would like the Parameter/-s xRec, CurrFieldNo added to the following event.
Original:
Table 37 Sales Line - procedure CopyFromItem - event OnAfterAssignItemValues

OnAfterAssignItemValues(Rec, Item, SalesHeader);

[IntegrationEvent(false, false)]
local procedure OnAfterAssignItemValues(var SalesLine: Record "Sales Line"; Item: Record Item; SalesHeader: Record "Sales Header")
begin
end;

Change:
OnAfterAssignItemValues(Rec, Item, SalesHeader, xRec, CurrFieldNo); // <-- xRec, CurrFieldNo ADDED

[IntegrationEvent(false, false)]
local procedure OnAfterAssignItemValues(var SalesLine: Record "Sales Line"; Item: Record Item; SalesHeader: Record "Sales Header"; var xSalesLine: Record "Sales Line"; CurrentFieldNo: Integer) // <-- var xSalesLine + CurrentFieldNo ADDED (xSalesLine has to be declared as var like in OnAfterAssignFieldsForNo and others)
begin
end;

Availability update: We will publish a fix for this issue in the next update for release 22.
Build ID to track: 58441.


mod_http2: TLS throughput/record sizes

Current implementation of mod_http2 buffers output to manage desired TLS record sizes. This achieves much better throughput than passing frame meta data and resource data down the connection filters unchanged.
To illustrate the problem: serving response bodies using Apache's bucket brigades gives a small 9/10 bytes frame header bucket, followed by a larger bucket with the frame data (often 16 KB) and an optional bucket with padding bytes. Rinse and repeat.
Existing mod_sslconnection filters try to coalesce small buckets into larger ones for more efficiency, but are unable to manage a chain of 9+16K+9+16k+... But small TLS record sizes produce too much overhead. Copying all data into larger buffers and sending those give currently better performance. But the copying should be optimized to look at the bucket patterns and only copy data when necessary for better TLS record sizes.
This should be implemented as part of a new connection output filter in mod_ssl and replace the implementation in mod_http2. This would then also benefit http/1.1 connections.

Existing mod_sslconnection filters try to coalesce small buckets into larger ones for more efficiency, but are unable to manage a chain of 9+16K+9+16k+... But small TLS record sizes produce too much overhead. Copying all data into larger buffers and sending those give currently better performance. But the copying should be optimized to look at the bucket patterns and only copy data when necessary for better TLS record sizes.

I wasn't aware this was a problem for mod_http2 as well; the buffering done in h2_conn_io.c exists to mitigate this problem?
The mod_ssl coalesce filter was made smarter on trunk but still won't cope with cases like 9+16K+9+16K.  Will ponder this one.

Yes, I measured it and it was quite a difference. However I was not certain about changing the ssl filters and thus introduced the buffering.
This is a very h2 specific pattern, so if you do not find a good way to do it in a generic way, it should be fine.


Blurry icons

Contact Details
<EMAIL_ADDRESS>What happened?
Linkedin logo not visible clearly and Github Repo button is blurry
Relevant log output
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@maityamit please asign the issue


[BUG] 24 is not a valid MessageType

Describe the bug.
When a message gets blocked by Discord Content Moderation Beta it will send Message Type 24 which isn't currently registered and therefore panics.
List the steps.

Open Discord Server Settings
Enter  tab
Block some words

Type the blocked words in a channel the bot has access to.

What you expected.
I expect it to gracefully ignore or handle the Message Type.
What you saw.
Instead, I received this traceback error given from my Python terminal:
Traceback (most recent call last):
  ...
  File "C:\Users\Justin\Projects\private\.venv\lib\site-packages\interactions\client\bot.py", line 124, in start
    self._loop.run_until_complete(self._ready())
  File "C:\Python310\lib\asyncio\base_events.py", line 646, in run_until_complete
    return future.result()
  File "C:\Users\Justin\Projects\private\.venv\lib\site-packages\interactions\client\bot.py", line 363, in _ready
    await self._login()
  File "C:\Users\Justin\Projects\private\.venv\lib\site-packages\interactions\client\bot.py", line 368, in _login
    await self._websocket._establish_connection(self._shard, self._presence)
  File "C:\Users\Justin\Projects\private\.venv\lib\site-packages\interactions\api\gateway\client.py", line 189, in _establish_connection
    await self._handle_connection(stream, shard, presence)
  File "C:\Users\Justin\Projects\private\.venv\lib\site-packages\interactions\api\gateway\client.py", line 257, in _handle_connection
    self._dispatch_event(event, data)
  File "C:\Users\Justin\Projects\private\.venv\lib\site-packages\interactions\api\gateway\client.py", line 390, in _dispatch_event
    self._dispatch.dispatch(f"on_{name}", __obj(**data))  # noqa
  File "C:\Users\Justin\Projects\private\.venv\lib\site-packages\interactions\api\models\message.py", line 267, in __init__
    self.type = MessageType(self.type) if self._json.get("type") else None
  File "C:\Python310\lib\enum.py", line 385, in __call__
    return cls.__new__(cls, value)
  File "C:\Python310\lib\enum.py", line 710, in __new__
    raise ve_exc
ValueError: 24 is not a valid MessageType

What version of the library did you use?
stable
Version specification
4.2.1
Code of Conduct

[X] I agree to follow the contribution requirements.

I thought I would also post a message JSON:
{'type': 24, 'tts': False, 'timestamp': '2022-06-16T02:19:10.998000+00:00', 'pinned': False, 'mentions': [], 'mention_roles': [], 'mention_everyone': False, 'member': {'roles': ['956263275887218808', '981189079263764530', '970332150891155607'], 'premium_since': None, 'pending': False, 'nick': '<redacted>', 'mute': False, 'joined_at': '2022-03-25T22:48:57.246000+00:00', 'flags': 0, 'deaf': False, 'communication_disabled_until': '2022-04-02T03:16:14.258000+00:00', 'avatar': None}, 'id': '<redacted>', 'flags': 0, 'embeds': [], 'edited_timestamp': None, 'content': '', 'components': [], 'channel_id': '<redacted>', 'author': {'username': '<redacted>', 'public_flags': 0, 'id': '<redacted>', 'discriminator': '9990', 'avatar_decoration': None, 'avatar': '<redacted>'}, 'attachments': [], 'guild_id': '<redacted>', '_client': <interactions.api.http.client.HTTPClient object at 0x000001B126B329D0>}

I would also like to mention a very minimal change of just adding CONTENT_MODERATION = 24 to MessageType with no other change at all seems to work fine. However, I'm not sure in what context Message object will be used so I cannot say it's a full-fledged fix. I have not read the code enough to say that.

Is this implemented in unstable?

Is this implemented in unstable?

It is not implemented as of latest stable nor latest unstable branches:
stable: https://github.com/interactions-py/library/blob/stable/interactions/api/models/message.py#L37=
unstable: https://github.com/interactions-py/library/blob/unstable/interactions/api/models/message.py#L71=

Hiya!
Currently, the message type points to an undocumented feature in Discord's side of things, but there's already a PR (#768) made to interface with automod, including the messagetype. Moreover, there may also be an unstable PR on the side (talked about at #unstable in the support server) that may deal with conversion issues like this that can also deal with the issue in a different approach.

Closing this issue as the AutoMod PR has been merged, docs are reflected and as per the issue, the MessageType is defined ^^


#717: Changing the property prefix to "versions"

Alternative to #718, changing the prefix to "versions" since it's already being used for versions.skip.

https://github.com/mojohaus/versions-maven-plugin/issues/717


Does Swagger Node only produce raw doc endpoints? Does it generate a UI endpoint as well?

Hi,
Does Swagger Node produce a URL such as /swagger/ui/index.html to serve the HTML human readable documentation?
Or do we have to manually install swagger-ui and configure it?

We do intend to support this, but unfortunately you'll have to install and configure it manually for now.

You can look at this file as a sample of doing it manually.

@theganyo no worries. Looking forward to it :smiley:
@mohsen1 I just installed swagger-ui and configured my Express app to serve it via static middleware. But ideally I was looking for it to be configured automatically.

Closing as duplicate of #4.


Add different license options (currently only supports MIT).

Beside the open source licences, they should be the possibility to use a "private" licence.
Use Case
I'm currently writing an application for my company intranet. There is data in there that should not go out.
A "private" licence option should let me write the licence file and the text that will be injected in my files.
Then, using licence-injector, I would be able to put a licence header in all my files.
That would also be useful for OS projects. There are so many OS licences out there... You would provide for the most used licences, but people would be able to add a licence that isn't provided by your nice extension.


Incorporate progress bar for file uploads and downloads

It appears that the ubiops.utils.upload_file function doesn't have a progress_callback argument. To track the progress of the file upload
from ubiops import utils

#Upload a file
file_uri = ubiops.utils.upload_file(
  client = api_client, #a UbiOps API client
  project_name = "my-project",
  file_path = "file",
  bucket_name = "your-bucket",
  file_name = "files/file"
)

@Sascha-UbiOps @ricardo-ubiops @elif-ubiops

Hi,
The ubiops.utils.upload_file function already shows an upload progress bar by default. You can turn that of with the _progress_bar=False parameter.
Can you explain why you want to be able to configure the progress_callback yourself?
Kind regards,
Sascha


Adding side view to mssautoplot CLI tool

The mssautoplot CLI tool facilitates downloading of top view plots. This PR adds the functionality to download side view plots along with top view plots. The user has to specify the parameter view which can take the value' 'top' or 'side'. By default it is set to download top view plots.

@swsrkty please sync with develop branch


Multithreading?

Is multithreading support coming to wasmtime in the forseeable future?

Blocked on https://github.com/CraneStation/cranelift/issues/582

Yep, and the bigger picture is, multithreading is really important to us, so while I don't have a specific timeframe yet, it will happen.

Looks like we have a few of these issues open in the issue tracker so I'm going to close this in favor of a dedicated tracking issue -- https://github.com/bytecodealliance/wasmtime/issues/888


Question about deauther/troubleshooting.md

I have a question regarding the information on the Wiki at deauther/troubleshooting.md

I'm having trouble with deauths- my first question is does "increasing pkts/s" mean increasing the value in "deauths-per-target"?

That is correct

That is correct


Gesture Control added for Bounce

Issue #94
@TusharAMD I have added bounce gesture control. pls check and merge

Content copied from https://github.com/uvipen/AirGesture/blob/master/battle_city.py
Remove labels
cc: @TusharAMD


Trust Crev Id without a URL

Currently crev id trust wants to know a URL for the Id.

Error: URL not found for Id yadayada; Fetch proofs with fetch url <url> first

This means I can't tell others "I'm X98FCpyv5I7z-xv4u-zMWLsFgb_Y0cG7p5xNFHSjbLA add me to crev!" and requires sharing of URL + Id together. It also nudges users towards sharing just the URL, because an Id can be read from a URL.
If crev could create a trust proof without the url field, it would make crev Ids more practical for sharing as the identity, rather than a secondary thing with URL being the identity.
Related to #298

That is true ... . I guess we could make it an Option<...>


Add ReadOnly property to TextBox component

This PR adds a property ReadOnly to the TextBox component. This addresses issue https://github.com/mit-cml/appinventor-sources/issues/1841.

@matthewcoufal Are you sure that this has the correct behavior? I thought the original issue is that setting Enabled = false doesn't allow text to be selected in the textview, and on the surface this appears to me to not do anything different. Am I missing something?

Can one of the admins verify this patch?

@AppInventorWorkerBee OK to test


Reaching level 100 cause error.

There's no event "level up" when you reach level 100, it just goes silently and cause a error in the game.
2022-05-15.zip

fix in https://github.com/RogueCollab/RogueEssence/commit/6b9b68d91481506a1c68a916b556b24893414f8d


Added integration tests

quite a large LOC, if still interested in merging i'd ask the following:

find a smaller tokenizer config
explain makefile & vendor changes
not sold on distributing prebuilt binaries this way - wouldn't we have to install all combinations even if we only use one?


[BUG] - Opening multiple projects

Describe the bug
When opening multiple projects in succession, a strange effect occurs on the show / hide sidebar.
Seeing the video you understand very well.
To Reproduce
Steps to reproduce the behavior:

Open a project
Open a project other than the first
Open another project

Screenshots
https://user-images.githubusercontent.com/20476002/160005842-1034d302-248c-4d4b-8c05-b5114760d193.mov
Desktop (please complete the following information):

OS: macOS 12.3
Xcode: 13.3

I guess, it happens because NSToolbars have the same identifier in all projects. I haven't been digging whether there are any events which allow multiple NSToolbar to communicate based on identifier. But when I set NSToolbar.identifier to a value which is different for each window (e.g. same as WorkspaceDocument.fileURL), then everything works as intended

So, we can just set NSToolbar.identifier to some value unique for each project.
@lukepistrol, will it be okay if set NSToolbar.identifier to WorkspaceDocument.fileURL?
window.toolbar = NSToolbar(identifier: NSToolbar.Identifier(fileURL?.absoluteString ?? "unknown-workspace.toolbar"))

Do we need to reference this identifier somewhere for anything? If not using a UUID string would be the best and safest option

Do we need to reference this identifier somewhere for anything? If not using a UUID string would be the best and safest option

We don't need to reference it anywhere. Then, I'll just use a UUID string


Greetings, Jenkins HTML report is not displaying the charts and animations from Jenkins. However, opening the same HTML file locally works perfectly. I have seen some old posts but if anyone can direct me to the solution would be great. thank you.

Originally posted by @tobiswajitdas in https://github.com/trivago/cluecumber-report-plugin/issues/97#issuecomment-431734596

Hello @tobiswajitdas
Please see https://stackoverflow.com/questions/35783964/jenkins-html-publisher-plugin-no-css-is-displayed-when-report-is-viewed-in-j about the jenkins content policy settings.


Add Expect Event action

Make it possible to define an event structure that you expect. E.g.
Expect
{
   "user": "some_id",
   "text": "text goes here"
}

This would then verify an event is received with a user with id "some_id" and the text is "text goes here".

Closed by #44


PSF star selection notes

The objective of this issue is to gather notes on the various methods for selecting stars to modelling a PSF discussed during the reading group.

@andrevitorelli's notes in Google doc

@sfarrens's notes:
Star Selection

CFIS (PSFEx & MCCD)

Cut in magnitude-size (FWHM) plane
Assume size does not depend on luminosity (stellar locus)
Set range around the mode of the distribution to select stars
Difficult to distinguish faint galaxies from stars
Make a magnitude cut to remove these objects
This is the input of the PSF model
Some bright stars appear larger, cut at low mag to avoid this
brighter-fatter?

not a problem for MegaCAM

MCCD rejects outliers in terms of shape or size

LensFIT

Strongly dependent in star rejection
Start with a large sample of stars

HSC

Modified PSFEx (no star selection)
Need to address brighter-fatter effect
K-means selection of stellar locus
Selection done per CCD (~72 stars/CCD)
Method sensitive to seeing

KiDS

Friends-of-friends
2nd and 4th order moments of all objects
5 dithers, hence 5 points of measure for each star
Combine all exposures
Needs to be selected as a star in at least 3 exposures
Performs better than CFHTLenS, which used co-added images
claim <1% outliers
empirically determined linking parameters
skewness and kurtosis of light profile, stars should be peaky

DES

Size-magnitude plane
Pipeline includes a brighter-fatter effect correction
Flat stellar locus due to correction
Can include lower magnitude stars
Good match to Gaia

Other methods

Size-magnitude + colour information

Brighter-Fatter

Not all cameras suffer from this effect
Thicker the CCD, the stronger the effect
Affects all stars, strongest for brightest stars
Correction is robust, but takes a lot of work to implement

Completeness

Some surveys require more than others to model the PSF
Euclid wants 100 stars/CCD
Purity more important
Gaia has enough star coverage for many surveys

Perhaps not deep enough
Also not perfect
Parallax information
Very small astrometric error
Match stars to surveys or simply take Gaia positions and create postage stamps

Remove CCDs with fewer stars than the requirement to model the PSF
Zhang et al. (2019)
Ma et al. (2008)

Discussion, how to structure star selection in the book.
We agreed that the focus should be on the methods rather than surveys. We can cite papers and mention surveys with regards to specific details in each section.
Proposed Subsections

Subsection on size-magnitude plane and how to identify the stellar locus

This section will be longer than the others and include more detail
Important to include examples of both clear and messy loci

Four order moments-bases methods
Colour-based methods
Experimental methods (ML from pixels)

Also the use of GAIA

Important Details

Emphasise the importance of sample purity and this is why simple but trusted methods are often employed

Minor Details

Binary stars


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-master" workflow

Codecov Report
Patch and project coverage have no change.

Comparison is base (25904ad) 100.00% compared to head (32678ea) 100.00%.

Additional details and impacted files
@@            Coverage Diff            @@
##            master      #188   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            3         3
  Lines           52        52
  Branches        15        15
=========================================
  Hits            52        52

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


scan report file name depends from report type

Addresses #1942
Define a different default based on report type (e.g. junit wants xml file) otherwise fallback on previous behaviour.

+1 for this PR since CircleCI isn't picking up the junit test output because it doesn't have a .xml extension (I believe). I think the type-to-extension logic deserves its own method though:
...
    extension = report_extension(type)
    file_name = "report.#{extension}"
...

def report_extension(type)
    case type
    when "junit"
        return "xml"
    else
        return type
    end
end

(my Ruby is novice, but that's the idea)

The extension logic based on report type is not the best way.
Remember that report was generated by xcpretty so output could have very disparate usages.
Thing (for example) if this action does not have a whitelist for the report type, we could specify also the json-compilation-database report that generates a special db used from some other action (for coverage and so on) and they excepted exactly a compilation.json file. Using the method above you must hope that the next actions have the capability to specify the input parameter otherwise action will not find the file it are looking for.
Using the default name expected as output facilitates writing lane saving configuration of input parameters / output of following actions.
No one prohibits even have a parameter for the output file name, but at this point instead of doing joins  between name and extension as the default behavior I would keep the report type parameter to decide a default name and obviously overwritable by output_file parameter which in this case must contain both name and extension to have maximum freedom.
Keep in mind that these kind of changes on a widely used action as is this one, will force to review all lane script of a lot of used that would update

Hi @nfalco79 - thanks for the contribution! 😄
While I agree with you and the original ticket that the JUnit report name would be better as TEST-something.xml, I don't think we can go forward with the approach here.
Because the report.junit filename has been the default for so long, we can't change it because some large number of people may currently be depending on that file name during their build process.
I think it would be best to introduce a new ConfigItem option that more explicitly describes the desired "output type" to "output file name" relationship. Please let me know if you'd like to discuss how that might look and work 👍


App fails to rerender when launched with an External Accessory [EAAccessory] in iOS

Steps to reproduce

Create a hello world flutter app
Add supported external accessories
Enable External Accessory background mode
App must be built/ran using --release (because it will be launched in an unattached state via the accessory)
Launch app via connecting external accessory

May be related to https://github.com/flutter/flutter/issues/55969
I have confirmed that this launch method works on flutter 1.12 pre-metal as per the above issue

Expected results
App should launch and render hello world flutter create screen
Actual results
App fails to render hello world screen and gets stuck at launch/blank page
Code sample
Code sample
import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or simply save your changes to "hot reload" in a Flutter IDE).
        // Notice that the counter didn't reset back to zero; the application
        // is not restarted.
        primarySwatch: Colors.blue,
        // This makes the visual density adapt to the platform that you run
        // the app on. For desktop platforms, the controls will be smaller and
        // closer together (more dense) than on mobile platforms.
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key, this.title}) : super(key: key);

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Invoke "debug painting" (press "p" in the console, choose the
          // "Toggle Debug Paint" action from the Flutter Inspector in Android
          // Studio, or the "Toggle Debug Paint" command in Visual Studio Code)
          // to see the wireframe for each widget.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}```

</details>

### Screenshots or Video

<details open>
<summary>Screenshots / Video demonstration</summary>

[Upload media here]

</details>

### Logs

<details open><summary>Logs</summary>

```console
[Paste your logs here]

Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.27.0, on macOS 15.0 24A335 darwin-arm64, locale
    en-US)
    • Flutter version 3.27.0 on channel stable at /usr/local/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 8495dee1fd (6 days ago), 2024-12-10 14:23:39 -0800
    • Engine revision 83bacfc525
    • Dart version 3.6.0
    • DevTools version 2.40.2

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/davewesterhoff/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      17.0.6+0-17.0.6b829.9-10027231)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 16A242d
    • CocoaPods version 1.16.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      17.0.6+0-17.0.6b829.9-10027231)

[✓] VS Code (version 1.95.3)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.102.0

[✓] Connected device (4 available)
    • macOS (desktop)                 • macos                     • darwin-arm64
      • macOS 15.0 24A335 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad     • darwin
      • macOS 15.0 24A335 darwin-arm64
    • Chrome (web)                    • chrome                    •
      web-javascript • Google Chrome 131.0.6778.140

[✓] Network resources
    • All expected network resources are available.```

</details>

@dwesterhoff-wdc
Thanks for the report. I am not much aware about EAAccessory you mentioned. Can you elaborate how to set it up and what hardware are you connecting using it ? Ex: if bluetooth, are you trying to launch the app via it ? I am not sure how so please elaborate on it as well. A short video demo would be useful too, to get more clear idea of the issue you are reporting.

Thanks for the update. I will keep the issue open for team's tracking. Meantime, I will try to set it up using the details provided above to see if I can replicate this or not.

For when we find time to look into this: looks like MFi accessories include items like certain hearing aids but also digital car keys, bicycle cadence trackers etc.