[image_picker] Fix exif orientation for pictures taken from camera

Added ability to automatically rotate picture taken from camera, with exif preservation.
Description
Problem:
Some phones take pictures with rotation right, top which causes photo to "rotate" 90 degrees when exif data is stripped.  This PR adds one variable to automatically rotate picture and write right exif data.
Related Issues
This fixes this: https://github.com/flutter/flutter/issues/19458
looking forward to suggestions / changes to this PR. Thanks!

I think this needs end-to-end tests to verify that for each physical orientation of the phone, the output image has the correct orientation and EXIF data.

@Hixie good idea, will try to add some.

@charafau I think what we really should do is to copy all the meta data over to the image, instead of just copying rotation. Also, it is prob not necessary to have it as an optional flag. The captured/picked image/video should have the correct meta data by default.

Hello @cyanglaz sorry for lack of movement with this one, I was busy with different plugin. Yes I think you're right, I should copy all exif data. and add tests. Not sure if we want to make it a default behaviour though, I will leave that decision to you ;)

@charafau Thank you for the quick response! I was thinking about this yesterday and I think that since the cropped image is actually not the original image, does it make sense to copy the exif from the original image?
Instead, we can also return the original image path from the pickImage/video API along with the cropped image path, that way if you want to access the original image's exif, you can get it from the original image. And the cropped image will contain whatever metadata the iOS or android platform decides put inside the image data. Will that solve the problem?
I think it will be more robust and easier to maintain, since we don't have to do these surgeries on metadata of the image.

@cyanglaz sorry for late reply. This code doesn't crop the image, it just rotates it, so it will be displayed correctly in portrait mode, when you strip exif data (some phones mount camera differently).
About your suggestions - yes we could just strip the exif data, but I'm not sure if that's what users would like to do (Imagine photo gallery with location of pictures and camera info).
Other aspect of it would be, is current API good for it? Meaning, would it be better to pass parameter rotate or create new method like pickRotatedImage ?

@charafau You are right. I have been reading all the issues that people have opened for image picker and I think people do want the exif to be copied over to the new image. Let's do that?
There is a PR doing something similar on the iOS side, for your reference. https://github.com/flutter/plugins/pull/866
Are you able to keep working on this PR?

@cyanglaz Yes, I think I should be able to. Might ask for help with tests on gitter but I will make it somehow.

@charafau I have a PR related to this ready if you want to take a look.
https://github.com/flutter/plugins/pull/1586#pullrequestreview-235881807

I made a plugin to fix this, hoping it's useful for you:
https://pub.dev/packages/flutter_exif_rotation

Sorry for slow progress, I'm on vacation now and cannot will be back soon-ish. Plan to do that when I'm back

@charafau I'm going to close this PR since it has been inactive for a while. :) I encourage you to open another one when you are ready with the fix!


HELP! HOW TO FIX THIS ERROR: Got unexpected extra argument

HELP! HOW TO FIX
Error: Got unexpected extra argument (<download_directory>)

I also got this error

You want to pass your local directory in with the -d option, the documentation in the README is out of date.

For example:
icloudpd -d ./Photos --username my-user@gmail,com --password abc123

Guys, the documentation is out of date, the example above is not fully correct either.
Here is a correct example which works 98% of time, if not do let me know.
icloudpd -d /pathToDesiredDirectory/DesiredDirectoryName -u<EMAIL_ADDRESS>Press Enter.
You will be prompted to enter your iCloud password and enter it precisely.
You will then be taken through two factor authentication.
Then this software will download your stuff.
I know because I used a Linux machine, namely Ubuntu and it worked. The same exact instruction work on Mac once you installed python3.7 and pip inside python3.7 and did pip upgrade and installed all necessary dependencies. Let me know if this was helpful.

@metahnetoutage - good job and thank you!  I found this script is very useful and your clarification here helped me to achieve a big step. Just let you know that, I am now to this script and looking for more functionalities, such as:

list all of the directory names on the icloud;
download from the "Shared Albums";
understand the actually meaning of --recent 500 \ (my test didn't download the recent ones);
understand the actually meaning of --auto-delete \ (my test didn't delete files on icloud) -- actually I don't want to delete files on icloud with this scripts, just sync a given directory on icloud would be greatly helpful.
...etc..., any those info you know would be greatly appreciated. many thanks!  btw, I am willing to work with your guys to maintain and further develop this wonderful scripts!! many thanks to @ndbroadbent as well!


CI/CD Docs Updates

Proposed changes:

CI/CD Docs Updates

Status (please check what you already did):

[ ] added some tests for the functionality
[x] updated the documentation
[ ] updated the changelog (please check changelog for instructions)
[ ] reformat files using black (please check Readme for instructions)

@desmarchris Made some changes to the docs to include a CI pipeline example, let me know what you think and if it makes sense

@b-quachtran @rgstephens do you mind also reviewing the CI Pipeline Overview section I added above the new example? I left some questions in there that I was hoping you would have more information about for best practices.

@desmarchris Added some suggestions about the intro section, let me know if it makes sense

@b-quachtran @rgstephens updated again based on comments

@desmarchris LGTM 🚀


Review Description of Primary Key

This first sentence in the section describing a primary key is not correct - it's said here that the primary key uniquely identifies the whole table instead of rows within the table.
https://learn.microsoft.com/en-us/azure/synapse-analytics/database-designer/concepts-database-templates#primary-key

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 5a1ca838-548a-348a-d5be-b1fac8d1a3f3
Version Independent ID: 3b9ef3a8-6939-6dec-96b3-230dbbf12b37
Content: Azure Synapse database templates concepts - Azure Synapse Analytics
Content Source: articles/synapse-analytics/database-designer/concepts-database-templates.md
Service: synapse-analytics
Sub-service: database-editor
GitHub Login: @gsaurer
Microsoft Alias: gesaur

@doyinosin
Thanks for your feedback! We will investigate and update as appropriate.

@doyinosin
Thanks for your feedback! We will investigate and update as appropriate.

@svchandramohan
Could you please review.

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. We are closing this issue for now, but if you feel that it's still a concern, please respond and let us know. If you determine another possible update to our documentation, please don't hesitate to reach out again. #please-close


Consider sub-crate feature exclusions for audits

Generally speaking cargo-vet always operates at crate granularity, and we've resisted anything finer-grained on the grounds that it's difficult to reason about, difficult to implement, and of questionable utility in practice.
Earlier today, @selenography was describing his in-progress audit of reqwest, and asking whether there was a way to exclude the socks feature, which was a lot of unfamiliar code which we aren't using anyway. This led @mystor to formulate a feature description that I thought was interesting enough to at least document and discuss further.
The key insight is that cargo-vet already assumes that any code not reachable by enabling all features on the top-level crates (i.e., passing --all-features to cargo-metadata) can be excluded from analysis. At the moment this happens by dropping conditional dependencies which can never be built, because that's all cargo-vet knows how to do. However, we could also apply this same conditionality to the inclusion/exclusion of audits.
A rough sketch of this would be an optional exclude-features = [...] field on audit entries. Cargo-vet would only consider the audit if it were able to verify that none of the features in the list could ever be enabled on that node in the crate graph. Getting this information out of cargo-metadata might be anywhere between trivial and impossible, I haven't investigated.
You could then imagine an additional add-features = [...] field, which would allow you to patch in coverage to an existing audit. So if @selenography audited reqwest-x.y.z without socks, and somebody comes along and wants to use socks, they could just audit the socks part, and cargo-vet could combine the two.

In terms of how difficult exclude-features is to implement, I think it should be fairly straightforward. Cargo doesn't support disabling a feature in a dependency when a feature is enabled (features are additive) so when we run cargo metadata with --all-features, we're seeing the maximal crate graph from the metadata command. We already have the list of enabled features for each package node in the graph (https://docs.rs/cargo_metadata/latest/cargo_metadata/struct.Node.html#structfield.features), so we could effectively disable an audit edge with exclude-features = [...] if any of the listed features are present in the given node.
This has a slightly awkward interaction with the evolution of crates over time, as a given feature could e.g. no longer exist in a newer version (as all checks are done against the current version in the dependency graph, as we don't know e.g. "what features would be enabled on a previous version X of this crate"), or the feature could be renamed.
The add-features idea is considerably more complicated, as it would require being able to combine together multiple separate audit paths to prove a single criteria for a crate, which isn't something our architecture is capable of handling. It might be possible to do with some changes to the resolver, but I haven't looked into it.

We'd also benefit from this on Fuchsia. There are a number of crates with features that we don't review code for, since we don't ever enable it.

Chromium will also benefit from this as we attempt to adopt vet.

I note that audits that ignore a particular crate feature are just one kind of a partial audit.  Another example of a partial audit is ignoring target platforms / OS-es that are irrelevant to the auditor - e.g. Chromium may want to ignore libc-0.2.152/src/unix/hurd/… while still auditing code for platforms relevant to Chromium.  OTOH other auditors may ignore some of the Chromium platforms.
I guess the observation above doesn't necessarily mean that "sub-crate feature exclusions for audits" wouldn't be useful to have in cargo vet.  OTOH, maybe this observation helps to look at the problem from another angle - for example, maybe just knowing if an audit covered the whole crate is sufficient in many cases.
Just my 2 cents...


Function filterSearchHighlight translations EN, AR, ES, FR, IT

We need translations of Documentation for filterSearchHighlight:

EN
AR
ES
FR
IT
URL: https://ilia-brykin.github.io/aloha/#/filter-search-highlight

ready to test

Ok

@shafedh We need translations of Documentation for filterSearchHighlight again.
I improved and changed the structure.
docs/src/views/Functions/PageFilterSearchHighlight/i18n (Arguments)

ready to test

Ok


Feedback : Git & OpenSourcing

Please Provide us your Valuable Feedback

How to pull files from a repo?

@abdullah-abwisk  when you create a change on Github Repository
You can pull from my following commands
git pull

Hello Dear Adil How are You ?
I was apply for GitHub Pack in 2 December 2020 but still I have not receive an email and my request has been approve but I  have not receive an email of Developer student Tool Pack please tell me what I to do now ?

It went quite well!

Hey @ubaidrao
You Request is already approved and will expire on DEC 2, 2021

@jawadaminkp  Thanks buddy

@jawadaminkp Thanks

Could you kindly guide about the process of becoming a Github Campus Expert? What are the prerequisites, what should we learn before the application in February 2021, what are the benefits?
Thank you.

@adilshehzad786 It was really an amazing session I learn a  lot of new things

Hey @omarsaleem101
You can check all the details here http://githubcampus.expert/
According to my experience

Github Commit & Contributions matter
Approved GitHub Student Developer Pack
Your Portfolio on GitHub pages (Optional)
Polish your GitHub Portfolio Creates chances to become a GitHub Campus Expert.

@ahsanmangal  Thanks

That was a remarkable webinar!

Thanks @AyeshaSiddi

It was indeed a very fruitful session Mr Adil. You delivered it the best way possible. It added to my enthusiasm for adopting professional practices and I am now more empowered on my way to becoming an employable graduate. Young minds like you with such technical expertise happens to be inspiration for many beginners like me. Keep Inspiring your way.
Much obliged!

Thank you so much   @jawadaminkp

This session was a quite interesting as well as very interactive further through this i know about industrial standards for version control thank you adil for this tremendous webinar

Thanks, @MuhammadZainTariq1999


[2.x] Dashboard customisable line width.

Use case:
The lines in the graphs and other dashboard elements are too thin. It's hard to see the lines from a distance (for instance when you put them on a big screen as a dashboard).
Proposal:
Editable line width. Also it would be great to be able to further customize width and look of other dashboard elements.
Current behaviour:
It's not possible to edit line width for the graphs.
Desired behaviour:
Customisable line width etc.
Alternatives considered:
Using grafana instead as it supports more customisation on graphical elements.
Thing is that I like how influx DB UI looks, it's almost at a point where it can replace grafana completely for me.

Please think about opening a pr to implement the change you desire.


Fix: execution_count message interfers with spinner

Bring back the smooth spinner

Thanks for fixing this! I started running into this bug after making the execution count changes, but couldn't figure out exactly what was going on.


some question about the code

Hi, excellent work.
I am trying to understand your code. But I am very confused about the following lines of code:
https://github.com/idstcv/InMaP/blob/main/main.py#L289-L299

I roughly know that this code calculates the gradient of KL divergence to update the vision proxy iteratively.
However,  I can not derive a formula to for it.  Can you help me or give me some references?
Thanks in advance.

Thank you for your interest. By getting rid of the constant term, the loss can be degenerated to a cross-entropy loss with the soft label as in Theorem 2. Then, the standard gradient of the cross entropy loss can be applied, for which you can find the analysis for the cross entropy loss conveniently by google.

Thanks for your prompt response. I understand your point of replacing KL divergence with cross-entropy loss.


fix: Initial theme breaks after update to v4.9.0

🐛 Bug Report
I bumped my app to v.4.9.0 and now my theme's acting weird. I'm using theme.js to set up my default theme (accent color, density, etc.), same as Aspire project. But here's the thing: when my app loads, my custom styles show up for a sec and then they get replaced by the default styles. I found that this behavior was introduced by https://github.com/microsoft/fluentui-blazor/commit/544a9c4fb7ba5363794b9ed6e7c45e6fca0745e6, so when i use any of design tokens it's importing new file with suffix 4.9.0 which i overwriting theme.
💻 Repro or Code Sample
https://github.com/bpsc-wkubis/fluentui-v4.9.0-breaks-theme
🤔 Expected Behavior
Default theme is applied and not overwritten when using any of DesignTokens
😯 Current Behavior
Default theme is overwritten when using any of DesignTokens, in repro i used Density.
💁 Possible Solution
🔦 Context
Set default theme for app
🌍 Your Environment

OS & Device: Windows on PC
Browser Microsoft Edge
.NET and Fluent UI Blazor library Version 8.0.6 and 4.9.0

I was just researching this same issue, as soon as you use design tokens it changes the theme back to defaults

I'm undoing the add suffix for the next release (4.9.1).

Confirmed this is fixed with reverting add of suffix. Will be in 4.9.1


feat(cmd/rpc): Automatically detect running node for RPC requests

Closes #2859.
TL;DR Previously, the node.store flag had to be specified manually for each RPC request. This commit introduces automatic detection of the running node.
Assumptions:

presence of lock indicates a running node
specific order of networks (mainnet, mocha, arabica, private, custom) and type (bridge, full, light)
1 network will only have 1 running node type. multiple nodes of same network, same type are disallowed (receive Error: node: store is in use).
auth token, other flags still retain prev behavior and have priority
address, port read in from config
skipAuth skips auth for trusted setups
aligns with Unix daemon conventions

Sample Test Cases

Node store set

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY= --node.store=$NODE_STORE
{
  "result": {
    "namespace": "AAAAAAAAAAAAAAAAAAAAAAAAAEJpDCBNOWAP3dM=",
    "data": "0x676d",
    "share_version": 0,
    "commitment": "0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=",
    "index": 23
  }
}

Node store not set, but flag specified

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY= --node.store=
Error: cant access the auth token: root directory was not specified: unable to load in config: open config.toml: no such file or directory

No node store flag specified, yay

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{ // valid result, same as 1. }

Multiple networks running, will go to mainnet before mocha

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{
  "result": "RPC client error: sendRequest failed: http status 401 Unauthorized unmarshaling response: EOF"
}

Multiple networks running, will go to mocha before arabica

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{
  "result": "header: given height is from the future: networkHeight: 802095, requestedHeight: 1318129"
}

Run node with rpc config Address: <IP_ADDRESS> and Port: 25231 -- accurately directs request

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{
  "result": "RPC client error: sendRequest failed: Post \"http://<IP_ADDRESS>:25231\": dial tcp <IP_ADDRESS>:25231: connect: no route to host"
}

Run node with --rpc.skip_auth=true and config with SkipAuth: true (prints log in rpc/server here)

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{ // valid result, same as 1. }

Run node with --rpc.skip_auth=true and config with SkipAuth: false (server skips auth, prints log in rpc/server here)

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{ // valid result, same as 1. }

@Wondertan updated!
also, note that flock now leaves .lock file in directory - https://github.com/gofrs/flock/blob/master/flock.go#L54

also, note that flock now leaves .lock file in director

I am aware and that's ok. The locking functionality should be around the flock syscall, not the existence of the file.

Hey @mastergaurang94. Gentle ping. We would want to have this PR merged and wonder if you are going to finish it up

@Wondertan yes. Been open too long! Will aim to merge this week. Thank you 🙏🏽

@Wondertan anything else?

I think that's all. I already skimmed through the code and its LGTM. The next step is to get another review from a team member

Haven't seen the approval in a long time 🥹 Thank you for lending your time & energy to this review @Wondertan! Had fun

cc @jcstein for visibility

Cross-posting from slack:
This flag also appears:

in the cel-key utility: https://docs.celestia.org/developers/celestia-node-key#steps-for-generating-node-keys
for using a different directory than the standard dir: https://docs.celestia.org/nodes/celestia-node-troubleshooting#changing-the-location-of-your-node-store

these are the two that i found on a quick search
what impact does this have on cel-key utility or someone wishing to use a different directory than the default?

what impact does this have on cel-key utility or someone wishing to use a different directory than the default?

@jcstein
No impact on cel-key, and I think, as it stands, cel-key does not integrate automatic path resolution.
For non-default path users, they have to specify the path using the flag.

For non-default path users, they have to specify the path using the flag.

Got it, thanks @Wondertan. If I were running with a non-default node store, making an RPC request, I would still use the node.store flag, correct?

No impact on cel-key, and I think, as it stands, cel-key does not integrate automatic path resolution.

Correction after looking at the code. cel-key does have automatics but they aren't as smart as in this PR and this PR doesn't chang those

Got it, thanks @Wondertan. If I were running with a non-default node store, making an RPC request, I would still use the node.store flag, correct?

Yes

Thanks @vgonkivs & @walldiss! Also, PR will need to be labeled by someone who has access for merge

gm @mastergaurang94 - I'm working on the docs for this. can you help me understand which network(s) you were running a node(s) for on for 4-6?

Multiple networks running, will go to mainnet before mocha

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{
 "result": "RPC client error: sendRequest failed: http status 401 Unauthorized unmarshaling response: EOF"
}

I'm assuming you had a mocha node running, the request was sent to mainnet node, but looks like the call failed?

Multiple networks running, will go to mocha before arabica

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{
  "result": "header: given height is from the future: networkHeight: 802095, requestedHeight: 1318129"
}

I'm guessing you had mocha and arabica running?

Run node with rpc config Address: <IP_ADDRESS> and Port: 25231 -- accurately directs request

❯ celestia blob get 1318129 0x42690c204d39600fddd3 0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=
{
  "result": "RPC client error: sendRequest failed: Post \"http://<IP_ADDRESS>:25231\": dial tcp <IP_ADDRESS>:25231: connect: no route to host"
}

It looks like this request fails?

hey @jcstein, sure:

mainnet full, mocha light. re: fail, I confirmed the request went to the right node, not the validity of the response. 401 showed up in the mainnet node logs (that blob nor namespace exists there)
yes, both mocha full & arabica light. re: header in future, I had deleted my node store so new one hadn't caught up yet.

main thing, node types are unique. these were the cases I listed but any variation will work

similar to above, the request directs to http://<IP_ADDRESS>:25231. failure because I wasn't running anything at that address, if I was, it would have to that node

awesome, thanks for the details here @mastergaurang94
could you please take a look at the corresponding docs PR?


fix issue where void tag parsing returns false _ attr name

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

Looks good to me.


Execute promotion item not found in the dashboard

From time to time, a banner may popup in the home page of Rewards containing another promotion.
This is not found in the dashboard, so at the moment is skipped.
Usually it's a urlreward promotion, meaning that just clicking the element is sufficient to complete it.
An option to execute it could be inserting a new step between promotions and searches where the homepage is checked for the presence of this element, using Selenium APIs.

Example of item not found in the dashboard variable, but found in the actual GUI dashboard.
<mee-hero-item id="promo-item" horz="left" vert="center" context="accessory" theme="light" ng-if="$ctrl.promotion" ng-show="$ctrl.show" data-m="{"cN":"Taart + kaas","id":"nlstar_rewards_adhoc_globalherobanner_recipe_20230904_20","sN":1,"aN":"Rewards.Dashboard.Promotion","cT":"Promotion"}" aria-label="Taart + kaas, Geprobeerd cheesecake te maken? Probeer maar eens. Dit is heel eenvoudig." class="ng-scope"><section ng-class="$heroItemCtrl.typeClass" class="m-hero-item f-x-left f-y-center context-accessory theme-light">...</mee-hero-item>


Convert all in page messages to Bootstrap conventions

Use the Variables style sheet to modify the styles if needed.
If designs have not been created for the colours of the varying messages then use CMS existing colours and swap out the variables.

I've just had a look at this. It's going to be tricky to do it in a backwards compatible way, because having message class as well as alert alert-success overrides the Bootstrap colours and renders a grey alert.
I've added a casting helper to switch SilverStripe message types to Bootstrap ones and this works, but it would also affect frontend users and probably break frontends that rely on good, bad etc as message types.
We could add a new method e.g. $AlertType and run that in parallel instead?

All merged =)


remove unnecessary log that pontetially take up lots of heap when reindexing

when users run large hadoop re-indexing job, size of list of segment specs is quite large and constructing strings to print them causes ununecessary memory churn. also, printing them many times causes excess amount of logs.
HadoopDruidIndexerConfig.verify(..) was printing the whole config which included segment spec in many different places as a side effect and isn't desired, so removed. It becomes more troublesome due to hadoop calling IndexGeneratorJob.IndexGeneratorCombiner.setup(..) ->  HadoopDruidIndexerConfig.fromConfiguration(..) -> HadoopDruidIndexerConfig.verify(..)multiple times leading to it getting printed many many times and filling up hadoop logs.
config is still available in the peon log as each peon process prints task to be run in the logs anyway and same is available in hadoop job config too which is visible from hadoop job history.

Thanks for this fix. We ran into the same issue about the same time this fix got posted. Since that time our clusters kept crashing due to full HDFS disks.
Took me a long time to investigate and finally find this PR.
I'd suggest to not even use log level INFO for logging such large text fragments.
The changed code still contains the following lines:

// Note: log is splitted into two lines so that a new String is not generated to print it.
// segmentsStr could be quite large when re-indexing multiple months of data.
logger.info("Segment to read are...");
logger.info(segmentsStr);

To my understanding, this still emits the large volumes of log-text. How about using DEBUG or TRACE level logging for the segmentsStr ?
thanks
Sascha

@sascha-coenen that log is not printed multiple times and very useful when something goes wrong. typically people don't have debug logs and that log helps with investigation in clusters where it is difficult to re-run the large jobs for debugging. I kept it INFO based on its usefulness and don't see any major issues in getting it printed. Also, splitting it ensures that it wouldn't use extra memory other than what is already being used to store that string. if you face specific issues please let us know.


New Azure Batch DataPlane SDK

BTW, I love how you not only keep the documentation current, but also expand on it. That's really above and beyond, and provides a lot of value!


Fix errors for bots usage.

Automated message from Dropbox CLA bot
@abhijeetkaur, it looks like you've already signed the Dropbox CLA. Thanks!

Merged, thanks @abhijeetkaur!  I had to do a bit of work to update the tests when rebasing this.

@timabbott Ahh, I was about to rebase and update this! I was just busy with other PRs. I am sorry for having you do the work.
Thank you! :)


Overflow error on 3D datasets

Hello and thank you for your work!
I am trying to use Ripser to compute persistence diagrams of rather large 3D datasets, coming from the Open-SciVis-Datasets website.
However, Ripser throws a std::overflow_error when I try to compute persistence pairs up to the dimension 2. If I understand correctly the C++ code, this exception is related to the computation of binomial coefficients which depend on the size of the input distance matrix and the maximum dimension requested.
This issue might be related to https://github.com/Ripser/ripser/issues/25 and https://github.com/Ripser/ripser/issues/32 since they feature similar problems. I already opened an issue in the scikit-tda Python wrapper (I wanted to use Ripser via this Python API) but since this is more related to the C++ code, this issue seems better to belong here.
Below, you will find a Python script I used to trigger this overflow error. It takes a raw file from Open-Scivis-Datasets and iterates over the edges of the cubical complex to generate a sparse matrix in sparse triplet format that is fed to the Ripser executable. Although the diagram is computed as expected with the smallest datasets (nucleon, marschner_lobb, silicium, up to 120k vertices), the error occurs with fuel, neghip (more than 250k vertices) and every larger dataset.
It there a way to circumvent this computation of these binomial coefficients to handle large datasets (up to 1M vertices)?
Thanks in advance for your help,
Best regards,
Pierre Guillou
import argparse
import subprocess
import time

import numpy as np

def load_raw(input_raw):
    extension = input_raw.split(".")[-1]
    if extension != "raw":
        print("Need a .raw file")
        raise TypeError

    # detect extent and data type from file name
    extent, dtype = input_raw.split(".")[0].split("_")[-2:]
    extent = [int(dim) for dim in extent.split("x")]

    dtype_np = {
        "uint8": np.uint8,
        "int16": np.int16,
        "uint16": np.uint16,
        "float32": np.float32,
        "float64": np.float64,
    }
    dtype = dtype_np[dtype]

    with open(input_raw) as src:
        data = np.fromfile(src, dtype=dtype)
        return data.reshape(extent)

def compute_persistence(sparse_triplets, output_diagram, ripser_executable):
    proc = subprocess.Popen(
        [ripser_executable, "--format", "sparse", "--dim", "2"],
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
        universal_newlines=True,
    )
    out, _ = proc.communicate(sparse_triplets)

    with open(output_diagram, "w") as dst:
        dst.write(out)

def build_sparse_triplets(data):
    triplets = list()
    # loop over every edge of the cubical complex to get the sparse matrix triplets
    with np.nditer(data, flags=["multi_index"]) as it:
        for v in it:
            i = np.ravel_multi_index(it.multi_index, data.shape)
            for d in range(data.ndim):
                for s in [-1, 1]:
                    coords = list(it.multi_index)
                    s = coords[d] + s
                    if s < 0 or s >= data.shape[d]:
                        continue
                    coords[d] = s
                    j = np.ravel_multi_index(coords, data.shape)
                    if i < j:  # upper triangle distance matrix
                        triplets.append([i, j, max(data.flat[j], v)])

    return "\n".join([f"{i} {j} {v}" for i, j, v in triplets])

def main(input_raw, output_diagram, ripser_executable):
    data = load_raw(input_raw)
    start = time.time()
    sparse_triplets = build_sparse_triplets(data)
    print(f"Build sparse triplets in {time.time() - start:.2f}s")
    compute_persistence(sparse_triplets, output_diagram, ripser_executable)

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Wrapper around Ripser C++")
    parser.add_argument("input_raw", help="Input raw file from OpenSciVis datasets")
    parser.add_argument(
        "-o",
        "--output_diagram",
        help="Output diagram in Ripser format",
        default="out.ripser",
    )
    parser.add_argument(
        "-e",
        "--ripser_executable",
        help="Path to the Ripser executable",
        default="ripser/ripser",
    )
    args = parser.parse_args()

    main(args.input_raw, args.output_diagram, args.ripser_executable)

Can you also use Ripser on high dimensional data, like 10D?

Homology dimension k computations for a dataset of size N require that it be possible in principle to enumerate all (k+1)-simplices of the full simplex with N vertices.  In your case (k=2) this means enumerating all 3-simplices (which are 4-tuples of vertices), and this requires that the binomial coefficient (N choose 4) not exceed 2^63 - 1 (the maximum index for signed 64-bit integers).  With N=250000, N choose 4 is approximately 1.6e20, while 2^63 is about 9.2e18.
After modifying the backend to work with 128-bit integers, you should be able to run Ripser on your dataset. Both @ubauer and @MonkeyBreaker have tried this at some point.

@MassEast the dimensionality of the data is not an obstacle, Ripser simply looks at the matrix of pairwise distances between all pairs of points in the dataset.  This takes a little longer to compute in higher dimensions, but only marginally so.

After modifying the backend to work with 128-bit integers, you should be able to run Ripser on your dataset. Both @ubauer and @MonkeyBreaker have tried this at some point AFAIK. It can be done!
Thanks @ulupo for the information.
@ubauer @MonkeyBreaker would it be possible that you post the diff to get ripser running with 128-bit integers (it doesn't seem like a one-line edit). Thanks!

There is a branch at https://github.com/Ripser/ripser/tree/128bit implementing this. It should be mostly up to date, and it's actually pretty much a one line edit. Since 128 bit ints seem to be non standardized, this might not work out of the box for some compilers.

Thanks!


##[error]System.Management.Automation.ActionPreferenceStopException: The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: The request failed with HTTP status 401: Unauthorized.

2021-09-15T05:57:44.6546386Z ##[warning]Task 'SqlReportingServicesFolderDeployment' (1.0.4) is using deprecated task execution handler. The task should use the supported task-lib: https://aka.ms/tasklib
2021-09-15T05:57:44.6561709Z Preparing task execution handler.
2021-09-15T05:57:45.0958142Z Executing the powershell script: E:\Azure_DevOps\vsts-agent-win-x64-2.181.2_work_ten_tasks\SqlReportingServicesFolderDeployment_40abe74e-bbd1-43af-ac02-4f2364f01d78\1.0.4\SqlReportingServicesFolderDeployment.ps1
2021-09-15T05:57:45.4449318Z Preparing Deployment of RDL files...
2021-09-15T05:57:45.4463319Z
2021-09-15T05:57:45.4489297Z
2021-09-15T05:57:45.4747756Z [VERBOSE] >> Preparing script... (Verbose = true) -
2021-09-15T05:57:45.4761142Z
2021-09-15T05:57:45.4763704Z
2021-09-15T05:57:45.4767826Z [VERBOSE] >> Using following parameters: -
2021-09-15T05:57:45.4791549Z
2021-09-15T05:57:45.4818798Z
2021-09-15T05:57:45.4915702Z [VERBOSE] >> Remote root path: /Easy2/Reports/ -
2021-09-15T05:57:45.4916747Z
2021-09-15T05:57:45.4916954Z
2021-09-15T05:57:45.4917546Z [VERBOSE] >> Local root path: E:\Azure_DevOps\vsts-agent-win-x64-2.181.2_work_ten\r6\a_Construction-CI (2)\drop -
2021-09-15T05:57:45.4918026Z
2021-09-15T05:57:45.4918172Z
2021-09-15T05:57:45.4918645Z [VERBOSE] >> Attempt to update the datasource path: false -
2021-09-15T05:57:45.4919232Z
2021-09-15T05:57:45.4919371Z
2021-09-15T05:57:45.4919901Z [VERBOSE] >> Webservice: http://reportserver/pages/reportserver/reportservers.aspx -
2021-09-15T05:57:45.4920339Z
2021-09-15T05:57:45.4920481Z
2021-09-15T05:57:45.4920861Z [VERBOSE] >> WsUsername:  -
2021-09-15T05:57:45.4921122Z
2021-09-15T05:57:45.4921260Z
2021-09-15T05:57:45.4949508Z [VERBOSE] >> WsPassword: *****-
2021-09-15T05:57:45.4951417Z
2021-09-15T05:57:45.4951936Z
2021-09-15T05:57:45.5141942Z Uploading files
2021-09-15T05:57:45.5156016Z Creating WebService proxy using credentials
2021-09-15T05:57:45.5156615Z
2021-09-15T05:57:45.5160479Z
2021-09-15T05:57:45.7144537Z ##[error]System.Management.Automation.ActionPreferenceStopException: The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: The request failed with HTTP status 401: Unauthorized.
This is the detailed log and its failing with the error mentioned,could you please help us out how to resolve this

Hi, did you ever find a resolution to this? I am having the same problem

Any update?


Problem Number | Title | Language

766 | Toeplitz Matrix
What will change -
Type of Issue -
Please add/delete options that are not relevant.

[x] Adding New Code

Programming Language
Please add/delete options that are not relevant.

[x] Python

Self Check

Ask for issue assignment and wait to get assigned to it before making Pull Request.
Add your file in the proper folder
Clean Code and Documentation for better readability
Add Title and Description of the program in the file

:star2: Star it :fork_and_knife:Fork it :handshake: Contribute to it!
Discord server ✉ - https://discord.gg/CXksGdn
Happy Coding,

would you please assign it to me in python


remove the @latest would be the right thing to do after 1.0

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[ ] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
[ ] This message body should clearly illustrate what problems it solves.
[ ] Ideally, include a test that fails without this PR but passes with it.

Tests

[ ] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

[ ] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. All changesets should be patch until SvelteKit 1.0

did nothing of the above since it is content of the webpage...  ;-)

There have been instances of npm caching the package locally when we didn't have @latest. It's really dumb, but we do need it.


A question regarding index searching

Hello!
I have a question: why do you use  this line
List<Long> positions = new ArrayList<>(index);
in the readLines method? Assume that we are processing a very large file with millions of lines, would not this be problematic?
Cheers
Behrang

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/31919131-a-question-regarding-index-searching?utm_campaign=plugin&utm_content=tracker%2F2578241&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F2578241&utm_medium=issues&utm_source=github).

Hi, the line converts the SortedSet into a more operable List.
Although a new List instance is created, it is referencing the same elements as the SortedSet so it won't add as much space as you'd expect. And because the List is local to the method, it will be marked for garbage collection as soon as the method exits.

how about converting the resulting sortedset from indexing process to an
array of doubles in the constructor?
i am not sure about efficiency of casting sortedset to array-list.
On Wednesday, March 16, 2016, Jan Amoyo<EMAIL_ADDRESS>wrote:

Hi, the line converts the SortedSet into a more operable List.
Although a new List instance is created, it is referencing the same
elements as the SortedSet so it won't add as much space as you'd expect.
And because the List is local to the method, it will be marked for
garbage collection as soon as the method exits.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/jramoyo/indexed-file-reader/issues/1#issuecomment-197192829

that's a good point, we can convert the SortedSet into a List from the constructor, so that we don't have to do it every time.

Yeap, I think it is even better to have final arrays, i.e.,
final double[] index
I think that must do the job of unmodifiableList but I am not really sure; you are the expert!
From: Jan Amoyo<EMAIL_ADDRESS>Sent: Wednesday 16 March 2016 11:01
To: jramoyo/indexed-file-reader<EMAIL_ADDRESS>Cc: languagerecipes<EMAIL_ADDRESS>Subject: Re: [indexed-file-reader] A question regarding index searching (#1)
that's a good point, we can convert the SortedSet into a List from the constructor, so that we don't have to do it every time.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub https://github.com/jramoyo/indexed-file-reader/issues/1#issuecomment-197238131 https://github.com/notifications/beacon/AHuwE22d_meHuLULMNQd8273xfvL-W5Gks5pt9TrgaJpZM4HxTKX.gif


emit constant strings as utf8 in the compiler

Motivation
rb_fstring_new was treating the characters passed to it as ASCII.  So for the newly-enabled testcase in this PR, we were emitting the UTF-8 bytes for the constant string and reinterpreting those bytes as ASCII at runtime, which did not end well.
I am fairly confident that UTF-8 by default is Stripe's policy -- maybe even the default for Ruby? -- and I think this is the sort of change we should make anyway.
Test plan

See included automated tests.

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_K9F7K4JnaoELIJ
→ https://go/builds/bui_K9F73lEnjUQDOI


There needs to be a way to run USDM datasets/DDF rules in the -LR command

Currently we are not able to run USDM datasets with the -LR command.  The JSON is not parsed into dataset with the -lr flag.

@SFJohnson24 If the changes suggested in #840 are applied to get USDM data read in OK, I've found that it is possible to run local rules for USDM datasets - the -lr argument then works as expected.
The only real issue is that a CORE id is needed to uniquely identify the issues reported by the validation - but I found that assigning a temporary CORE id (e.g., "DRAFT-{Rule Id}") in the local rule YAML file works around that issue.


Specifying the LoadBalancer

Hello,
Sorry for the dumb question, but Is there a way to specify the LoadBalancer instead of creating a new one?
Or perhaps it is better if I describe my issue.
I use the Hetzner cloud with this automation tool to deploy the k8s cluster.
Whenever I deploy the stunner UDPRoute, it creates a new LB, which never gets to a healthy state. See the screenshot below.
Happy to learn how I can deal with this issue.

Hi! Which type of LB do you use? Klipper or Hetzner LB? Are there any other logs available? Klipper should use host ports and stunner fallbacks to nodePort if LB service cannot be created. So it shall work with Klipper.

Hi! Which type of LB do you use? Klipper or Hetzner LB? Are there any other logs available? Klipper should use host ports and stunner fallbacks to nodePort if LB service cannot be created. So it shall work with Klipper.

I use Hetzner LB.
I even tried with a different k8s distribution like this one. So far, no success. In this distribution, I cannot observe any LB created in the Hetzner dashboard.

My understanding is I need to customize LB settings for The Gateway resource.
If so, any idea how can I achieve this?
I believe you need to provide the following annotations for the Hetzner cloud.
    "load-balancer.hetzner.cloud/name": "stunner"
    "load-balancer.hetzner.cloud/hostname": "example.com"
    "load-balancer.hetzner.cloud/use-private-ip": "true"
    "load-balancer.hetzner.cloud/disable-private-ingress": "true"
    "load-balancer.hetzner.cloud/location": "nbg1"
    "load-balancer.hetzner.cloud/type": "lb11"
    "load-balancer.hetzner.cloud/uses-proxyprotocol": "true"

Any chance I can create the LB using those configurations?

UPDATE: I managed to have a "healthy" load balancer. See the image below.
The issue is that the Hetzner LB doesn't support UDP, so instead, I ended up having only TCP.
After this, I noticed that my stream was almost unwatchable. Huge latencies, I assume? Even though the stunner re-forwards the TCP port to UDP, I think it still uses TCP from Client to Stunner. Am I right?

Great to hear that it became healthy! I would need to check the stunner config to be able to figure out what happens now. Stunner supports TCP afaik, but it is useless for media. Could you share the content of the stunnerd configmap? You can contact me on Discord if its confidential. Btw, may I ask why don't you just switch LB?

Stunner supports TCP afaik

Can we confirm if this is the case? Because I do stream a media. Which won't be the best experience using TCP.

Btw, may I ask why don't you just switch LB?

To be honest, I would love to :)
Not sure how to do that or to what LB to switch.

You can contact me on Discord if its confidential.

I would love to catch-up on Discord. How can I find you?

I would love to catch-up on Discord. How can I find you?

Hey @Biacode! You can join here: https://discord.gg/DyPgEsbwzc

Even though the stunner re-forwards the TCP port to UDP, I think it still uses TCP from Client to Stunner. Am I right?
Yes, this is right.

I agree, this is definitely a "RTP over TCP sux" issue. Nevertheless, I find that most of the time TCP can still provide a decent media experience unless the Internet connection is extremely poor, can it be the case that there is a crowded Wifi or a 3G mobile uplink somewhere in the media path?

My understanding is I need to customize LB settings for The Gateway resource.
If so, any idea how can I achieve this?
I believe you need to provide the following annotations for the Hetzner cloud.

You can set the annotations you want STUNner to add to the LB services it creates in the GatewayConfig spec.loadBalancerServiceAnnotations field. We plan to make this Gateway specific in a later release.
If we are at it, can you share a few words about your use case? We'd be delighted to learn how/why you find STUNner useful in your WebRTC endeavors (if at all...:-)).

I agree, this is definitely a "RTP over TCP sux" issue. Nevertheless, I find that most of the time TCP can still provide a decent media experience unless the Internet connection is extremely poor, can it be the case that there is a crowded Wifi or a 3G mobile uplink somewhere in the media path?

No, nothing like crowded wifi or 3G mobile. My use case works perfectly fine with a UDP or let's say non K8S setup.
So I guess this is the issue with forwarding traffic through LB and using TCP combination.

You can set the annotations you want STUNner to add to the LB services it creates in the GatewayConfig spec.loadBalancerServiceAnnotations field. We plan to make this Gateway specific in a later release.

That helps. Thanks. I guess I will stop using stunner since TCP is not my solution.

If we are at it, can you share a few words about your use case? We'd be delighted to learn how/why you find STUNner useful in your WebRTC endeavors (if at all...:-)).

I need to stream/capture desktop content in real-time.

I use the Hetzner cloud with this automation tool to deploy the k8s cluster.
Whenever I deploy the stunner UDPRoute, it creates a new LB, which never gets to a healthy state. See the screenshot below.

Can this happen because of no UDP support in Hetzner LB? https://github.com/hetznercloud/hcloud-cloud-controller-manager/issues/252#issuecomment-1008793956
It looks like the automation tool supports klipper, which should work with UDP.  AFAIK, our neko example was originally done with klipper (and k3s).

Can this happen because of no UDP support in Hetzner LB? https://github.com/hetznercloud/hcloud-cloud-controller-manager/issues/252#issuecomment-1008793956

Thank you, @levaitamas. I appreciate your efforts on trying to help.
Previously I've tried to deploy the hetzner provider I use using Klipper without that much success.
I'll try to set it up again and post the configs.

Hi @Biacode, the newest STUNner point release supports HTTP health checking, does this solve your problem? Sorry, no testbed here, that's why I'm asking. Also, the NodePort fallback is always available if you don't like the LB your provider offers. At the moment you cannot prevent STUNner from creating an LB, but once we fix this issue you will be able to override this default.
If not, then do you see any way we could help you on the STUNner side? Happy to chat about your use case of you're interested, just drop by on the Discord.

Closing this issue due to inactivity. Feel free to reopen if the problem persists.


install script was compiled for MacOSX

Don't compile install script - should work on both?

Fixed - now compiles for either linux or mac


Constant timeout errors with the mqtt client

Hi,
I've got bluetti_mqtt running on a Raspberry Pi 1 B+ with a usb wifi and a usb bluetooth dongle.
In the past it used to work sort of fine. Once in a while I could see in home assistant (running on another server in a VM) that the values wouldn't come in each second (I start the program with that interval set) but more like a rapid fire of 2 values and then 1-2 seconds nothing and then another rapid fire and so on. For that I simply added a cron that runs every 6 hours and restarts the service and that seemed to work good enough.
Now I've started using the "Battery Range Start" slider to make my AC500 stop charging during the night instead of unplugging it every evening when the sun goes down. That has led to the rapid fire becoming a lot more common.
And once the rapid fire behaviour starts, I can adjust fields of the AC500 via the Home Assistant gui and the AC500 will adopt these changes, but I don't get a confirmation back, so if I adjust the "Battery Range Start" from 100% to 50% then the AC500 will set it to 50%, but when I reload the gui of Home Assistant, it shows 100% again. And things like the total battery percentage will not update until I've restarted the service.
I've updated Home Assistant to the newest version and that way the built in MQTT broker was updated as well. And then I've used "pip3 install <url to tar ball of version 0.15.0>" to install the latest release of bluetti_mqtt.
And now I'm constantly seeing these errors (log is since the last systemctl restart):
Jul 19 11:40:47 raspberrypi systemd[1]: Stopping Bluetti MQTT...
Jul 19 11:40:47 raspberrypi systemd[1]: bluetti-mqtt.service: Main process exited, code=killed, status=15/TERM
Jul 19 11:40:47 raspberrypi ac500.sh[356]: 2023-07-19 11:40:47 INFO     Shutting down...
Jul 19 11:40:47 raspberrypi ac500.sh[356]: 2023-07-19 11:40:47 INFO     Shutting down...
Jul 19 11:40:48 raspberrypi systemd[1]: bluetti-mqtt.service: Succeeded.
Jul 19 11:40:48 raspberrypi systemd[1]: Stopped Bluetti MQTT.
Jul 19 11:40:48 raspberrypi systemd[1]: Started Bluetti MQTT.
Jul 19 11:40:53 raspberrypi ac500.sh[574]: 2023-07-19 11:40:53 INFO     Connecting to MQTT broker...
Jul 19 11:40:53 raspberrypi ac500.sh[574]: 2023-07-19 11:40:53 INFO     Starting to poll clients...
Jul 19 11:40:53 raspberrypi ac500.sh[574]: 2023-07-19 11:40:53 INFO     Connecting to clients: ['24:4C:AB:2E:21:76']
Jul 19 11:40:54 raspberrypi ac500.sh[574]: 2023-07-19 11:40:54 INFO     Connected to MQTT broker
Jul 19 11:40:55 raspberrypi ac500.sh[574]: 2023-07-19 11:40:55 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.

Jul 19 12:25:09 raspberrypi ac500.sh[574]: 2023-07-19 12:25:09 INFO     Connected to MQTT broker
Jul 19 12:25:09 raspberrypi ac500.sh[574]: 2023-07-19 12:25:09 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.

I'm writing this 40 minutes later. No further log entries, but the rapid fire status updates in Home Assistant are there again and I can only get it back to working normally by restarting the service.

I restartet 2 hours ago after I made the initial post and since then this has happened (including the last line from the last log):
Jul 19 12:25:09 raspberrypi ac500.sh[574]: 2023-07-19 12:25:09 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.
Jul 19 13:08:01 raspberrypi systemd[1]: Stopping Bluetti MQTT...
Jul 19 13:08:01 raspberrypi ac500.sh[574]: 2023-07-19 13:08:01 INFO     Shutting down...
Jul 19 13:08:01 raspberrypi systemd[1]: bluetti-mqtt.service: Main process exited, code=killed, status=15/TERM
Jul 19 13:08:02 raspberrypi ac500.sh[574]: 2023-07-19 13:08:02 INFO     Shutting down...
Jul 19 13:08:02 raspberrypi systemd[1]: bluetti-mqtt.service: Succeeded.
Jul 19 13:08:02 raspberrypi systemd[1]: Stopped Bluetti MQTT.
Jul 19 13:08:02 raspberrypi systemd[1]: Started Bluetti MQTT.
Jul 19 13:08:08 raspberrypi ac500.sh[650]: 2023-07-19 13:08:08 INFO     Connecting to MQTT broker...
Jul 19 13:08:08 raspberrypi ac500.sh[650]: 2023-07-19 13:08:08 INFO     Starting to poll clients...
Jul 19 13:08:08 raspberrypi ac500.sh[650]: 2023-07-19 13:08:08 INFO     Connecting to clients: ['24:4C:AB:2E:21:76']
Jul 19 13:08:10 raspberrypi ac500.sh[650]: 2023-07-19 13:08:10 INFO     Connected to MQTT broker
Jul 19 13:08:10 raspberrypi ac500.sh[650]: 2023-07-19 13:08:10 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.
Jul 19 13:08:17 raspberrypi ac500.sh[650]: 2023-07-19 13:08:17 INFO     Connected to device: 24:4C:AB:2E:21:76
Jul 19 13:08:17 raspberrypi ac500.sh[650]: 2023-07-19 13:08:17 INFO     Device 24:4C:AB:2E:21:76 has name: AC5002242000779423
Jul 19 13:08:18 raspberrypi ac500.sh[650]: 2023-07-19 13:08:18 INFO     Sent discovery message of AC500-2242000779423 to Home Assistant
Jul 19 13:52:17 raspberrypi ac500.sh[650]: 2023-07-19 13:52:17 WARNING  Could not gracefully disconnect due to "Operation timed out". Forcing disconnection.

Jul 19 13:52:22 raspberrypi ac500.sh[650]: 2023-07-19 13:52:22 INFO     Connected to MQTT broker
Jul 19 13:52:22 raspberrypi ac500.sh[650]: 2023-07-19 13:52:22 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.

Jul 19 14:39:42 raspberrypi ac500.sh[650]: 2023-07-19 14:39:42 INFO     Connected to MQTT broker
Jul 19 14:39:42 raspberrypi ac500.sh[650]: 2023-07-19 14:39:42 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.

Jul 19 14:40:26 raspberrypi ac500.sh[650]: 2023-07-19 14:40:26 INFO     Connected to MQTT broker
Jul 19 14:40:26 raspberrypi ac500.sh[650]: 2023-07-19 14:40:26 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.

Jul 19 14:42:37 raspberrypi ac500.sh[650]: 2023-07-19 14:42:37 INFO     Connected to MQTT broker
Jul 19 14:42:37 raspberrypi ac500.sh[650]: 2023-07-19 14:42:37 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.

Jul 19 14:59:52 raspberrypi ac500.sh[650]: 2023-07-19 14:59:52 INFO     Connected to MQTT broker
Jul 19 14:59:52 raspberrypi ac500.sh[650]: 2023-07-19 14:59:52 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.
Jul 19 15:29:53 raspberrypi ac500.sh[650]: 2023-07-19 15:29:53 WARNING  Could not gracefully disconnect due to "Operation timed out". Forcing disconnection.

Jul 19 15:29:59 raspberrypi ac500.sh[650]: 2023-07-19 15:29:59 INFO     Connected to MQTT broker
Jul 19 15:29:59 raspberrypi ac500.sh[650]: 2023-07-19 15:29:59 WARNING  filtered_messages() is deprecated and will be removed in a future version. Use messages() together with Topic.matches() instead.

I've found the problem. The raspberry pi was generally accessible via SSH, so it never really occurred to me that the wifi connection might be the problem. But turns out that the access point closest to the raspberry pi got unplugged and the next access point was too far away and too many packages got dropped.
No more exceptions since the access point got plugged in again.


python312Packages.codecov: drop

Removes the codecov dependency from every package that depends on it, and the package itself. Per upstream, this package is deprecated and does not work. Further, we shouldn't be wasting energy and time testing coverage in CI.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 350862

x86_64-linux

:fast_forward: 4 packages marked as broken and skipped:

krr
krr.dist
opsdroid
opsdroid.dist

:white_check_mark: 74 packages built:

bepasty
bepasty.dist
prowler
prowler.dist
python311Packages.asteroid-filterbanks
python311Packages.asteroid-filterbanks.dist
python311Packages.heudiconv
python311Packages.heudiconv.dist
python311Packages.mkdocs-rss-plugin
python311Packages.mkdocs-rss-plugin.dist
python311Packages.nipype
python311Packages.nipype.dist
python311Packages.niworkflows
python311Packages.niworkflows.dist
python311Packages.pyannote-audio
python311Packages.pyannote-audio.dist
python311Packages.slack-bolt
python311Packages.slack-bolt.dist
python311Packages.slack-sdk
python311Packages.slack-sdk.dist
python311Packages.snakemake
python311Packages.snakemake-executor-plugin-cluster-generic
python311Packages.snakemake-executor-plugin-cluster-generic.dist
python311Packages.snakemake-interface-executor-plugins
python311Packages.snakemake-interface-executor-plugins.dist
python311Packages.snakemake-interface-storage-plugins
python311Packages.snakemake-interface-storage-plugins.dist
python311Packages.snakemake-storage-plugin-fs
python311Packages.snakemake-storage-plugin-fs.dist
python311Packages.snakemake-storage-plugin-s3
python311Packages.snakemake-storage-plugin-s3.dist
python311Packages.snakemake-storage-plugin-xrootd
python311Packages.snakemake-storage-plugin-xrootd.dist
python311Packages.snakemake.dist
python311Packages.throttler
python311Packages.throttler.dist
python311Packages.validator-collection
python311Packages.validator-collection.dist
python312Packages.asteroid-filterbanks
python312Packages.asteroid-filterbanks.dist
python312Packages.heudiconv
python312Packages.heudiconv.dist
python312Packages.mkdocs-rss-plugin
python312Packages.mkdocs-rss-plugin.dist
python312Packages.nipype
python312Packages.nipype.dist
python312Packages.niworkflows
python312Packages.niworkflows.dist
python312Packages.pyannote-audio
python312Packages.pyannote-audio.dist
python312Packages.slack-bolt
python312Packages.slack-bolt.dist
python312Packages.slack-sdk
python312Packages.slack-sdk.dist
snakemake (python312Packages.snakemake)
python312Packages.snakemake-executor-plugin-cluster-generic
python312Packages.snakemake-executor-plugin-cluster-generic.dist
python312Packages.snakemake-interface-executor-plugins
python312Packages.snakemake-interface-executor-plugins.dist
python312Packages.snakemake-interface-storage-plugins
python312Packages.snakemake-interface-storage-plugins.dist
python312Packages.snakemake-storage-plugin-fs
python312Packages.snakemake-storage-plugin-fs.dist
python312Packages.snakemake-storage-plugin-s3
python312Packages.snakemake-storage-plugin-s3.dist
python312Packages.snakemake-storage-plugin-xrootd
python312Packages.snakemake-storage-plugin-xrootd.dist
snakemake.dist (python312Packages.snakemake.dist)
python312Packages.throttler
python312Packages.throttler.dist
python312Packages.validator-collection
python312Packages.validator-collection.dist
whisper-ctranslate2
whisper-ctranslate2.dist

(Apparently we’re going with anywhere‐on‐Earth rather than UTC so this was totally fine!)

While I think it’s probably fine in this case, note that package removals for non‐security reasons are not really allowed right now due to having passed the breaking change freeze point of the release cycle.

so for future removals until master allows breaking again, I should target staging? If I need to remove a package of course

You’ll need to wait until approximately November 14; the reason the freeze ends earlier on staging{,-next} is just because those branches won’t get merged until after the branch‐off happens. You can apply the shiny new label to PRs that are waiting for that to happen (and preferably mark them as drafts to avoid accidental merges).


Update doc comments, typehinting, and code structure

As part of the software sustainability hackathon, efforts are ongoing to make CIL easier to use for users and developers.
This issue is focused on improving the python code documentation with Numpy-style docstrings, modern python typehinting, and following a standard format for class structure.
'Standards' followed:

Using Numpy's doc format for python

If the function has typehints, omit type hinting from the docstring:def __init__(self, input:AcquisitionData, image_geometry:Optional[ImageGeometry]=None, backend="tigre"):
	"""Abstract class for a tomographic reconstructor.

	Parameters
	----------
	input
		Input data for reconstruction
	image_geometry
		Image geometry of input data, by default None
	backend
		Engine backend used for reconstruction, by default "tigre"

	Raises
	------
	TypeError
		Raised if input data is not an AcquisitionData class
	"""

Private functions do not need to be documented (they are for developer internal use)

Class Structure (basically an inverse of c):

__init__

Init is first due to its documentation existing in the class's docstring, and properties may be large

Properties
Static functions
Functions
Private functions
Special functions (__str__())

Use " for strings (May change later on, but uniformity is good)
Use relative imports from cil.framework.recon import Reconstructor -> from .Reconstructor import Reconstructor to fix circular packaging issues #1681

There are still ongoing talks for enforcing a code style with a formatter, as
such this issue ignores code style, and focuses on class order and doc comments.
Note: this change doesn't include functional changes that have been discussed:

Functions should return out if passed in, not None #1657

Progress:
cil/

[ ] framework/
[ ] io/
[ ] optimisation/
[ ] plugins/
[ ] processors/
[ ] recon/ #1685
[ ] utilities/

Note: This does depend on #1686 to be merged, and then another pass to ensure typing compatabillity across CIL to finally pass static checkers


🐛Facing Knex DB migration issue while updating Backstage to 1.7 from 1.5

📜 Description
Upgrading backstage to 1.7 from 1.5 throws below error at the time of server start.
**The migration directory is corrupt, the following files are missing:<PHONE_NUMBER>0000_created_by.js**
👍 Expected behavior
It should not throw any error in upgradation to latest backstage version
👎 Actual Behavior with Screenshots
Throwing error while running yarn dev command locally after updated all dependencies in package.json file
👟 Reproduction steps

Upgrading all the package.json files (app/backend/root) as per the update helper guide
removing yarn.lock file
running yarn install to fetch updated npms
4 yarn dev to run it locally, server failed to restart and throw this error - **The migration directory is corrupt, the following files are missing:<PHONE_NUMBER>0000_created_by.js**

📃 Provide the context for the Bug.
Server is not getting restarted because of this issue, hence not able to access backstage locally.
🖥 Your Environment
S:   Darwin 21.6.0 - darwin/arm64
node: v16.17.0
yarn: 1.22.19
cli:  0.20.0 (installed)
backstage:  1.7.0
Dependencies:
@backstage/app-defaults                               1.0.7
@backstage/backend-common                             0.13.5, 0.15.2
@backstage/backend-plugin-api                         0.1.3
@backstage/backend-tasks                              0.3.6
@backstage/catalog-client                             0.9.0, 1.1.1
@backstage/catalog-model                              0.13.0, 1.1.2
@backstage/cli-common                                 0.1.10
@backstage/cli                                        0.20.0
@backstage/config-loader                              1.1.5
@backstage/config                                     0.1.15, 1.0.3
@backstage/core-app-api                               1.1.1
@backstage/core-components                            0.11.2, 0.9.5
@backstage/core-plugin-api                            0.8.0, 1.0.7
@backstage/dev-utils                                  1.0.7
@backstage/errors                                     0.2.2, 1.1.2
@backstage/integration-react                          0.1.25, 1.1.5
@backstage/integration                                0.8.0, 1.3.2
@backstage/plugin-api-docs                            0.8.10
@backstage/plugin-app-backend                         0.3.37
@backstage/plugin-auth-backend                        0.17.0
@backstage/plugin-auth-node                           0.1.6, 0.2.6
@backstage/plugin-catalog-backend-module-github       0.1.8
@backstage/plugin-catalog-backend                     0.24.0, 1.5.0
@backstage/plugin-catalog-common                      0.2.2, 1.0.7
@backstage/plugin-catalog-graph                       0.2.22
@backstage/plugin-catalog-import                      0.9.0
@backstage/plugin-catalog-node                        1.2.0
@backstage/plugin-catalog-react                       0.9.0, 1.2.0
@backstage/plugin-catalog                             1.6.0
@backstage/plugin-explore-react                       0.0.22
@backstage/plugin-explore                             0.3.41
@backstage/plugin-github-actions                      0.5.10
@backstage/plugin-github-deployments                  0.1.41
@backstage/plugin-home                                0.4.26
@backstage/plugin-org                                 0.5.10
@backstage/plugin-permission-common                   0.5.3, 0.6.4, 0.7.0
@backstage/plugin-permission-node                     0.5.5, 0.7.0
@backstage/plugin-permission-react                    0.3.4, 0.4.6
@backstage/plugin-proxy-backend                       0.2.31
@backstage/plugin-scaffolder-backend                  0.18.0, 1.7.0
@backstage/plugin-scaffolder-common                   0.3.0, 1.2.1
@backstage/plugin-scaffolder                          1.7.0
@backstage/plugin-search-backend-module-elasticsearch 1.0.3
@backstage/plugin-search-backend-module-pg            0.4.1
@backstage/plugin-search-backend-node                 1.0.3
@backstage/plugin-search-backend                      1.1.0
@backstage/plugin-search-common                       0.3.4, 0.3.5, 1.1.0
@backstage/plugin-search-react                        1.2.0
@backstage/plugin-search                              1.0.3
@backstage/plugin-sonarqube-backend                   0.1.2
@backstage/plugin-sonarqube                           0.4.2
@backstage/plugin-stack-overflow                      0.1.6
@backstage/plugin-tech-radar                          0.5.17
@backstage/plugin-techdocs-backend                    1.4.0
@backstage/plugin-techdocs-module-addons-contrib      1.0.5
@backstage/plugin-techdocs-node                       1.4.1
@backstage/plugin-techdocs-react                      1.0.5
@backstage/plugin-techdocs                            1.3.3
@backstage/plugin-user-settings                       0.5.0
@backstage/release-manifests                          0.0.6
@backstage/search-common                              0.3.4
@backstage/test-utils                                 1.2.1
@backstage/theme                                      0.2.16
@backstage/types                                      0.1.3, 1.0.0
@backstage/version-bridge                             0.1.2, 1.0.1
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
No response

That duplicate scaffolder backend version doesn't look good, and is likely the cause of the issue:
@backstage/plugin-scaffolder-backend 0.18.0, 1.7.0

Could you trace where that is coming from with yarn why @backstage/plugin-scaffolder-backend? You want to get to the point where you only have the latest version installed.

works like a charm, thanks for the quick response....


Add beachball sync & update naming for build pipeline

Included minor tweaks to README file to include setup instructions.


Add class qualified is broken for enums

Bug Description
When the flag is activated, the plugin adds class qualifier to static member access outside classes which is nice. However, when you are inside a switch-case for an enum. It is adding the enum name as qualifier which is not allowed
To Reproduce

Create an enum, say:
enum Day {
   MON, WED, FRI
}

Create a Main class, say:
public class Main {
     public static void main(String[] args) {
            Day day = Day.MON;
            switch (day) {
                case MON:
                     System.out.println("oh no");
                    break;
                case WED:
                     System.out.println("oh ok");
                    break;
                 case FRI:
                     System.out.println("oh nice");
                    break;
                  default: break;
            }
     }
}

Expected behavior
The above should be kept as is!
Actual behavior
The plugin does:
case Day.MON, case Day.WED, case Day.FRI
which weirdly is not allowed in Java
On compilation, this error is thrown:

an enum switch case label must be the unqualified name of an enumeration constant

Versions
IntelliJ IDEA 2019.1 EAP (Ultimate Edition)
Build #IU-191.5849.21, built on February 27, 2019
IntelliJ IDEA EAP User
Expiration date: March 29, 2019
JRE: 1.8.0_202-release-1483-b31 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 7 6.1

Didn't think of that case. PR welcomed.

a note:
this only fails for idea 2019.1 - guess they broke something when adding java 12 support for new switch stuff.

IntelliJ version 2019.1.1 was released which addressed the bug in the inspection. No further action in this plugin required.

IntelliJ version 2019.1.1 was released which addressed the bug in the inspection. No further action in this plugin required.

Thanks closing for now

This bug seems to not be completly fixed.
If you configure save actions to "Add class qualifier to static member access" everything is fine, but it you set to it "Add class qualifier to static member access outside of declaring class" it still occures.
Not sure if I should add this here or add a new issue...
Versions:
Save Actions v1.5.0+2018.3
IntelliJ IDEA 2019.1.3 (Ultimate Edition)
Build #IU-191.7479.19, built on May 28, 2019
Subscription is active until March 3, 2020
For educational use only.
JRE: 1.8.0_202-release-1483-b58 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 10 10.0

I am getting the same error as @RSLak1. Save Actions version 1.9.0+2019.3

@dubreuia: I can confirm the not fully fixded issue of @RSLak1 in 2019.3.1 with version 1.9.0 and 2019.2.4 with unknown version both linux and community edition

Ok, reopening. PRs welcomed.

The same behavior is observed on Idea 2022.2.3 (ultimate), build IU-222.4345.14, Save Actions version 2.3.0


Possibility to assign rooms to the devices

As mentioned in the title. can you add the option of entering the room per device?
thank you very much for your great work!

My Device: Xiaomi Purifier 3C

Sorry, I do not quite understand what you mean with entering the room per device, can you please try explain more?

So that I can already assign the corresponding final room to the respective device.
If I add a device today (via plugin) all devices end up in the hallway/standard room.
Now I would like to assign e.g. device 1 to the living room, device 2 to the kitchen and so on. - so that the devices do not all show up in the standard room. This would also clean up the designations:
Today the device 1 is called e.g.
Air purifier kitchen
But should only be called air purifier and stand in the room kitchen

This is something i cannot change! This is how HomeKit behaves. All new devices which you add will be added to the "Standard room". After that you can easily move them in the Home app to whatever room you want and they will stay there.

Okay Thanks a lot!!!


Tip Cannot Be Removed or Set to Zero

Description:
In the Enatega customer app, users are unable to remove or set the tip amount to zero. This creates an inconvenience for customers who may wish to opt-out of tipping for their orders.
Steps to Reproduce:
1-Open the Enatega customer app.
2-Add items to the cart and proceed to checkout.
3-Attempt to set the tip amount to zero or remove it entirely.
Expected Result:
Users should be able to set the tip amount to zero or remove it if they do not wish to tip.
Actual Result:
The app does not allow users to set the tip amount to zero or remove it, forcing them to leave a tip.
Additional Information:
https://github.com/user-attachments/assets/3343942b-ff6e-46b1-aaad-83310725afa5

Fixed


build user interface

there needs to be a user interface that we can interact with and begin developing laraedit.

User Interface is established with commit: 00f8bac3ba91be0baa02bd3bdc6fcbfcbba1d5f6


Microsoft Store

If Microsoft Store gets removed by the script is there a way to restore it?  I've looked all over the web and tried the various PowerShell commands, but I can't seem to find a way to get the Microsoft Store back.

Id' say revert it back, run the script as it prompts to revert changes.
OR
Edit the script, search the script for "store" and command is there

I've tried both options with no luck. The files for the Store from the C:\Program Files\WindowsApps are completely gone and the registry items are also.  So I'm not understanding how the script or commands from the script can restore the store.

@iPhoneMaster I'm surprised the revert option didn't help you as that's what I use in my testing.
Unfortunately you may need to do a system restore.

Enter this in powershell
Get-AppXPackage | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register "$($_.InstallLocation)\AppXManifest.xml"}
It will restore everything, including Microsoft Store.  The whitelist feature on this script doesnt work.  It removes EVERYTHING.  Which is not good.

@007craft Interesting that you say the whitelist doesn't work. I haven't had any issues with it. What version of Windows 10 are you running?

@Sycnex I'm running Windows 10 Pro build 17134.1

I will write up a quick fix for this. So sorry about that! I haven't experienced that.

@Sycnex No worries!  The link you provided allowed me to restore the missing folders from C:\program files\windowsapps and re-register the MS Store in Windows.  I would consider this issue resolved. Thanks again Sycnex! :-)


[Bug]: Bug when using SPA mode

What happened?
When I use SPA mode, the lightbox will appear temporarily and then switch pages, resulting in all pages being unscrollable.
How to reproduce the bug
This bug only happens when I use SPA mode.
Package Version
0.0.5
PHP Version
8.3.2
Laravel Version
10.48.3
Which operating systems does with happen with?
Windows
Notes
No response

will look into , similar case , not load the js when using SPA mode
https://github.com/filamentphp/filament/pull/11647


Genericize hospital admissions to delayed_incidence_observations

There is nothing special about the hospital admissions concept. In general, there are many cases where you would want to take some history of latent (or observed) infections/cases/hospitalizations and observe some proportion of them with some delay: E.g. latent infections -> observed cases -> observed hospital admissions -> observed icu admissions -> observed deaths.
More broadly, I would like remove the hospitalization model as a module, but leave in place the tools to create a hospitalization model, which would be demonstrated in a tutorial.

Closed in https://github.com/CDCgov/PyRenew/pull/398


minor code improvements

:( Conflicts? Must have edited while also was changing. Anyway my test are the same pre and post my changes. You can have a quick test also.
Happy from my side to merge.

Crap. Hold a sec; introcued a bug. Sorry about that.


Duplo-14922 TF provider changes for Azure maintenance schedule

Overview
Implementation resource for azure vm maintenance
Summary of changes
Implementation of resource duplocloud_azure_vm_maintenance_configuration which manages to schedule maintenance window vm's in azure
This PR does the following:

Implemented CURD operation for duplocloud_azure_vm_maintenance_configuration resource
Added example and updated documentation

Testing performed

[ ] Using unit tests
[ ] Manually, on my local system
[✓ ] Manually, on a remote test system

Describe any breaking changes

...

Task linked: DUPLO-14922 TF provider changes for Azure maintenance schedule


supporting multiple ports

This is a feature request. I just wanted to through out a use case that might not have been identified yet.
Some UE4 dedicated servers need access to both a game port and something called a "beacon" port. It's possible other dedicated servers require the ability to expose multiple ports as well.
It looks like the GameServer CRD is currently only able to express a single port. It might help to make that a list in the future.

Just been thinking about this some more, thinking of a configuration design that would look like the following (moving ports to an array, and giving it a name):
apiVersion: "stable.agones.dev/v1alpha1"
kind: GameServer
metadata:
  name: "gds-example"
spec:
  container: example-server
  ports: # (new) this is an array of ports
    - name: "main" # (optional) name is purely descriptive, so it's easy to know what each port is for
      portPolicy: "dynamic"  # still defaults to dynamic, but as an example
      containerPort: 7654
      protocol: UDP
    - name: "beacon"
      containerPort: 7652
      protocol: UDP # can now mix UDP/TCP, if you _really_ want to
# ....

This would still only apply to the single gameserver container. I was debating allowing this to be applied to any container in the pod, but I think it will get too confusing overall.
WDYT?

This would still only apply to the single gameserver container. I was debating allowing this to be applied to any container in the pod, but I think it will get too confusing overall.

+1 for just game server container.

WDYT?

I'm uncertain about how ports are handled in general. Do we intend to run the GameServer pods with hostNet=true?
I thought that was the case, but it looks like you might actually be detecting the external ip of the node the GameServer pod is running on and using that in combination with the dynamically generated hostPort value as the address of the GameServer.
That avoids requiring hostNet=true and ensures no port collisions occur, but I'm uncertain about the performance impact.
Also, some dedicated servers actually advertise the port they are bound to with another service for discovery. If clients connect via a different port using the nodes external IP, then this could cause connectivity issues.

I thought that was the case, but it looks like you might actually be detecting the external ip of the node the GameServer pod is running on and using that in combination with the dynamically generated hostPort value as the address of the GameServer.
That's exactly how it works.

Right now it goes through iptables - max 0.5ms extra latency - but gives us the ability to run sidecars due to actually having network separation between each gameserver and related pods, which gives us simplicity, the ability to still use k8s constructs and for now, segregation between each game server. Nothing to say we couldn't have an "escape hatch" to switch it to hostNetwork=true down the line, but simple first.
Down the line the iptables is being replaced by ipvs proxying, which should be much faster - but I should confirm if this is not just Services, but also hostPort as well (I assume so, but should double check).

Also, some dedicated servers actually advertise the port they are bound to with another service for discovery. If clients connect via a different port using the nodes external IP, then this could cause connectivity issues.

I would say that this is current out of scope, and if this is needed, build an adapter to watch Agones gameserver events through the k8s api, and broadcast events that way (and since we would have names on each port, it should be relatively straightforward to know what port does what).

I would say that this is current out of scope, and if this is needed, build an adapter to watch Agones gameserver events through the k8s api, and broadcast events that way (and since we would have names on each port, it should be relatively straightforward to know what port does what).

Yep, this is technically true, but realize that it may limit who is able to consider using agones.
This forces people to integrate their system with agones at a deeper level than they may have resources for. For example, the port situation would mean there's no practical migration path from the infrastructure I used to deploy UE4 dedicated servers over to agones.  Our UE4 dedicated servers registered their port with another service and that service expects the port to be reachable.
In the above scenario, agones might be attractive for it's sdk and scale in/out behavior, but the network connectivity issue would have forced us to pass on using the project :/
I think this is fairly simple to solve though. Just make some option that toggles hostNetworking:true on the Pod and tell people the whole port situation is up to them to handle if they use that option. That should satisfy both use cases.
:+1:  for the port list though. Looks good!

thinks
Another potentially interesting idea we could also float, is have a mechanism that pushes the currently allocated port down to the gameserver container as an env var / available through the SDK - in which case, it (in theory) could be relatively trivial to tweak the current setup to register with that value rather than the one the game server process starts on.  Would that work?

Another potentially interesting idea we could also float, is have a mechanism that pushes the currently allocated port down to the gameserver container as an env var / available through the SDK

interesting. that would satisfy the use case I outlined as well.
I'm just not sure how you'd accomplishing getting the host port to the SDK like that. I'm not sure k8s exposes that information into the pod. The sdk would have to maybe look up it's own pod in the apiserver or something. I dunno... Then you'd have to handle dealing with the game server trying to learn about the ports before the sdk learned about them.
maybe it's possible though, good thought!

or the SDK could talk to the sidecar, which also has the ability to read the data for the currently running GameServer , and could just look it up (and cache it, since it isn't going to change). That's actually probably the easier option.

I tend to believe that a configmap with the serialized CRD object is
better, since it'd be strictly read only for the current game pod.
Le jeu. 3 mai 2018 à 17:29, Mark Mandel<EMAIL_ADDRESS>a écrit :

or the SDK could talk to the sidecar, which also has the ability to
read the data for the currently running GameServer , and could just look
it up (and cache it, since it isn't going to change). That's actually
probably the easier option.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/agones/issues/151#issuecomment-386442425,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAX4vr1k6pkDntauu-ZUBdXaYylDh3k_ks5tu3avgaJpZM4S7a2O
.

--
Alexandre

Work on this starting here: https://github.com/markmandel/agones/tree/feature/multiple-ports

Implemented in #283


[FUZZ] Beaconfuzz_v2 slow-unit-2bf37695e00fc0acb9be197d6f7fe2b1bc558544

I've identified a fuzzer crash and am contributing to the security of Ethereum 2!
I've done and provided the following:

[x] Checked to see if any other [FUZZ] issue already refers to that crasher
[x] Attached the crashing input (either attached to the issue as a .zip or .gz, or as a link to a file sharing service)
[x] Noted the beacon-fuzz version or commit used.
[x] Provided crash output
[x] Noted the command or fuzzer used to generate the crash
[x] Name of the original crash file
[x] (Optional but optimal) Checked if the crash can be consistently replicated by re-running the input.

Info to Reproduce

Command run: make fuzz_attester_slashing-struct
Crasher file name: slow-unit-2bf37695e00fc0acb9be197d6f7fe2b1bc558544
slow-unit-2bf37695e00fc0acb9be197d6f7fe2b1bc558544.zip
Client exercised: Unsure, I ran it with the beaconfuzz_v2 differential fuzzer, I think it is Nim
Fuzzing engine used (if applicable): Structural fuzzing using libfuzzer

Crash output and stacktrace
Finished release [optimized] target(s) in 0.25s
     Running `fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing -artifact_prefix=/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/artifacts/struct_attester_slashing/ fuzz/artifacts/struct_attester_slashing/slow-unit-2bf37695e00fc0acb9be197d6f7fe2b1bc558544`
INFO: Seed: 553237533
INFO: Loaded 1 modules   (201440 inline 8-bit counters): 201440 [0x5555594799a1, 0x5555594aac81),
INFO: Loaded 1 PC tables (201440 PCs): 201440 [0x5555594aac88,0x5555597bda88),
fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing: Running 1 inputs 1 time(s) each.
Running: fuzz/artifacts/struct_attester_slashing/slow-unit-2bf37695e00fc0acb9be197d6f7fe2b1bc558544
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Could not get rough time result: no reply     prefix=roughtime
ERRO[0018] Failed to calculate roughtime offset          error="no valid responses" prefix=roughtime
INFO[0018] New calculated roughtime offset is 0 ns       prefix=roughtime
couldn't interpret ETH2FUZZ_BEACONSTATE: environment variable not found
thread '<unnamed>' panicked at 'No valid beaconstate in the seed folder', fuzz_targets/struct_attester_slashing.rs:122:13
stack backtrace:
   0: std::panicking::begin_panic
   1: core::ops::function::FnOnce::call_once
   2: std::sync::once::Once::call_once::{{closure}}
   3: std::sync::once::Once::call_inner
             at /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sync/once.rs:421
   4: rust_fuzzer_test_input
   5: __rust_try
   6: LLVMFuzzerTestOneInput
   7: _ZN6fuzzer6Fuzzer15ExecuteCallbackEPKhm
   8: _ZN6fuzzer10RunOneTestEPNS_6FuzzerEPKcm
   9: _ZN6fuzzer12FuzzerDriverEPiPPPcPFiPKhmE
  10: main
  11: __libc_start_main
  12: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Traceback (most recent call last, using override)
/root/fuzzing/beaconfuzz_v2/nim-beacon-chain/vendor/nimbus-build-system/vendor/Nim/lib/system/excpt.nim(614) signalHandler
SIGABRT: Abnormal termination.
==1212412== ERROR: libFuzzer: fuzz target exited
    #0 0x5555560d6901  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xb82901)
    #1 0x55555836c630  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2e18630)
    #2 0x55555835fb3b  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2e0bb3b)
    #3 0x7ffff7bf2a26  (/lib/x86_64-linux-gnu/libc.so.6+0x49a26)
    #4 0x7ffff7bf2bdf  (/lib/x86_64-linux-gnu/libc.so.6+0x49bdf)
    #5 0x55555635a2d7  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xe062d7)
    #6 0x7ffff7bef20f  (/lib/x86_64-linux-gnu/libc.so.6+0x4620f)
    #7 0x7ffff7bef18a  (/lib/x86_64-linux-gnu/libc.so.6+0x4618a)
    #8 0x7ffff7bce858  (/lib/x86_64-linux-gnu/libc.so.6+0x25858)
    #9 0x55555841d146  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2ec9146)
    #10 0x5555584061d5  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2eb21d5)
    #11 0x555558359176  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2e05176)
    #12 0x55555840d4e7  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2eb94e7)
    #13 0x555556120954  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xbcc954)
    #14 0x55555611d569  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xbc9569)
    #15 0x5555561207f4  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xbcc7f4)
    #16 0x55555612877c  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xbd477c)
    #17 0x55555611ff6b  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xbcbf6b)
    #18 0x555558407164  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2eb3164)
    #19 0x55555618605c  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xc3205c)
    #20 0x5555583591a0  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2e051a0)
    #21 0x555558358dff  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2e04dff)
    #22 0x55555835ff9c  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2e0bf9c)
    #23 0x55555833cbf9  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2de8bf9)
    #24 0x5555583469f2  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0x2df29f2)
    #25 0x5555560534b6  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xaff4b6)
    #26 0x7ffff7bd00b2  (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #27 0x55555605365d  (/root/fuzzing/beaconfuzz_v2/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_attester_slashing+0xaff65d)

SUMMARY: libFuzzer: fuzz target exited
────────────────────────────────────────────────────────────────────────────────

Error: Fuzz target exited with exit code: 77

Stack backtrace:
   0: cargo_fuzz::project::FuzzProject::exec_fuzz
   1: <cargo_fuzz::options::run::Run as cargo_fuzz::RunCommand>::run_command
   2: cargo_fuzz::main
   3: std::sys_common::backtrace::__rust_begin_short_backtrace
   4: std::rt::lang_start::{{closure}}
   5: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/ops/function.rs:259
      std::panicking::try::do_call
             at /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:381
      std::panicking::try
             at /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:345
      std::panic::catch_unwind
             at /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panic.rs:382
      std::rt::lang_start_internal
             at /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/rt.rs:51
   6: main
   7: __libc_start_main
   8: _start

Your Environment

Fuzzer ran:  Beaconfuzz_v2 using libfuzzer
Version/Commit used: b56bf452adc05ac5b0a73aa8844dac0465894ebc refs/heads/beaconfuzz_v2
Operating System and version: Ubuntu 20.04, Docker version 19.03.12, Linux 4.15.0 kernel, Running on a OpenVZ virtualized VPS

I just noticed that the stacktrace has a statement couldn't interpret ETH2FUZZ_BEACONSTATE: environment variable not found. I can't find any info on setting this variable in the README, please do let me know how one can set the variable. The issue might then just be a config related error and not a fuzzing crash.

Thanks @parithosh for the report.
I confirm, when ETH2FUZZ_BEACONSTATE is not set, the program will panic and crash ;)


dotnet build fails on Linux citing missing build target Microsoft.NET.Sdk.WindowsDesktop.targets

Hi there, great project and awesome to see you're building on the great work of the C# Onnx Runtime API working to make generative AI more accessible to C# developers, so we don't have to endlessly wade through lots of python code written hastily by academic researchers!
I was keen to see if I can play around with this project and figure out how to add support for other ONNX based models as it seems like the format is really gaining some traction these days. However, I'm running Ubuntu 23.04 and after cloning the project and attempting to run dotnet build I get the following compilation error:
dotnet build

Welcome to .NET 7.0!
---------------------
SDK Version: 7.0.113

----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only).
Learn about HTTPS: https://aka.ms/dotnet-https
----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
--------------------------------------------------------------------------------------
MSBuild version 17.4.8+6918b863a for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
/usr/lib/dotnet/sdk/7.0.113/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(1226,3): error MSB4019: The imported project "/usr/lib/dotnet/sdk/7.0.113/Sdks/Microsoft.NET.Sdk.WindowsDesktop/targets/Microsoft.NET.Sdk.WindowsDesktop.targets" was not found. Confirm that the expression in the Import declaration ";/usr/lib/dotnet/sdk/7.0.113/Sdks/Microsoft.NET.Sdk/targets/../../Microsoft.NET.Sdk.WindowsDesktop/targets/Microsoft.NET.Sdk.WindowsDesktop.targets" is correct, and that the file exists on disk. [/home/yolo/source/OnnxStack/OnnxStack.UI/OnnxStack.UI.csproj]
  OnnxStack.Core -> /home/yolo/source/OnnxStack/OnnxStack.Core/bin/Debug/net7.0/OnnxStack.Core.dll
  OnnxStack.StableDiffusion -> /home/yolo/source/OnnxStack/OnnxStack.StableDiffusion/bin/Debug/net7.0/OnnxStack.StableDiffusion.dll
  OnnxStack.Console -> /home/yolo/source/OnnxStack/OnnxStack.Console/bin/Debug/net7.0/OnnxStack.Console.dll
  OnnxStack.WebUI -> /home/yolo/source/OnnxStack/OnnxStack.WebUI/bin/Debug/net7.0/OnnxStack.WebUI.dll

Build FAILED.

/usr/lib/dotnet/sdk/7.0.113/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(1226,3): error MSB4019: The imported project "/usr/lib/dotnet/sdk/7.0.113/Sdks/Microsoft.NET.Sdk.WindowsDesktop/targets/Microsoft.NET.Sdk.WindowsDesktop.targets" was not found. Confirm that the expression in the Import declaration ";/usr/lib/dotnet/sdk/7.0.113/Sdks/Microsoft.NET.Sdk/targets/../../Microsoft.NET.Sdk.WindowsDesktop/targets/Microsoft.NET.Sdk.WindowsDesktop.targets" is correct, and that the file exists on disk. [/home/yolo/source/OnnxStack/OnnxStack.UI/OnnxStack.UI.csproj]
    0 Warning(s)
    1 Error(s)

 Time Elapsed 00:00:02.90

I guess it makes sense the WindowsDesktop target is not available on a linux distribution, but it's a shame that's preventing me from compiling it.

yeah, this was due to the addition of the WPF project last week, its not cross-platform so will fail build on mac an linux
I was not planning on making any UI's at all, but ended up adding a basic Web and Desktop UI so people could test
The console and web apps should work fine on linux and mac
Thanks you for the PR, I have merged that in exposing just the core projects


Epsilon symbol

How to write 'ε'?

Hi. Thank you for you help.
I've found mistake in template file.
It will be fixed in 1.3.3 release.

Release v.1.3.3 has been published.