Chrysler False Positive Forward Collision Warnings

Describe the bug
Will be driving with Comma 3 enabled, and seemingly randomly, the Forward Collision warning will go off. It startles everyone in the car, and does not appear to have reason. Has been going on for a while on the master-ci branch.
Which car does this affect?
2022 Chrysler Pacifica Hybrid
Provide a route where the issue occurs
4395faadf0a373f2|2022-12-26--22-49-31--48
openpilot version
0.9.1 nightly
Additional info
4395faadf0a373f2|2022-12-26--13-10-38--73
4395faadf0a373f2|2022-12-26--13-10-38--81
4395faadf0a373f2|2022-12-26--13-10-38--86
4395faadf0a373f2|2022-12-26--13-10-38--93
Here are a couple instances on a single drive. If you need more examples, I can grab more.

I have been experiencing these False FCWs in my vehicle too, in my 2018 GMC Acadia.
My wife said that she had at least six on her way home from work yesterday.
df51b2195de66a9b|2022-12-26--16-32-25--0

I have been experiencing these False FCWs in my vehicle too, in my 2018 GMC Acadia.
My wife said that she had at least six on her way home from work yesterday.
df51b2195de66a9b|2022-12-26--16-32-25--0

@Verylukyguy that route doesn't have any FCWs

These are factory FCWs being triggered in the vehicle, not on the device; we get them almost every drive, along with seemingly random dash errors.
I am loading a current fork of Comma Master now to see if they are still occurring.

@HaraldSchafer I disabled radar parsing for now (#27190) since it seems the planner FCW has some thresholds tuned for Honda/Toyota radars. Leaving this open if you want to still take a look anyway.

The radar looks super noisy. Are we sure we are parsing it correctly? It frequently seems to detect slow but not quite stationary objects. The combination of not having a radar point on the true lead and having faulty non-stationary points can cause this.

Added a comment about why this is disabled https://github.com/commaai/openpilot/commit/dad948983a7ff26ffc073c536bd1e99b5d826cec


Can not run project without artisan

I can not run this project with url: localhost:88/laravel-adminpanel
Please help !
Regards,

How can I run this project by url: localhost/projectname  don't need execute run project with "php artisan serve" ?
Could you please me ?

Hello, you need to create a virtual host to run with a url. There are plenty of docs on Google on how to do that for a laravel project. Do that, and add that new url to your .env file and you are good to go

@baduypaltalk  You need to place the project files into server directory , lets say /var/www/html/laravel-adminpanel
This will provide you a external link as http://localhost/laravel-adminpanel/public this will work for you in most case. and if you are using apache you need to rewrite the laravel-adminpanel directory.
for how to rewrite you can find my blog post over

@baduypaltalk You need to place the project files into server directory , lets say /var/www/html/laravel-adminpanel
This will provide you a external link as http://localhost/laravel-adminpanel/public this will work for you in most case. and if you are using apache you need to rewrite the laravel-adminpanel directory.
for how to rewrite you can find my blog post over

Its done, thanks bro

Hello, you need to create a virtual host to run with a url. There are plenty of docs on Google on how to do that for a laravel project. Do that, and add that new url to your .env file and you are good to go

Thanks you, i've emplemented on windows os & ubuntu successfully.

@baduypaltalk  please can you help me to create virtual host

@baduypaltalk please can you help me to create virtual host

Oh Guy,,
Hope here that you need :
https://austin.passy.co/2012/setting-up-virtual-hosts-wordpress-multisite-with-xampp-on-windows-7/

hi think i use this https://github.com/worldtutorial/vhost-for-laravel/blob/master/v-host.txt and work fine


nice-grpc-web still depends on @improbable-eng/grpc-web

https://github.com/deeplay-io/nice-grpc/blob/master/packages/nice-grpc-web/package.json#L44
It seems obsolete, couldn't find any imports from @improbable-eng in codebase.

Didn't realise that. Thanks!


Request BYE returned 500 Invalid CSeq

Hi,
This happened when call hangup process, the SIP response sent by the server got 500 Invalid CSeq.

I think this is because when the digest auth request is sent, the last CSeq header is not saved, so the next request will still use the old one.

i use<EMAIL_ADDRESS>and<EMAIL_ADDRESS>

Thanks for reporting.
Can you tell me are you using sigpox.Phone or what is your flow

Yes, I'm using sipgox.Phone for dialing and this happened when calling phone.Hangup().
I have tried to do it this way and it does work, but I have no idea if it is the right way.

hi @muchlisegeroo there is now refactored code on sipgo to handle more correctly Dialog request.
one of is CSEQ. I probably have not ported this.
So there is RFC for dialog generaly. You have dialog.TransactionRequest This should be used for now when dealing with request within dialog.
This will correctly follow CSEQ and other.
Thanks for reporting, I will have to move auth to this. pls wait for fix

Ok, great. Thanks.

@muchlisegeroo can you pls checkout latest commits on sipgox and see is issue resolved for you.
Now auth should travel as well through dialog.TransactionRequest
If not resolved. pls reopen


Use idevice_id for devicelab iOS device listing

This patch migrates iOS device listing from using Xcode instruments to
using the libimobiledevice tools idevice_id and ideviceinfo.
ideviceinfo was previously incompatible with iOS 11 physical devices;
this has now been fixed.
In 58fe8237d2ec012ea51a673bc228f6545af8bb3c flutter_tools migrated from
libimobiledevice-based device listing on iOS to using Xcode instruments
to work around the lack of support for iOS 11. Using instruments entails
several downsides, including a significantly higher performance hit, and
leaking hung DTServiceHub processes in certain cases when a simulator is
running, necessitating workarounds in which we watched for, and cleaned
up leaked DTServiceHub processes. This patch returns reverts the move to
instruments now that it's no longer necessary.
This reverts commit 58fe8237d2ec012ea51a673bc228f6545af8bb3c.

Related flutter_tools revert: flutter/flutter#11883


update schema and test data from upstream spec

Mainly ran the /tests/_data/schemaTestData/fetch.sh script to sync the upstream spec schema files

is blocked by a pending change to the subject: CycloneDX/specification#514

Oh, I see. We can let it sit here as a reminder then.

it would also be required to

[ ] bump the schema files,
[ ] bump some SBOM generators, maybe?

I will investigate and add needed changes.

superseded by https://github.com/CycloneDX/cyclonedx-python-lib/pull/742


set mask_zero=True in embedding layer

see https://groups.google.com/forum/#!topic/keras-users/5U_E04hE7bE

Not very useful.


boost error

I got the following error on windows 10, with the version downloaded from proteowizard currently on the website. Let me know if you would like more information.
$ msconvert 200705_PhosB_2pHDMSE_GF_0sec_01_vial01.raw
format: mzML
m/z: Compression-None, 64-bit
intensity: Compression-None, 32-bit
rt: Compression-None, 64-bit
ByteOrder_LittleEndian
indexed="true"
outputPath: .
extension: .mzML
contactFilename:
filters:
filenames:
200705_PhosB_2pHDMSE_GF_0sec_01_vial01.raw
boost::filesystem::create_directories: Invalid argument
Please report this error to<EMAIL_ADDRESS>Attach the command output and this version information in your report:
ProteoWizard release: 3.0.10827 (2017-5-11)
ProteoWizard MSData: 3.0.10827 (2017-5-11)
ProteoWizard Analysis: 3.0.10827 (2017-5-11)
Build date: Nov  1 2017 15:08:32

You downloaded the most recent version, but you're not executing it with your command above. Try the absolute path to msconvert.exe, like "c:\Program Files\Proteowizard\ProteoWizard 1.2.3\msconvert.exe"

ahh yes, I see there is a file path issue - thank you!


Use chpwd instead of precmd hook, and add-zsh-hook

This triggers z at every working directory change, instead of at every prompt, in zsh. Use add-zsh-hook to set up the _z_chpwd function.
Also, update the async job call inside _z_chpwd. The parenthesis, which do the call in a subshell, are not necessary, and &! makes sure the job is immediately disowned (and there's no output of the process ID or it's termination status). See http://zsh.sourceforge.net/Doc/Release/Jobs-_0026-Signals.html
This code is compatible with zsh versions >= 5.0.
Closes #210

add-zsh-hook guarantees that the same hook function is not added twice, so removed the [ -n "${precmd_functions[(r)_z_precmd]}" ] test.

I see that using cmd &! instead of (cmd &) breaks sourcing z.sh in other shells, like in sh. Keeping the original (cmd &) construct is probably a better idea here.

I would keep using the previous construct to add the function to chpwd, since it supports earlier zsh versions and you don't need to autoload add-zsh-hook. We could use a better, clearer syntax though:
chpwd_functions+=(_z_chpwd)

I have to say that I haven't checked it in other shells though.

Indeed, dash chokes on that syntax because the parentheses makes it look like a function. We could use the old syntax that we know works everywhere, or we could do
typeset -a chpwd_functions
chpwd_functions+=_z_chpwd

The typeset is to force declare chpwd_functions as an array, otherwise we could get a string if it wasn't defined previously. This one's confirmed to pass the syntax check in bash and dash.

Fair enough. I'm closing this then.


Whether or not you can specify a mounting location

Whether or not you can specify a mounting location？For example, using the specified installation location as an additional command to the installation script

All packages are installed to the common opkg location. You may specify a different OpenWrt overlay location and potentially it will work. But it is not tested:
https://openwrt.org/docs/guide-user/additional-software/extroot_configuration


undefined is not an object (evaluating'_this.internals.offset[dir]')

Which OS ?
Android
Version
Which versions are you using:

react-native-swiper v1.5.10?
react-native v0.46.4

Describe
I'm developing a shopping website. On the goods description page,I used swiper. Goods description page can jump to the shopping cart page (navigator.push) and shopping cart page can jump to the goods description page too (infinite stack). The problem is that when there are more than one swiper in the stack, the code prompts "undefined is not an object" in 'updateIndex' function.

Sup. I had a very similar problem, but my swiper just swiped correctly, but I had an error same as yours. If everything works ok, but the problem is just an error you can use not so gentle way to null it. In the "updateIndex" method wrap the content in if(this.internals.offset){##content}. This validator helped me get rid off this error. In my case it worked on 2 android devices. On ios I have a problem with scrolling to index, so right now I'm not sure about it.

Can you try with the latest version, 1.5.13 ?
If it does it again, can you fork this project and reproduce the bug in the examples ?

I meet the same question

I meet the same question :( please

+1

reverting back to version 1.5.8, it's working fine.

How can i know the vertion of my swiper?
how to reverting back to version 1.5.8 ?

+1
react-native-swiper 1.5.8
RN 48.0

In my case,
RN 0.49
react-native-swiper 1.5.13
In Android, problem caused by 'onLayout' event fired after this.swiper.scrollBy(...)..
But react-native-swiper initialized in onLayout event phase..
So.. I solved this issue.. by..
0. Add isSwiperLoaded state.. like this.state={..., isSwiperLoaded: false};

attach onLayout listener like <Swiper onLayout={() => {this.setState({...this.state, isSwiperLoaded: true});} ... />
call this.swiper.scrollBy(...) in componentDidUpdate like...
componentDidUpdate(prevProps, prevState) {
..
if (!this.state.isSwiperLoading) {
this.swiper.scrollBy(...);

That's all.
I hope someone helped.

can anybody help ?

@yoyo0427 add this code, not crash, but can not swip auto. see thishttps://github.com/leecade/react-native-swiper/issues/521 there have another way to solve in comments

I solve this problem.
"react-native-swiper": "https://github.com/Tennen/react-native-swiper.git",
loop={this.state.dataList.length>1}

+1

+1 for slow devices - navigating out of view - need to protect against unmounting or it will crash the app

Same here. My app is in production, crashes for a lot of users because of this issue :(

Same question.

+1

+1

I have the same problem on 1.5.13
com.facebook.react.common.JavascriptException: undefined is not an object (evaluating 'n.internals.offset[e]')
can anyone help me?

我也有同样的问题，已经哭晕了

Same

@zhaoqigang Your repo working with me. Thank you.

I solve this problem.
node_modules/react-native-swiper/src/index.js 398 line add
 if(offset === undefined || this.internals.offset === undefined){ return; }

Please implement this in the repo

@leecade @arribbar can any of you guys do something about this issue please...?

I have same this issue，it make me crazzy。

+1 for me would be nice to have a fix please

I have same issue with react native 0.58.6 !!!


mistune ==> 2.04

Describe the bug
A clear and concise description of what the bug is.
Steps to reproduce the bug
Steps used to install present:

Add step here (you can add more steps too)

Steps to reproduce the behavior:

Add step here (you can add more steps too)

Expected behavior
A clear and concise description of what you expected to happen.
Slides
If applicable, add slides to help explain your problem.
Add markdown for the slide that caused the problem

Screenshots
If applicable, add screenshots to help explain your problem.
Environment

OS: [e.g. MacOS]
Terminal emulator: [e.g. guake]
Python version: [e.g. 3.7]
present version: [e.g. 0.5.1]

Additional context
Add any other context about the problem here.

The latest version has a new rendering and an exception occurs

No description provided.


Add Serde support to HyperLogLog

I have a use-case where I need to send HyperLogLog over the network. In order to support that, I'd like to extend it with Serde support.
The PR:

adds a new default serde feature
adds a hyperloglog::serde module that depends on the feature and defines a new struct HyperLogLogExport, which just stores the b and the registers and implements Serde's Serialize and Deserialize traits. Additionally it adds a from_exported_hyperloglog method on HyperLogLog that is able to reconstruct self from export
I don't know if there is a better way to tie the exporting and the reconstructing hashers, so right now it's up to the user to ensure the hasher is the same.
Tests and docs
Fixes various errors and clippy lints that popped up

Please let me know if this approach is acceptable and I can implement it also for some other structures, at a minimum, I also need it for the BloomFilter.

clippy is failing

Fixed. I've also realized that I'd forgotten to include a way to reconstruct the instance from the extracted registers, so I've added that

bors r+


URI problem with brackets

Hello everyone!
I'm learning Elixir and I'm using this library to access Kitsu API to retrieve some anime information.
To search for an anime, I need to use a route like this: https://kitsu.io/api/edge/anime/?filter[text]=naruto
And here is a example of my code:
defmodule Cardinal.Kitsu.Client do
  use Tesla

  plug Tesla.Middleware.BaseUrl, "https://kitsu.io/api/edge"
  plug Tesla.Middleware.DecodeJson, decode_content_types: ["application/vnd.api+json"]
  plug Tesla.Middleware.Headers, [{"Accept", "application/json, application/vnd.api+json"}]
  plug Tesla.Middleware.JSON

  def get_anime() do
    get("/anime/?filter[text]=naruto")
    |> handle_get()
  end

  defp handle_get({:ok, %Tesla.Env{status: 200, body: body}}), do: {:ok, body}
  defp handle_get({:ok, %Tesla.Env{status: 404}}), do: {:error, "Anime not found"}
  defp handle_get({:error, _reason} = error), do: error
end

When I try to execute it, it returns an error {:error, :invalid_uri}, but if I change the URI to /anime/1/genres or /anime/1/relationships/genres, it works perfectly.
So it seems that URI isn't accepting brackets. Is this a problem in the library or I am doing something wrong?
Thanks!

For anyone with this problem, you can represent the brackets in a percent-encoded format.
So, for example, instead of
"/anime/?filter[text]=naruto"
you can use
/anime?filter%5Btext%5D=naruto
And it'll work perfectly. I'll close this issue for now


Identity 2866

Fixed in identity-framework with https://github.com/wso2/carbon-identity-framework/pull/593


Firefox not working

Hi, there are some Firefox bugs with the sprite.
You can open your example on https://jannicz.github.io/ng-svg-icon-sprite/ in Firefox 63.0.3 (Linux) and see that icons are different than on Chrome.
I think that you manually have to add viewBox for each icon on  component, and also preserveAspectRatio to 'xMidYMid meet'.
At least that was my case on my project with same issues, but those things aren't easy to implement on each icon.
Hope to get some new info.

Thank you for taking the time. You are right, the examples did not work correctly in Firefox. I improved the examples so they now work with all browsers constantly, see https://jannicz.github.io/ng-svg-icon-sprite/#viewBox
For your problem i recommend checking following things:

does your SVG source include a viewBox constantly across all exported files?
would adding/removing the viewBox help to get better results?

I think that you have to establish one export and include pattern that will work with all your icons, and then scale them via CSS. Doing so you will not have to write width/viewBox attributes for each icon separately.
If you don't have any objection, i will close this issue tomorrow. Thank you!

Thank you for your reply.
Your fix did the job, all works great now. :)
I will have to test it in IE and other browsers to be sure.
Yes, you can close this issue.
Good job.

Thank you for your reply.
Your fix did the job for your examples, but I am having different issues.
I think it is related to the fact that I am receiving svg's with width and height, and If I strip those attributes with your command than it strips width and height from not only svg but from all inside elements, like rect and so on.
I think I will have manually to remove those attributes.
Thank you.


Lore revision and standardization

Guidelines used are outlined under this document
inb4 everything gets rejected :sadge:

I'll look over it closer to reset. Don't want to redo all villager drops just yet.
If it is mainly restructuring and minor color tweaks

redoing as i feel current is way too rushed :siggy:


[GTKA] Add a validation rule

Atlasmap supports the ability to validate mapping files ahead of runtime execution. Additionally, these validation rules can inform the UI at design time to assist the user in correcting their mapping definition.

Identify a useful validation scenario
Identify where it should live-- core, java, xml or json modules
Review design-time and runtime use cases
Implement validation
Implement unit test

I'm interested in looking into this one as a starter

Hi @mattrpav ,
It seems that the mapping validation is already implemented as an API here
https://github.com/atlasmap/atlasmap/blob/master/atlas-service-parent/atlas-service/src/main/java/io/atlasmap/service/AtlasService.java#L268-L274
What else is supposed to be done with this ticket? Is this just an UI task to invoke this API at design time and show the results? If so I may took a wrong ticket as I'm at runtime side - camel related task would be better.

Hi @igarashitm,
The Validation rules are shared at design time and runtime. For example, if there is something that would cause a mapping to not be executable, it would create a Validation.ERROR and the runtime engine would not continue processing.
If you were to have a mapping defined with a source as "atlas:json", but it referenced a XmlField, that would cause a Validation.ERROR. In the UI, it would provide an error message and at runtime it would short circuit so we don't have to handle all permutations of unpredictable errors.
A WARN would be something like a mapping instruction that is a Double -> Short. There is a probability that it works, but also a chance it causes an error at runtime. We inform the user and track it at runtime.
Make sense?

A next step would be to define a new Validation that hasn't been implemented, add it and then create a unit test to validate.
Updating the Camel Component is also a good task =)

OK so the validation feature has already been implemented and the goal of this ticket is to add more rules. Is that correct? Then I'd look into the existing rules and see if I can have another idea to add.

Yep.  Heads up.. there is a Validation refactor task that I need to get to that aligns with a AtlasMap UI integration task. I was hoping to have it wrapped today, but its looking more like M-Tu next week.
It might make sense to pick this up mid-next week. If you are looking for an immediate task.. a Field Action would be a great thing to look into.

Tomo is working on a full-fledged Validation refactor


Chip Click listener

Awesome library! I love it.
Is it somehow possible to add a onChipClickListener?
if not, could you please add it in to your next release
Regards

@Sese-Schneider according to the README you can set a onClick listener to your chip:
ChipView chip = (ChipView) findViewById(R.id.chip_view);
chip.setOnChipClicked(new View.OnClickListener() {
    @Override
    public void onClick(View view) {
        // handle click
    }
});


How to load models to predict data

This is a potential project. How can I load the model and predict the new data? Can you give me a code example? Thank you

Hi, take supervised learning as an example. You can download all the datasets from Autoformer. For supervised learning, create a seperate folder ./dataset and put all the csv files in the directory. Then create new script by mimicking PatchTST/PatchTST_supervised/scripts/PatchTST/electricity.sh, and then you would be able to run it.

thank you!!!


Case-insensitive search

As the title.

made the searchbar case sensitive by making both lower case, the joined values and controlOption.search value
on lines 56, 57 and 58 of DataPanel.tsx


METRON-915 add node and npm to platform_info.sh

Contributor Comments
Now that we depend on node and npm, put them into the platform_info.sh for troubleshooting build failures.
We may want to follow on with the pr template?
to verify.. run the script on platforms you use
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[x ] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[x ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[ x] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[ NA] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[ x] Have you included steps or a guide to how the change may be verified and tested manually?

[NA ] Have you ensured that the full suite of tests and checks have been executed in the root incubating-metron folder via:
mvn -q clean integration-test install && build_utils/verify_licenses.sh

[ NA] Have you written or updated unit tests and or integration tests to verify your changes?

[ NA] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[NA ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[NA ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
mvn site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

+1 Thanks, Otto!  Tested on OSX and CentOS.  Worked as expected.


[My Collection] Add ability for user to select photos

The type of this PR is: Enhancement
Description
This PR adds support for selecting photos from a user's camera roll and managing (in memory -- no persistence just yet).
Note there's a weird bg transparency issue on the close icon; question is out to MX.
Screenshots

#trivial
#skip_new_tests

Warnings

:warning:
Missing Test Files:

src/lib/Scenes/Consignments/v2/Screens/AddArtwork/Screens/__tests__/MyCollectionAddArtworkAddPhotos-tests.tsx

If these files are supposed to not exist, please update your PR body to include "#skip_new_tests".

  Generated by :no_entry_sign: dangerJS

Warnings

:warning:
Missing Test Files:

src/lib/Scenes/Consignments/v2/Screens/AddArtwork/Screens/__tests__/MyCollectionAddArtworkAddPhotos-tests.tsx

If these files are supposed to not exist, please update your PR body to include "#skip_new_tests".

  Generated by :no_entry_sign: dangerJS


DatabaseBackend.GetLatestLedgerSequence can crash with "index out of range" error

If SelectRaw in DatabaseBackend.GetLatestLedgerSequence returns no rows it crashes with:
runtime error: index out of range [0] with length 0

Check why SelectRaw doesn't return ErrNoRows in such case.

fixed in https://github.com/stellar/go/pull/1914


Make an HTTPS request to endpoint instead of touch after deploy/update

Context

When a zappa project is updated/deployed, zappa by default will "touch" the lambda to warm it up. This is helpful for speeding up initial requests.
Instead of doing a touch, zappa could make a GET request to some endpoint as specified in the zappa_settings.json (defaults to '/'). This would have the same effect as the touch to warm up the lambda, but the added benefit is that zappa could catch 500 errors right away. These 500 errors are currently one of the most common problems for migrating to zappa and often indicate missing or incompatible packages in the project for the lambda environment.
Expected Behavior

Zappa informs users when their app returns a 500 error after deployment.
Actual Behavior

Users must visit their app to initiate any errors.
Possible Fix

Instead of the touch functionality, make a request to a "touch_endpoint" setting, defaults to '/'. Zappa would only alert users if a 500 error is returned.

Bonus points if the zappa update exits uncleanly (non-0 exit code), so it can be better scripted, too.

It looks like the "touch" is done here and here.
We can add better logging here for when the request is unsuccessful, and add an option in zappa_settings for a touch_endpoint.
Would we need to change the way the request is performed? It looks like we're already performing a GET

This issue could be closed, it's been implemented
https://github.com/Miserlou/Zappa/commit/e7d0f8e


VEGA-1469 Display fee reduction details

#minor

Codecov Report
Base: 73.44% // Head: 73.78% // Increases project coverage by +0.34% :tada:

Coverage data is based on head (0c4f711) compared to base (a3d1a44).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #147      +/-   ##
==========================================
+ Coverage   73.44%   73.78%   +0.34%
==========================================
  Files          65       65
  Lines        1555     1568      +13
==========================================
+ Hits         1142     1157      +15
+ Misses        251      249       -2
  Partials      162      162

Impacted Files
Coverage Δ

internal/sirius/payment.go
42.85% <ø> (ø)

internal/server/get_payments.go
94.59% <100.00%> (+2.00%)
:arrow_up:

internal/sirius/apply_fee_reduction.go
53.57% <100.00%> (ø)

internal/sirius/case.go
100.00% <0.00%> (+25.00%)
:arrow_up:

internal/sirius/task.go
100.00% <0.00%> (+25.00%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


gbm build dependency

I’ve been building wlroots for quite some time now, and yesterday wlroots now requires “gbm” to build. I have a Radeon card, so I for sure have gbm. Would I say the mesa package provides gbm for this to build again?

wlroots has always required GBM. GBM is part of Mesa. I'm not sure I understand why your build fails.

wlroots has always required GBM. GBM is part of Mesa. I'm not sure I understand why your build fails.

Maybe I need to add gbm to  the provides of my mesa package. Could I be compiling mesa without gbm? I do compile a minimal version of mesa. I havent had any issues with it until yesterday and sway works fine. Heres the log in case it helps.
wlroots-nosystemd-git-0.12.0.r247.g32c30481-1-x86_64-build.log

Maybe I need to add gbm to  the provides of my mesa package. Could I be compiling mesa without gbm? I do compile a minimal version of mesa. I havent had any issues with it until yesterday and sway works fine. Heres the log in case it helps.
wlroots-nosystemd-git-0.12.0.r247.g32c30481-1-x86_64-build.log

Maybe. Mesa has a -Dgbm=enabled flag you can set.
Closing because this isn't a wlroots issue.

Maybe. Mesa has a -Dgbm=enabled flag you can set.
Closing because this isn't a wlroots issue.


Update generated headers and remove CL/GL types from cl_platform.h

Move CL/GL type declarations from cl_platform.h to cl_gl.h
Include version macros for extensions

Signed-off-by: Kevin Petit<EMAIL_ADDRESS>

Oh yes, you're right. The few seconds I spent observing the absence of a mako template for cl_gl.h were not enough!
I've created a PR (https://github.com/KhronosGroup/OpenCL-Docs/pull/1210) separate from KhronosGroup/OpenCL-Docs#960 (that I agree will likely require a bit more discussion) and repurposed this one to update the generated headers and manually remove the types from cl_platform.h.

I think this should be fairly non-contentious, so I'm going to go ahead and merge this offline.


fix:  set empty plan id in upcoming phase for subscription cancel

Pull Request Test Coverage Report for Build<PHONE_NUMBER>2
Details

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.001%) to 9.468%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

billing/subscription/service.go
0
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>2:
-0.001%

Covered Lines:
10266

Relevant Lines:
108423

💛  - Coveralls


Make drivers use operator.Operand use the Dialect to expand

Currently, we have Operator creating its own strings, however, depending on the DB type, these values might be different. Right now, Operator follows ANSI rules, but might change in the future. The current way does not allow Dialects to add their own Operator types.

Already occurring. Should instead move order and operator etc into expressions.


Issue in using training AnimateLCM SVD

Thanks for the great work, also for releasing the training script train_svd_lcm.py.
I am trying to reproduce the results using the provided train_svd_lcm.py, but after half of the training (20,000 / 50,000 itrs) don't see any improvement neither in loss value nor generation qualities (training on a single A100 on WebVid2M).
Could you please confirm if Ishould set the hyper-params as follows?
accelerate launch train_svd_lcm.py \
    --pretrained_model_name_or_path=stabilityai/stable-video-diffusion-img2vid-xt \
    --per_gpu_batch_size=1 --gradient_accumulation_steps=1 \
    --max_train_steps=50000 \
    --width=576 \
    --height=320 \
    --checkpointing_steps=1000 --checkpoints_total_limit=1 \
    --learning_rate=1e-6 --lr_warmup_steps=1000 \
    --seed=123 \
    --adam_weight_decay=1e-3 \
    --mixed_precision="fp16" \
    --N=40 \
    --validation_steps=500 \
    --enable_xformers_memory_efficient_attention \
    --gradient_checkpointing \
    --output_dir="outputs" \
In the current train_svd_lcm.py, the model is being trained on 576x320 resolutions, which is much lower than the standard SVD, i.e., 1024x572. Would not this cause a problem as normal (non LCM) SVD suffer from generating lower resolution videos?
Any input is much appreciated :)

Hi, thanks for the interest!

I set the default resolution 576x 320 because  I find that even trained with lower resolution, it still works to accelerate the generation of SVD. The generation quality of SVD on resolution 576 x 320 is not ideal, and therefore even trained with lower resolutions,  you'd better log the generation results in the resolution of 1024x576.  Also, for better results, using 1024x576 for training is definitely a better choice when the resource is sufficient.
Also, I would recommend slightly decreasing the learning rate since you trained the model with only one GPU.

Also would say that the default hyper-parameters applied in the training script are not carefully tailored and should just be sub-optimal. For example, using EMA should generally increase the generation stability.

Thanks for the swift response :)
I am now switching to 4xA100 and unfortunately still see vague blobs like the attachment. Curious to know at what iterations should I expect the generations start looking like a video? :)
Thanks!

The results uploaded seem to be un-normal. It should not flash like this with unnatural colour. Here's what I obtained trained on 576x320.
Training beginning, 0-iter, cfg = 1, inference step = 4

10k iter, cfg=1, inference step = 4

I just found the code at this line was a typo, and I fixed it. Just hope it did not mislead you.

Amazing! It start to look good after fixing the typo.
ThanQ :)

Awesome! Very glad to hear that : D.

Hey @habibian, just uploaded an example dataset.py.
In addition to that, I would recommend freezing all the convolutional layers when training because convolution layers seem to be more vulnerable for fine-tuning.
Hope this will help for better performance.

Thanks for the response @G-U-N .
Regarding the freezing the convolutional layers, do you mean the ones in ResBlocks? Is is part of your implementation, or I need to implement it?
Thanks!

Hi @habibian,
Yes, the ResBlocks. That was not implemented in the training script. But  it should be easy to achieve that through modifying this line.

Hey @G-U-N ,
Thanks for the input. Following your suggestion, I kept conv layers in resblocks frozen during the training as:
    for name, para in unet.named_parameters():
        # freeze resnet convs as suggested in https://github.com/G-U-N/AnimateLCM/issues/22#issuecomment-2094802365
        if 'conv' in name and not ('conv_in' in name or 'conv_out' in name):
            para.requires_grad = False
        else:
            para.requires_grad = True
            parameters_list.append(para)

I actually observe some improvements in training with this modification as:
Convs Frozen: 20k iter, cfg=1, inference step = 4

All Finetuned: 20k iter, cfg=1, inference step = 4

However, I still see my trained models to have much lower quality compared to the SVD checkpoint that you guys have released SVD checkpoint: . Here are some more test examples to give  you some idea about how poor the quality of my replications are. So wonder if you have trained SVD checkpoint as I am doing here, or maybe there are some differences, i.e., in code, data, etc?
Thanks a lot for your guidance and support in replicating your excellent work :)
Convs Frozen: 20k iter, cfg=1, inference step = 4

Hey @habibian , I would say there's no too much difference. The only difference is that I tried to freeze more weights at the beginning of training instead of fully fine-tuing.  I didn't do too much ablation on that due to my limited GPU resources.
What about trying this:
 for name, para in unet.named_parameters():
    if "transformer_block" in name and "temporal_transformer_block" not in name:
        para.requires_grad = True
        parameters_list.append(para)

Again, I would recommend logging the generated videos in resolution 1024 x 576. You would get ideal results  on low resolutions even if you train the model successfully.
LMK if you get better results.

Hi @habibian, just checking in to see if you have any updates. Hope everything is going well on your end!

Hey @G-U-N
Thanks for the suggestion and your great support here, much appreciated!
Following your last suggestion, instead of finetuning all except resblocks I am now only finetuning spatial_transformer_blocks that is actually improving the results as follows:
Finetuning all except resblocks: 20k iter, cfg=1, inference step = 4

Finetuning spatial_transformer_blocks: 20k iter, cfg=1, inference step = 4

Finetuning all except resblocks: 20k iter, cfg=1, inference step = 4

Finetuning spatial_transformer_blocks: 20k iter, cfg=1, inference step = 4

Finetuning all except resblocks: 20k iter, cfg=1, inference step = 4

Finetuning spatial_transformer_blocks: 20k iter, cfg=1, inference step = 4

Finetuning all except resblocks: 20k iter, cfg=1, inference step = 4

Finetuning spatial_transformer_blocks: 20k iter, cfg=1, inference step = 4

And, here are the 1024 x 576 generated videos using my trained checkpoint (compared to your released checkpoint):
Finetuning spatial_transformer_blocks: 20k iter, cfg=1, inference step = 4

You released AnimateLCM-SVD-xt-1.1 checkpoint: ? iter, cfg=1, inference step = 4

Finetuning spatial_transformer_blocks: 20k iter, cfg=1, inference step = 4

You released AnimateLCM-SVD-xt-1.1 checkpoint: ? iter, cfg=1, inference step = 4

As you see, there is still a gap in generation qualities, which I am not sure how can be reduced. Is the released checkpoint trained with 50K iterations? Any particular multi-stage training or lr scheduling involved?
Thanks :)

Hey @habibian. Very glad to see the improvement! And I really appreciate the detailed visual ablations.
I actually conducted the training in two-stage.

30k iterations with only spatial transformer block tuned with learning rate 1e-6.

if "temporal_transformer_block" not in name and "transformer_block" in name

50k iterations with only temporal transformer block tuned with learning rate 3e-7. (The temporal weights of SVD is relatively large and vulnerable.)

if "temporal_transformer_block" in name

Additionally, some more iterations on larger resolutions will help enhance the performance.
Hope this will make better performance!

Hey @G-U-N ,
Great, thanks for the elaboration. I will follow this multi stage training and get back to you about results.
For that, could you please describe a bit the details of the large resolution training? More specifically:

Training videos and resolution
Number of iterations and learning rate

Thanks!

@habibian
Training videos: bilinear interpolated webvid-2M.
Resolution: Only spatial transformer block, a 80 GB GPU should be able to train on resolution 1024x576.
Iterations: 10k~30k
learning rate: 1e-6

05/11/2024 00:34:18 - INFO - main -   Total optimization steps = 50000
Steps:   0%|                                                                                | 0/50000 [00:00<?, ?it/s]Traceback (most recent call last):
File "/data/vjuicefs_hz_cv_enhance_v1/public_data/11105507/video_edit/AnimateLCM/animatelcm_svd/train_svd_lcm.py", line 1337, in
main()
File "/data/vjuicefs_hz_cv_enhance_v1/public_data/11105507/video_edit/AnimateLCM/animatelcm_svd/train_svd_lcm.py", line 1081, in main
teacher_pred_cond = teacher_unet(inp_noisy_latents, timesteps, encoder_hidden_states, added_time_ids=added_time_ids).sample
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/diffusers/models/unet_spatio_temporal_condition.py", line 434, in forward
sample, res_samples = downsample_block(
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/diffusers/models/unet_3d_blocks.py", line 2173, in forward
hidden_states = attn(
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/diffusers/models/transformer_temporal.py", line 359, in forward
hidden_states_mix = temporal_block(
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/diffusers/models/attention.py", line 509, in forward
attn_output = self.attn1(norm_hidden_states, encoder_hidden_states=None)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/diffusers/models/attention_processor.py", line 527, in forward
return self.processor(
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/diffusers/models/attention_processor.py", line 1175, in call
hidden_states = xformers.ops.memory_efficient_attention(
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/xformers/ops/fmha/init.py", line 223, in memory_efficient_attention
return _memory_efficient_attention(
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/xformers/ops/fmha/init.py", line 321, in _memory_efficient_attention
return _memory_efficient_attention_forward(
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/xformers/ops/fmha/init.py", line 341, in memory_efficient_attention_forward
out, * = op.apply(inp, needs_gradient=False)
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/xformers/ops/fmha/flash.py", line 458, in apply
out, softmax_lse, rng_state = cls.OPERATOR(
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/torch/_ops.py", line 755, in call
return self._op(*args, **(kwargs or {}))
File "/root/miniconda3/envs/animatelcm_svd/lib/python3.9/site-packages/xformers/ops/fmha/flash.py", line 106, in _flash_fwd
) = _C_flashattention.fwd(
RuntimeError: CUDA error: invalid configuration argument
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Hey @ersanliqiao.
You should load the unet and target unet from your finetuned weight and initialize the teacher unet with stability weight.
Try this at this line
from safetensors.torch import load_file
finetuned_weight = load_file("xxx.safetensors","cpu")
unet.load_state_dict(finetuned_weight)
target_unet.load_state_dict(finetuned_weight)
del finetuned_weight

thank you!!

hi @habibian
can i ask you why you are trying to train the model?
I am trying to use AnimateLCM model, but do not check weather training is better or not yet.
Do you have any specific reason?

hi @dreamyou070
I needed to retrain AnimateLCM on a different UNet to run faster than standard SVD architecture.

Hi @G-U-N, thanks for your great open-source work
I have some questions about loss weighting when training svd-lcm (codes):
loss = torch.mean(weights) * ...,
where the weights is defined here:
self.weights = (1/(self.sigmas[:-1] - self.sigmas[1:]))**0.1
This formulation seems a bit different from the representation of λn in the arXiv paper:
$$\lambda_n = ((1 - \delta \frac{n}{N}))^{\gamma}$$
I'd like to know if the formulation used in the code is based on any reference paper or if it is just a heuristic setting.

Hey, @haohang96 . Yes, I would say the choice of weights is very heuristic and hard to give an explicit analysis. Most designs are  heuristic and should be sub-optimal.

@habibian Hi, have you obtained results similar to the released AnimateLCM-svd-xt? I fine-tuned the Spatial Transformer layer for 30k iterations, the results look like what you have shown above.

@habibian Hi, have you obtained results similar to the released AnimateLCM-svd-xt? I fine-tuned the Spatial Transformer layer for 30k iterations, the results appear as blurry as what you've shown above.

trainable parameters are set as follows:

    unet.requires_grad_(False)
    parameters_list = []

    # Customize the parameters that need to be trained; if necessary, you can uncomment them yourself.

    for name, para in unet.named_parameters():
        # 1 stage: 30k iterations with only spatial transformer block tuned with learning rate 1e-6.
        #  Only temporal transformer block, a 80 GB GPU should be able to train on resolution 768x448.
        if args.training_stage == 1:
            if "temporal_transformer_blocks" not in name and "transformer_blocks" in name:
                para.requires_grad = True
                parameters_list.append(para)
        elif args.training_stage == 2:
        # 2 stage: 50k iterations with only temporal transformer block tuned with learning rate 3e-7. (The temporal weights of SVD is relatively large and vulnerable.)
        # Only spatial transformer block, a 80 GB GPU should be able to train on resolution 1024x576.
            if "temporal_transformer_blocks" in name:
                para.requires_grad = True
                parameters_list.append(para)


Fix for OUJS

OUJS has made a change recently to require SPDX codes for OSI approved licensing.
In order to improve the appearance of your script homepages it would be appreciated if you could modify all of your affected script. These are syntactically equivalent.
Until this change is made you will be unable to update those affected scripts.
Thanks,
OUJS Staff

Yvonne gained +1 knowledge.
Thank you! :-)

Welcome. Thanks.


Problems connecting to GKE cluster

I've noticed this pattern of connecting to a Google Kubernetes Engine cluster where it works for a short period of time, then errors out and I'm never able to reconnect again.
Environment (please complete the following information):

Lens Version: 3.6.0
OS: Windows 10
Installation method (e.g. snap or AppImage in Linux): MSI from Lens website

020/09/21 19:40:29 http: proxy error: error executing access token command "/usr/lib/google-cloud-sdk/bin/gcloud config config-helper --format=json": err=exec: "/usr/lib/google-cloud-sdk/bin/gcloud": file does not exist output= stderr=

Fixed this by just updating the cmd-path to just gcloud
Here is the kubeconfig snippet for reference:
users:
  - name: omitted
    user:
      auth-provider:
        config:
          cmd-args: config config-helper --format=json
          cmd-path: gcloud
          expiry-key: '{.credential.token_expiry}'
          token-key: '{.credential.access_token}'
        name: gcp

@ariesyous Awesome, thanks! This fixed my problem with WSL2, as the paths are Linux and Lens is installed on Windows, using Windows paths 😅

@ariesyous Awesome, thanks! This fixed my problem with WSL2, as the paths are Linux and Lens is installed on Windows, using Windows paths 😅

Hi, stack with the same problem, can you share yours solution please?

@davidgourde How do you exactly fixed it? IDK what to do with @ariesyous snippet!

Did you manually edited it through vi?

I faced the issue one more time, I had to edit manually my kube_config as follows:
This configuration fails with the following error:
- name: name
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: gke-gcloud-auth-plugin
      installHint: Install gke-gcloud-auth-plugin for use with kubectl by following
        https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl#install_plugin
      provideClusterInfo: true

proxy_server.go:147] Error while proxying request: getting credentials: exec: executable gke-gcloud-auth-plugin not found It looks like you are trying to use a client-go credential plugin that is not installed. To learn more about this feature, consult the documentation available at: https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins Install gke-gcloud-auth-plugin for use with kubectl by following https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl#install_plugin

I had to manually edit my .kube/config to point to the path where the plugin was installed, I have to do this with every cluster:
- name: name
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: /Users/google-cloud-sdk-2/bin/gke-gcloud-auth-plugin
      installHint: Install gke-gcloud-auth-plugin for use with kubectl by following
        https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl#install_plugin
      provideClusterInfo: true


ENH: stats: fit: function for fitting discrete and continuous distributions

Reference issue
Closes gh-11948
What does this implement/fix?
As discussed in gh-11948, this adds a function scipy.stats.fit(dist, data, ...) to fit discrete and continuous distributions to data. Thanks to the addition of integrality constraints to differential evolution (thanks @andyfaff!), the algorithmic part is extremely simple. Almost all of this is input validation, distribution domain info, and tests.
Example:
import numpy as np
from scipy import stats
rng = np.random.default_rng()
dist = stats.nbinom
shapes = (5, 0.5)
data = dist.rvs(*shapes, size=1000, random_state=rng)
shape_bounds = {'n': (0, 30)}
res = stats.fit(dist, data, shape_bounds)
res.params  # FitShapes(n=5.0, p=0.4969793457498593, loc=0.0)
res.plot()

Suggested code for test driving:

import numpy as np
from scipy import stats
from scipy.optimize import differential_evolution
from scipy.stats._distr_params import distdiscrete, distcont

# for now, works for all discrete distributions plus norm and nakagami
# ['bernoulli', 'betabinom', 'binom', 'boltzmann', 'dlaplace', 'geom',
#  'hypergeom', 'hypergeom', 'nhypergeom', 'nchypergeom_fisher',
#  'nchypergeom_wallenius', 'logser', 'nbinom', 'planck',
#  'poisson', 'randint', 'skellam', 'zipf', 'zipfian', 'yulesimon',
#  'nakagami', 'norm']
dist_name = 'nakagami'
N = 5000
rng = np.random.default_rng(65146251968)

# get information about distribution
dist_data = dict(distdiscrete + distcont)  # example parameters
dist = getattr(stats, dist_name)
shapes = dist_data[dist_name]
shape_names = dist.shapes.split(', ')

# automatically prepare some bounds; feel free to adjust
bounds = np.empty((len(shapes), 2), dtype=np.float64)
bounds[:, 0] = np.array(shapes)/10  # essentially all shapes are > 0
bounds[:, 1] = np.array(shapes)*10

# bounds can be specified as either an array_like or a dictionary of tuples
bounds_dict = {param: bound.tolist()
               for param, bound in zip(shape_names, bounds)}

# generate some data
data = dist.rvs(*shapes, size=N, random_state=rng)

# not required; just demonstrating how a custom optimizer / optimizer options
# can be configured
def opt(fun, bounds, integrality=None, x0=None):
    return differential_evolution(fun, bounds, integrality=integrality, x0=x0,
                                  seed=rng)

res = stats.fit(dist, data, bounds, optimizer=opt)  # or bounds_dict
res.plot()
print(res)

Additional information
Currently, all discrete distributions and two continuous distributions (norm and nakagami) are supported. The rest are easy to add, but it's important to take some time for each one to make sure there are no surprises. I think that adding the remaining distributions can and should be crowdsourced to get more people testing out the code.
In the future, this function could power a fit method of discrete distributions, but for now, a new function with a cleaner interface and codebase than rv_continuous.fit can be useful for both continuous and discrete distributions.
Of course, users can immediately fit custom distributions by subclassing rv_continuous or rv_discrete. We could add support for simpler custom distributions, but I'd prefer to start with the basic functionality here.

Can you amend the PR description with stats.test() and stats.test('full') timings before/after?  The overwhelming majority of tests failures will just mean that optimization didn't converge to the global optimum. That's not a bug in fit or differential_evolution. We just can't guarantee that we will find the global MLE with arbitrary bounds and random state. So I'm happy with skipping lots of them to keep the time down.
Can you suggest a time limit for regular tests and a time limit for slow tests, and I'll mark the rest as xslow?
Or, how much time would be acceptable to add to the regular and slow test suites? I can figure out the details to get us there.

I wonder from time to time how to figure out if there's an improved way of detecting convergence with differential_evolution. At the moment it figures out if something is converged by dividing the stdev of the population by the mean, and stop if it's less than the tol. I suspect for some situations there's could be wasted iterations because outliers prevent halting. (for me wasted iterations don't matter so much, I just need to know that the optimum is a good one). I reckon for these tests you could experiment with relaxing tol, and seeing if they converge more quickly.
Can the objective functions be vectorised (see #14528)? If so, then that might be something to experiment with, it may provide a speed-up.

Can the objective functions be vectorised (see #14528)?

Yes, easy. I'll do that.

I'd use timeit first, not sure how much it'll change the situation.

Oops by "tests" I didn't mean unit tests; I meant "tests" in the generic sense using timeit.
Specifically, before I submitted the integrality PR for brute, I wrote my own version of vectorized brute and used it in the zooming/iterative brute strategy for solving these problems. It was not really any faster, and I think the reason is that each objective function evaluation involves calculating the PDF of a distribution at N (1000 ~ 10000) data points, and that calculation is already vectorized. At that point, vectorizing w.r.t. the decision variables mostly starts to eat up memory without making the calculation too much faster.
Vectorizing DE (and brute) is a great idea in general, it just doesn't seem to do too much for these problems.

Can you suggest a time limit for regular tests and a time limit for slow tests, and I'll mark the rest as xslow?
Or, how much time would be acceptable to add to the regular and slow test suites? I can figure out the details to get us there.

How about >1 sec for xslow and >0.25 sec forslow? There are a lot of distributions, so I think setting the limits a bit more strictly than for a single test makes sense.

How about >1 sec for xslow and >0.25 sec for slow

Sounds good. I'm going to leave them in while we're developing, but before merge I'll add these marks.

Thanks @tupui. I think I implemented most suggestions and responded to the rest.

Hi Matt,
this looks very useful, thanks for working on it. I just had a quick look at the code and I have the following question:
I saw that you can provide bounds on the parameter. I did not see an option to provide an initial guess of the parameters, like _fitstart in rv_continuous. Unless I overlooked it, this might be a useful option. What do you think?

Unless I overlooked it, this might be a useful option. What do you think?

I agree it is useful to support that. A few reasons why it's not already included as a separate parameter:

Global optimizers don't need it like the local optimizers used (by default) by rv_continuous.fit. We trust them to adequately explore the space specified by bounds.
I'm trying to keep the interface really simple. Partially this is a reaction (logical or not) to users' misunderstandings about rv_continuous.fit, especially the distinction between e.g. floc and loc. Want to get far away from that.
A user can still pass a guess if desired using optimizer. I'd be happy to include an example in the documentation, if that would be enough?

optimizer = lambda fun, bounds, integrality: differential_evolution(fun, bounds, integrality=integrality, x0=my_guess)

I thought about combining all the parameter bounds (including loc, scale) into a single structure, e.g. param_bounds. Then I suppose it wouldn't be too bad to add one more argument param_guesses. But right now it would probably need to be three.

thanks for the clarification why there is no parameter to specify starting values.

A user can still pass a guess if desired using optimizer. I'd be happy to include an example in the documentation, if that would be enough?

I think that would be helpful

What do you think about specifying all parameter bounds in the same input? That is, instead of shape_bounds, loc_bounds, and scale_bounds, the user would just pass param_bounds. The ways of specifying them would be similar to how shape_bounds works now.
Then I wouldn't mind adding a param_guesses input.

What do you think about specifying all parameter bounds in the same input? That is, instead of shape_bounds, loc_bounds, and scale_bounds, the user would just pass param_bounds. The ways of specifying them would be similar to how shape_bounds works now.

this like this idea!

@tupui if you agree, I'll make this change?

@tupui if you agree, I'll make this change?

If it's just a matrix it would be less explicit, but if it's a dict or something else why not.

Ok. I'd still like to allow a sequence of tuples, too - just like I do now. I am think it will be useful when the bounds are generated by other code rather than direct user input. But in that case they will still have to provide all shapes. Yes, the dict is the more explicit, careful way of providing the input.

@chrisb83 @tupui Thanks. I hope to get to this tomorrow or this weekend, then I'll send an email to the mailing list.
If you're interested in fit confidence intervals, please see gh-15491 and mdhaber/scipy#70. With those merged, I think bootstrapped confidence intervals would be a pretty easy followup PR. (Analytical confidence intervals using the inverse information matrix can be done later... but I still don't know how to do that if there are discrete parameters.) They'll be somewhat slow - especially if discrete parameters are being fit - but I think it's a big win just to have the feature.
And I already mentioned it to @tupui, but @chrisb83 if you'd like to see fitting to censored data, we should resolve gh-13757 (and merge gh-13699) first.

@chrisb83 @tupui what do you think? ready for a message to the mailing list?

I have a general question. Do you know of strategies to combine non parametric with parametric distribution fitting? Since we will have KDE and this PR, I am thinking about a way to provide a sample to a method that would output a distribution directly. Naive approach being, KDE to help get the type of distribution and then call the fit method of said distribution (or small group of distribution and take the best fit).

@tupui I don't know personally but please see what fitdistrplus does:
https://www.jstatsoft.org/article/view/v064i04
Toward the beginning of that paper, they talk about this sort of thing.

@tupui @chrisb83 Message was sent to the mailing list on Wednesday, 2/8/2022. Shall we merge this Wednesday if there are no more comments?

I had another look, all good for me with the last changes (the dict/tuple for bounds looks good and more flexible)

If so, let me know if you'd like this moved to stats/_fit.py as discussed here, and I'll also mark slow tests as discussed here.

We should do this, but can be a follow up PR. As you wish here.

OK, marked tests slow/xslow as discussed, so I think this is ready if tests pass. (Most likely failure is mypy. If it fails, I'll patch without re-running tests.)
After this merges, I'll submit a followup to move it all to a new file. We can remove dist and data as public attributes of the result object in a followup PR, if desired.

OK, marked tests slow/xslow as discussed, so I think this is ready if tests pass. (Most likely failure is mypy. If it fails, I'll patch without re-running tests.)
After this merges, I'll submit a followup to move it all to a new file. We can remove dist and data as public attributes of the result object in a followup PR, if desired.

👍 I can merge once green.

I went ahead and made the FitResult _dist and _data private. We can always make them public later if desired, but it's tough to go the other way around.

Doc looks good (https://43309-1460385-gh.circle-artifacts.com/0/html-scipyorg/reference/generated/scipy.stats.fit.html) and CI failures are not related. Thanks again everyone! 😃

Thanks @tupui, @chrisb83. Added this to the release notes. Let me know if that looks appropriate.
I thought I'd use this as the tracker for additional features:

[ ] move source to _fit.py (gh-15615)
[ ] _shape_info to remaining distributions
[ ] dispatch to overridden fit/_fit methods
[ ] vectorize NLLF / call differential_evolution with vectorized=True
[ ] other fit objective functions (e.g. method of moments)
[ ] other goodness-of-fit plots
[ ] easy support for other optimizers (e.g. optimizer='minimize')
[ ] support censored data (gh-13699 pending)
[ ] inverse fisher information standard errors / confidence intervals (for all continuous parameters)
[ ] bootstrap standard errors / confidence intervals (easy once gh-15491 and mdhaber/scipy#70 merge)

@chrisb83 @tupui I think it would be good to add _shape_info to the remaining distributions first. Would you prefer to write some of this code and I'll review or vice-versa?

I can work on it this weekend: as a start, I would try to work up to line 3349 in continuous_distns.py, i.e., up to class gumbel_r_gen(rv_continuous)

Perfect! I can help in real-time, if you ever need it. Should be quick to review, too.

@rgommers a message was send to the ML https://github.com/scipy/scipy/pull/15436#issuecomment-1037496401

Hmm, I missed that and it didn't have any replies. Ah well, a bit late now perhaps to do anything about it.

We still have a few days. If you have a better name I can refactor this quickly.

Ok thanks, I can prepare this. @mdhaber can you confirm the name?

I can do it, @tupui. I think fit_distributions was suggested, bit I preferred fit. Is the concern that people will read through the list of functions available and try to use it for the wrong purpose without reading the first line of the documentation?

On the optimization side. I prefer to also force users to be explicit about bounds on shape/loc since these have strong effects on the output. It's too hard otherwise for the optimisers to play along with second order effect params.

Those are the issues I was thinking of.  I can't think of others, but I don't imagine the idea is limited to those distributions. For instance, I think loc being free for discrete distributions could also be surprising.

I was fine with fit when I looked at this initially because in sklearn for instance, almost everything has a fit method and all of these are doing very different things for the given algo. So context matters.

Methods are quite different, because they're already attached to the thing you provide. fit as a function just breaks descriptive API naming rules, it's always fit_<what> (or estimate_<what> or calc_<what>).

I suppose there is some other fitting. I thought distribution fitting was the most fundamental fitting in stats, and the name is familiar to users of distributions' fit method, but fit_distribution is ok if need be.

You can argue that for scipy.stats as it as today indeed, but I'd think not for statistics. Regressions like https://www.statsmodels.org/stable/user-guide.html#regression-and-linear-models are a lot more common. So if you keep fit, I guess you're looking at scipy.stats as "lots of statistical distributions, plus some other assorted things" as the scope (and it won't change over time)?
Anyway, it was proposed so you can keep it if y'all agree. I should have flagged this before merging.

I have the same feelings as @rgommers on this. I'd far prefer a fit_distribution name for the reasons given, but with the timing, I won't insist on it.


Wrong type of comments for React code

VSCode Version: 1.1.1
OS Version: Windows 10

Steps to Reproduce:

position cursor on a line of react html
press ctrl-/  to comment out the line

It will insert // instead of {/*    */}

@keithn What is the filename? What language appears in the bottom right?

here's a sample piece of code,  line 19 got the wrong comments, its a .jsx and says JavaScript React

Ok, now I think I understand. The inline react html section does not recognize // as a comment. Thanks for clarifying!

yeah, // is not a comment, it has to be commented out with {/*  */}

Any updates?

any updates on this issue?

Wrong toggle line comment and syntax coloring for JSX tags is the only that blocks me moving from atom/nuclide to VSCode

Any updates?

yes, we now support embeddedLanguages when contributing a grammar. @aeschli Can we change the JSX grammar such that it wraps all embedded html/jsx code in a scope that can be used to configure us to detect the language inside there as HTML?

@alexandrudima By looking for meta.tag.js and meta.tag.without-attributes.js we should already be able to detect this...

That's great. I suggest the grammar contribution of JavaScript to also include an embeddedLanguages, similar to HTML: https://github.com/Microsoft/vscode/blob/master/extensions/html/package.json#L58

Can you please provide an update on when commenting out JSX (in a .js file) will be supported ? thanks.

Would be great if someone added this function!

I just did, as you can see from the bug resolution and the attached change set.
It will be in the January build,

I'm having this issue again in the March 2017 build v1.11.
Steps to reproduce:

create new file
save as 'test.js'
enter:

import React from 'react'

export default class Test extends React.Component {
  render() {
    return (
      <div>
       Test
      </div>
    )
  }
}

Press cmd + / on line 6: "Test"

Expected
Line 6 to equal {/*Test*/}
Got instead
Line 6 equals // Test

@mattblackdev I quickly tried your example in the latest insiders build and I got {/Test/}.
Please file a new issue if you still see this

Tested in Version 1.11.2 on Mac OS X and it is not working.

I finally made it work (I was puzzled it worked on my windows but not on my Mac).
SOLUTION: Just change the language to JavaScript (React).
I previously had JavaScript (babel) and it gave bad commenting on jsx.

I have 1.12.1 and I'm still seeing the issue as well. Specifically for me, I have the language set to "Javascript React (Babel)" and it's still commenting HTML-area code inside the return() statement with the double-slash. Not sure what's going on there... Is there a setting somewhere that I might have set that I might have set earlier that I need to clear out now?

Still having the issue as well on 1.12.1 and on "Javascript React (Babel)”. It is with a certain set of projects, however. When I start a new project comments work fine inside JSX.
On May 7, 2017, 6:53 PM -0400, Ryan Mitchell<EMAIL_ADDRESS>wrote:

I have 1.12.1 and I'm still seeing the issue as well. Specifically for me, I have the language set to "Javascript React (Babel)" and it's still commenting HTML-area code inside the return() statement with the double-slash. Not sure what's going on there... Is there a setting somewhere that I might have set that I might have set earlier that I need to clear out now?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

The problem is with the extension associated with Javascript React (Babel). The grammar and the grammar registration from the extension must let us know that the language contains embedded languages.
e.g. From our built-in JavaScript language https://github.com/Microsoft/vscode/blame/master/extensions/javascript/package.json#L72
{
        "language": "javascript",
        "scopeName": "source.js",
        "path": "./syntaxes/JavaScript.tmLanguage.json",
        "embeddedLanguages": {
          "meta.tag.js": "jsx-tags",
          "meta.tag.without-attributes.js": "jsx-tags"
        }
},

This lets VS Code know that within the scopes meta.tag.js and meta.tag.without-attributes.js (specific to this grammar), VS Code should consider the language to be jsx-tags. Then, when the selection is within such scopes, VS Code will use the comment settings defined for jsx-tags.
The resolution is simple: we must find out who defines JavaScript React (Babel) (i.e. what extension) and let them know (file an issue) they should also define the embeddedLanguages section in their contribution.

Found it for me - looks like it's Sublime Babel. I'm appending a note to an existing issue covering this. I disabled the Sublime Babel extension and it appeared to work fine after that. Re-enabled, issue reappeared.

I followed the suggested solution here: Disabled Sublime Babel and switched the file syntax to JavaScript React, and it worked. But, if you comment a line twice it will add // instead of removing the {/* */} comment so you end up with // {/* <Button /> */}

Just use sublime babel, it works fine!

For anyone searching for "vscode comment jsx" (like me)... here is the fix:
Put these in your settings (user or workspace)
"files.associations": {
  "*.js": "javascriptreact"
},

Setting language to Javascript(React), you can comment it properly.
But When you try to uncomment it with ctrl + /, it only deletes /* */ and leaves {  } there.
Please fix it.

@FateRiddle and everyone else. This here is a closed bug. Instead of commenting on this one, please file new issues with the steps to reproduce.


[Backtracing] Fix crash in CrashAsync test on Ubuntu 18.04.

This seems to be happening because we're hitting an invalid frame on the stack where the program counter value is read back as zero.
If we find one of those, treat it as the end of our stack walk.
rdar://110846253

@swift-ci Please smoke test


fix issue 19208 - swapEndian doesn't support floating point types

Prevent implicit truncations using cast to pointers, which we trust since only values can be passed.
For the same reason set the function parameter const  because this can reduce how many instances are generated.

buildkite seems to run the Style check using an older DMD, leading to failures :

std/conv.d(4369): Error: unrecognized trait isZeroInit

ping @wilzbach.


[heroic] Kafka Consumer IT

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@7d5139e). Click here to learn what that means.
The diff coverage is 55.72%.

@@            Coverage Diff            @@
##             master     #233   +/-   ##
=========================================
  Coverage          ?   47.46%
  Complexity        ?     2542
=========================================
  Files             ?      610
  Lines             ?    16033
  Branches          ?     1202
=========================================
  Hits              ?     7610
  Misses            ?     7967
  Partials          ?      456

Impacted Files
Coverage Δ
Complexity Δ

...spotify/heroic/consumer/kafka/RealKafkaStream.java
0% <0%> (ø)
0 <0> (?)

...ify/heroic/consumer/kafka/RealKafkaConnection.java
0% <0%> (ø)
0 <0> (?)

.../com/spotify/heroic/consumer/kafka/Connection.java
76.92% <100%> (ø)
11 <1> (?)

...ify/heroic/consumer/kafka/KafkaConsumerModule.java
75.67% <55.17%> (ø)
8 <0> (?)

...spotify/heroic/consumer/kafka/FakeKafkaStream.java
65% <65%> (ø)
4 <4> (?)

.../spotify/heroic/consumer/kafka/ConsumerThread.java
61.87% <66.66%> (ø)
21 <3> (?)

...e/src/main/java/com/spotify/heroic/HeroicCore.java
71.07% <77.27%> (ø)
29 <0> (?)

...ify/heroic/consumer/kafka/FakeKafkaConnection.java
87.5% <87.5%> (ø)
6 <6> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7d5139e...97bc899. Read the comment docs.


perf(combineLatest): avoid splice and indexOf

Small speed up of combineLatest according to micro-perf benchmarks. CombineLatestSubscriber notifyNext() has toRespond as a number, not an array which is then spliced or searched with indexOf. We prefill the values array with a so-called none object, which is useful for the calculation of the toRespond number.
Before

                                         |                     RxJS 4.1.0 |              RxJS 5.0.0-beta.7 |          factor |      % improved
---------------------------------------------------------------------------------------------------------------------------------------------------
               combinelatest - immediate |                19,142 (±4.60%) |               239,359 (±1.12%) |          12.50x |        1,150.4%
                           combinelatest |                18,787 (±1.83%) |               134,440 (±1.75%) |           7.16x |          615.6%

After
                                        |                     RxJS 4.1.0 |              RxJS 5.0.0-beta.7 |          factor |      % improved
---------------------------------------------------------------------------------------------------------------------------------------------------
              combinelatest - immediate |                17,281 (±0.99%) |               263,042 (±1.21%) |          15.22x |        1,422.1%
                          combinelatest |                17,203 (±1.64%) |               144,099 (±1.57%) |           8.38x |          737.6%

Merged with 33599cd... thanks @staltz


Add option for ignoring volumes defined in images

This adds a new config option for ignoring volumes defined in image metadata.
In Kubernetes volumes (even temporary ones) are generally specified in the pod configuration. Mapping the ones in the image metadata can lead to resource accounting/exhaustion issues (think writing an unlimited amount of inodes or data into the volume) and policy enforcement issues (a PSP cannot disable these).
Specifically for containers started with ReadOnlyRootFilesystem it can also lead to unexpected behavior, for example if there's a typo in the volume mount path the container will still start since the process can write to the data location, but the data ends up in the volume defined by the image which is ephemeral and subsequently gets lost when the pod gets descheduled. The user expected the process in the container to fail since there's no volume mounted at the data path and the root filesystem is readonly, but since the image volume is mounted there this works.

/ok-to-test

@lorenz sounds like this was a concrete issue you ran into. Can you provide link(s)?

I've added a debug log output if we're ignoring volumes and documented the option in config.md. I looked at the commit for the last-added option (TolerateMissingHugePagesCgroupController) and added mine to all the files it touched, but it wasn't added to config.md, that's why I missed it.
The main reason why I've been running this patch for quite some time is because of the issue where incorrectly configured volumes end up writing data into an ephemeral containerd-allocated volume and misbehaving containers writing lots of data into the containerd directory.

/test pull-cri-containerd-verify

I've added a debug log output if we're ignoring volumes and documented the option in config.md. I looked at the commit for the last-added option (TolerateMissingHugePagesCgroupController) and added mine to all the files it touched, but it wasn't added to config.md, that's why I missed it.
The main reason why I've been running this patch for quite some time is because of the issue where incorrectly configured volumes end up writing data into an ephemeral containerd-allocated volume and misbehaving containers writing lots of data into the containerd directory.

yeah I figured as much.. Which of course was my bad, either I missed it .. or just let it go in that pr review and flagged it as a docs issue to be worked on later, can't remember which was a week ago and seems like months :-)

LGTM
@mikebrow Do we actually need the volume here? The user from eBay filed issue about this before containerd/containerd#3511.
The volume handled by CRI plugin doesn't provide interface for user to handle this. The CRI-API spec defines volumes from kubelet, not image config. If the user uses devicemapper as snapshotter with disk quota, the size of rootfs will be ensured with quota. But volume from image config will be leak point.
cc @wenlxie

Hmm.  Reading the other issue (I re-assigned it to cri) it sounds like @Random-Liu already had some ideas for properly handling image specified volumes (report disk usage and /or ask kublet to manage these volumes). Maybe we can recruit him for a discussion at a future sig-node call.
Either way I think this flag provides a good ignore option for now at least until we have a more elaborate solution to map and/or report usage for the image volumes.

Either way I think this flag provides a good ignore option for now at least until we have a more elaborate solution to map and/or report usage for the image volumes.

SGTM

Is this waiting on something to get merged?

Is this waiting on something to get merged?

Just giving maintainers a sufficient opportunity to weigh in.


Fix generation and handling of dimension assocations

This PR fixes these problems:

We didn’t support specification of a multi-dimensional requirement with the pivoted dimension (let’s say commercial end uses only with commercial sectors). Nor did we have proper checking of that condition with unpivoted datasets.
Creation of dimension association tables did not work when there large numbers of dimensions, especially with an unpivoted dataset.
A previous PR to handle unpivoted datasets had a bug when removing null timestamps.

Also, our lack of consistent use of type annotations was leading to bugs. I added lots of annotations.

I just completed full resubmission of all StandardScenarios datasets and derived datasets with this branch. I also created a registry with DECBARB and submitted the buildings dataset. The last few commits were to fix bugs found during that process. We now have much higher confidence in our handling of the various dataset formats.


hugo-head: alternate links break validation tools

Currently the ahrefs check complains about the feed links having language attributes that all point to the same language. Find out how to properly mark up these links without them being marked as translations.
Also: shouldn't they be listed only on the home page?

this was fixed at some point.


Basic Verilator infrastructure

Once Verilator supports all VPI primitives, this is the basic
infrastructure to support it.
This is a work-in-progress PR until Verilator is ready.

I wouldn't consider it for an actual merge until all features are in a verilator release. Would a branch make more sense? My thinking was that if someone searches for it, this PR is the entry point to the joy of testing ;)

@wallento what do you think about closing this PR and opening a new one (or series of PRs) from https://github.com/chipsalliance/cocotb/tree/verilator-support?
@imphil If you'd like to test the integration you can follow the guide from readme in https://github.com/chipsalliance/tools-cocotb-verilator-build

@kgugala let's leave this PR open to make sure the review comments are not lost. And as I said in #227, we can only continue if the VPI changes are merged into Verilator. Please continue with that first, and let me know if cocotb insight is required on any of the upstream PRs.

I am also for keeping this open for now. We will clean up when we're more advanced merging things to master of verilator and cocotb.

Hi,
this can be merged now. I added basic documentation and it works with upstream Verilator >= 4.020.
Thanks to everyone for helping out!
Best,
Stefan

@wallento I'm curious if you've done any benchmarking of this.  Out of the box, Verilator sims are running slower that Icarus ones.  Giving EXTRA_ARGS='-O3 -CFLAGS -O3' to the Verilator build helps a lot, but it still ends up being slower.  Here's what I see on my system for the endian swapper tests:
ics.log: **                              REAL TIME : 3.68 S                                 **
ics.notrace.log: **                              REAL TIME : 3.51 S                                 **
vlt.log: **                              REAL TIME : 40.80 S                                **
vlt.notrace.log: **                              REAL TIME : 26.98 S                                **
vlt.notrace.O3.log: **                              REAL TIME : 6.33 S                                 **

It's been a while since I've done much with Icarus, but my assumption is that Verilator should be faster.  Do you get similar results?

@toddstrader those numbers look indeed not very great. One reason might be the time scale, or its actually the problem of VPI in Verilator. I will have a look too.

Valgrind says something is happening in VerilatedVpi::callValueCbs():
         Ir  file:function
--------------------------------------------------------------------------------
147,987,884  ???:__dynamic_cast [/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25]
127,877,834  ???:VerilatedVpi::callValueCbs() [/home/tstrader/git/cocotb/examples/endian_swapper/tests/sim_build/endian_swapper_sv]
124,508,707  ???:PyEval_EvalFrameEx'2 [/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0]

for reference, the Python EvalFrameEx is pretty consistent between Verilator and Icarus.    I haven't gotten any further than this yet but wanted to post what I've seen so far.

Thanks, tbh, I was not able to fully understand the current travis.yml and heard rumors there may be updates, so I was not too tempted ;)

Hi,
added commits afters reviews. Ready to merge?
Cheers,
Stefan

Rebased and cleaned up, last item is to complete https://www.veripool.org/issues/1636-Verilator-Add-vpiTimeUnit-and-allow-to-specify-time-as-string which is around the corner.

Thanks Stefan! LGTM with the nits addressed. The move to unique_ptr can come as a follow-up, but please add the free (it's strictly optional, but good to have nonetheless).

Added it already, its in the scope of the Verilator build infrastructure anyways..

The patch about the timescale is upstream: https://github.com/verilator/verilator/commit/3ac6745658253bb45a74a0827b0b9260bda69027
This can be merged now imo


What happens if source storage account has firewall enabled and blocks the destination storage account?

The article states that when we copy blobs between storage accounts, the copy is performed service-to-service. Based on another older announcement and some Storage Log analysis, it appears that the destination Storage Account attempts to read the blobs from the source Storage Account. What happens if we have the Storage Account firewall enabled on the source Storage Account that would inadvertently block the copy operations from the destination Storage Account? If this would halt or block the overall copy operation, would it be recommended that we include in this article that the source Storage Account Firewall should allow connections from the destination Storage Account's IP in order for a service-to-service operation to be successful?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 59827b34-a728-ac8d-0ba0-0b60cde2321d
Version Independent ID: 8daf39eb-e037-b5e5-874e-c9b556cc2433
Content: Copy blobs between Azure storage accounts with AzCopy v10
Content Source: articles/storage/common/storage-use-azcopy-blobs-copy.md
Service: storage
Sub-service: common
GitHub Login: @normesta
Microsoft Alias: normesta

@milope Thank you for reaching out to us with your query, our team will review and get back to you at the earliest.

@milope According to the documentation: Your client must have network access to both the source and destination storage accounts. To learn how to configure the network settings for each storage account, see Configure Azure Storage firewalls and virtual networks.
I tested this by creating two blob storage accounts, in the firewall settings I only allowed my IP for both source and destination accounts. AzCopy command completed without issue.


Hashie::Extensions::DeepGrep

Grep for hash.
 require 'hashie'

 options = {
   users: [
     { location: { address: '123 Street', name: 'vmarutha' } },
     { location: { address: '234 Street', name: 'dblock' } }
   ],
   other_users: ['BobbyMcWho', 'michaelherold']
 }
 options.extend(Hashie::Extensions::DeepGrep)

Currently, if a key/value matches the pattern, the result will include the parent hash. This is because we are preserving the match context, and I think it would be useful to retain.
 options.deep_grep(/Street/) # => [{:address=>"123 Street", :name=>"vmarutha"}, {:address=>"234 Street", :name=>"dblock"}]

If a value matches the pattern and is in a list, the associated key is not returned. Thinking on how to make this a more uniform results list irregardless of matching data type. This may be a bit tricky though especially with enumerables as the key or value in the hash. Let me know if you have suggestions for this case!
 options.deep_grep(/Bob/) # => [["BobbyMcWho", "michaelherold"]], note no key :other_users returned

Open to merge as is, and comeback to above edge case later on since the main functionality still works.
Edit: Thanks for all suggestions, finally opened this one up :p

I like it. Would vote to merge if you can finish it with docs & al.

Because there is already an Enumerable#grep, I wonder if this would be better as DeepGrep? Since that's what it's adding.

LGTM, leaving final call(s) to @michaelherold, needs a README update, please

Hi @vmarutha, thank you for your contribution!
I've started to dive into this and I have a few things that I've found.

Could you please add an implementation of DeepGrep.deep_grep that you can use without mixing in the module? Our extensions follow that pattern so they are useful even if you do not wish to mix things in. You can see DeepLocate for an example of how this could work.
The current implementation seems too greedy. Consider the following Hash:

books = {
  "Uncategorized" => [
    { title: "Ruby for beginners", pages: 120 },
    { title: "CSS for intermediates", pages: 80 },
  ],
  "Collection of Elixir books" => [
    { title: "Programming Elixir 1.9", pages: 689 }
  ],
  "Collection of Ruby books" => [
    { title: "That other programming language by Matz", pages: 578 }
  ]
}

Given the way you wrote up the intent of the method, I would expect a #deep_grep /ruby/i to return the following:
[
  { title: "Ruby for beginners", pages: 120 },
  {
    "Collection of Ruby books" => [
      { title: "That other programming language by Matz", pages: 578 }
    ]
  }
]

I expect the first Hash in the result because "Ruby" is in the title of the book. I expect the second Hash because the key "Collection of Ruby Books" has the string "Ruby" in it, which would select that whole subtree of the parent Hash.
Instead, this example returns [books], which hardly seems helpful.

There are a few nits that I have that I will be happy to address later.

Does my understanding of the intent of the method mesh with your intention in (2)? Or am I off-base in my understanding?
If it would be helpful, I can attach a commit with some specs.

Hi @michaelherold,
Thanks for the feedback :)
So I tried that exact example, and I got two items as the results, one of which was books and the other was  {:title=>"Ruby for beginners", :pages=>120} as expected.
I agree that returning books is not very useful, and this seems to be because I am returning the container object when there is a match on either key or value. I think this can be attributed to using DeepLocate.
I noticed this about the implementation, and I guess it is a question on whether we would like to have the container object returned. It may be inelegant if the match is near the parent node of the hash (ex books)
But I think it would be effective for deeper nested hashes.
ex { title: "Ruby for beginners", pages: 120 } Notice how pages is returned here, which is apart of the container object.
What do you think about this?
Following your suggestion about not using mixins, I'm thinking of having DeepGrep very similar to DeepLocate, and to pretty much change this main line https://github.com/hashie/hashie/blob/master/lib/hashie/extensions/deep_locate.rb#L83 I was using DeepFind as inspiration initially, but I'm realizing I would need to have finer control of the returned object than DeepLocate has to offer.
Also maybe it is best to expose another function that is a more stricter grep, one without context.

Hi @vmarutha,
Sorry, it looks like you might have been waiting for feedback, which was my mistake. I'll answer your questions from your last message.

So I tried that exact example, and I got two items as the results, one of which was books and the other was {:title=>"Ruby for beginners", :pages=>120} as expected.

Hmm, I must have botched something in my setup then!

I think this can be attributed to using DeepLocate.

Yes, I think that's likely. I wonder if we can have an exit handler that prevents returning the original argument?

What do you think about this?

I think you're right. Ideally, we would prevent returning the outermost object but return any contained hash that matches. What do you think about that?

I'm thinking of having DeepGrep very similar to DeepLocate

Yes, that's exactly what I had in mind. :smile:

Also maybe it is best to expose another method that is a more stricter grep, one without returning the complete container object to try to address your main concern. I'd have to play around with this first.

That could be a good alternative to my suggestion above! I'd love to see how it feels.


How to deal with using non training data for inference, and the inference results are not realistic enough to restore

How to deal with using non training data for inference, and the inference results are not realistic enough to restore

Can you be more specific? Do you mean unseen speakers? Unseen samples? What kind of input that is not in training data?

I use two of my own audio files, hoping that A will imitate what B says. The output result obtained does not seem to be what A said, so it feels unreal


Update nginx to modern versions

Fixes https://github.com/kubernetes/website/issues/19195
I will do the same for Chinese and Japanese branches later

/assign @kbarnard10

@inductor, can you explain the steps to you've taken to verify that these changes are valid? For example, have you run some tests?
(if you have, then I'm willing to take your word on that)

@sftim Sorry for late reply. I confirmed!

Drawing on https://github.com/kubernetes/website/pull/19198#issuecomment-596154597:
/approve

It seems like we should have a way to automate these version / substitutions... I will bring this up at the weekly meeting. If no better solution comes around, this can be merged.
/approve

/lgtm


new question

Hi,
it would be great if we can have following question + the answer in this FYQ list:
"Can the same Gateway be used for environments in different tenants?"
Thank you

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c093e8d2-49bb-cf51-02a8-d7792913a5ad
Version Independent ID: 8fe1e76c-37e2-c6a8-4667-6578164d5b1e
Content: On-premises data gateway FAQ
Content Source: data-integration/gateway/service-gateway-onprem-faq.md
Product: on-premises-data-gateway
Technology: ****
GitHub Login: @arthiriyer
Microsoft Alias: arthii

Thanks @MichaelWeberMSFT for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. The requested updates have not been made since the creation of this issue, and the timeline for resolution may vary based on resourcing, so we've created an internal work item to incorporate your suggestions. We are closing this issue for now, but feel free to comment here as necessary. #please-close


feat(delete-user-data): firestore collection search and delete by User ID

Fixes
fixes: #773
Feature

[x]  add the option of query search and delete document in FIRESTORE_PATHS for a particular field containing User ID string using collection.field path.  e.g. posts.userId,docs.uid

Checklist

[x] add solution for query search created by @TylerAHolden explained in this comment
[x] add Firebase Emulator for testing purposes
[x] add tests with emulator

Problems

need a solution for the right admin.initalizeApp with Firebase Emulator
cannot force Firebase Storage Emulator to work correctly. It seems to be reaching some IAM resources online.

Draft manual test screens (will attach final after emulator tests)
Config

User Data

Succesful console log with no errors

Codecov Report

Merging #830 (70f538c) into next (649d6aa) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 70f538c differs from pull request most recent head 46798f9. Consider uploading reports for the commit 46798f9 to get more accurate results

@@           Coverage Diff           @@
##             next     #830   +/-   ##
=======================================
  Coverage   50.19%   50.19%
=======================================
  Files          30       30
  Lines        1329     1329
  Branches      197      197
=======================================
  Hits          667      667
  Misses        661      661
  Partials        1        1

@karayu what do you think about this feature?

Closing in favour of https://github.com/firebase/extensions/pull/1020