Document join reordering by the optimizer

Fixes #4330.
Summary of changes:

Update 'Cost-based optimizer' page with a new section 'Join
reordering', which discusses the new functionality, as well as
tradeoffs involved in its operation.  The new
experimental_reorder_joins_limit setting is described, along with
some warnings to the user about not setting it too high, since that
may lead to performance degradation.

Update the session variables page to include the
experimental_reorder_joins_limit variable.

This change is

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/98fffc974965e241c6d6f240e27c1e1cb3ec8461/

For convenience, direct link to the new join reordering section of the CBO page.
Specific questions:

Is the mention of "join associativity" too specific?  I read the PR which maybe I shouldn't have since I didn't "understand" it
Is it accurate to say the increase in the number of possible plans is exponential, as we do in this draft?
I made up a max number for the setting (12), but what should the max recommended value be?
Is there anything incorrect?
Is there anything missing that you think should be stated here?

On Feb 6, 2019, at 5:24 PM, Justin Jeffrey wrote:
@justinj approved this pull request.
I'm biased towards being less specific, but if you want to include the word "associativity" you should also include "commutativity" since they work together to explore everything.
I removed the mention of "associativity" then, since having both of these terms means we should probably get into what they mean, which is really handled better in your blog post(s), for now.  It would def blow up our word count here and be possibly confusing for non-join-experts.

It's actually worse, it follows the Catalan Numbers https://en.wikipedia.org/wiki/Catalan_number. But exponential is close enough for these purposes I think.
OK, leaving it at “exponential” then. 🙂
12 is super high, I would say going above 8 will begin to cause problems. Here's a rough graph I made showing the plan time for max depth+number of tables a couple weeks ago:

OK, will update to say the limit should be 8.
Related: should we add a line that says “Most users should not change this setting” ?   It might be good to discourage casually tweaking this. (Or not? What do you think?)

https://user-images.githubusercontent.com/409075/52377493-0c535f80-2a33-11e9-92f1-d951622e5bff.png
In v2.2/cost-based-optimizer.md https://github.com/cockroachdb/docs/pull/4332#discussion_r254475852:

+New in v2.2: The cost-based optimizer will explore additional join orderings in an attempt to find the lowest-cost execution plan for a query involving multiple joins. Specifically, it applies the rule of join associativity to generate even more possible join orderings to choose from, which can lead to significantly better performance in some cases.

+Because this process leads to an exponential increase in the number of possible execution plans for such queries, it's only used on queries with 4 or fewer joins by default, as determined by the experimental_reorder_joins_limit session variable.
+
+To change the number of joins that will be reordered by the optimizer, set the variable as shown below.
+
+{% include copy-clipboard.html %}
+~~~ sql
+> SET experimental_reorder_joins_limit = 8;
+~~~
+
+{{site.data.alerts.callout_danger}}
+We strongly recommend not setting this value higher than 12 to avoid performance degradation. If set too high, the cost of generating and costing execution plans can end up dominating the total execution time of the query.
+{{site.data.alerts.end}}
+
+For more information about the difficulty of selecting an optimal join ordering (spoiler: it's NP-complete), see our blog post An Introduction to Join Ordering.
I'm not actually sure it's NP-complete, it's definitely NP-hard.

Ah, typo on my part since I didn’t really know the difference!  Updated to say “NP-hard”, and made it a link to Wikipedia article on that topic for the curious (which I will now go read 🙂)

In v2.2/cost-based-optimizer.md https://github.com/cockroachdb/docs/pull/4332#discussion_r254476689:

@@ -143,6 +143,25 @@ The query plan cache is still under development and has the following limitation

If you use the query plan cache in conjunction with table statistics, cached plans do not yet get invalidated when new statistics are created.
{{site.data.alerts.end}}

+## Join reordering
+
+New in v2.2: The cost-based optimizer will explore additional join orderings in an attempt to find the lowest-cost execution plan for a query involving multiple joins. Specifically, it applies the rule of join associativity to generate even more possible join orderings to choose from, which can lead to significantly better performance in some cases.
+
+Because this process leads to an exponential increase in the number of possible execution plans for such queries, it's only used on queries with 4 or fewer joins by default, as determined by the experimental_reorder_joins_limit session variable.
It's not really "queries with 4 or fewer joins", it's "will reorder subtrees containing 4 or fewer joins." But I'm not sure if that's difficult to communicate here?

If it’s what this setting refers to, I think we must explain it - what does “subtree” mean in this context from a user’s POV?  Do I need to look at EXPLAIN output, or is it possible to determine by visually inspecting a SQL query?

The difference is that if a query has, say, 8 joins, the subtree it has containing 4 will be reordered.

I can definitely add this info once we explain subtrees (as discussed above).
Finally, I’ve pushed the changes I’ve made that incorporate your feedback so far.

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/524c67d7608f47956b3fdfec0584a1efc8e9b446/

Re: discussion above about describing "subtrees", I think that work can wait until after the current alpha release.  Leaving this as is for now for docs review.

I'd remove (spoiler: it's NP-hard).

Fixed.

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/a98524617d84ef727646d7aac22264e81cc9b5b4/


Capable to use TransiEnt in OpenModelica?

Hi, since in the description it is written:

Currently, only Dymola provides full suppport of TransiEnt. The development team has tested all models carefully using Dymola 2022x.

Can I use TransiEnt in OpenModelica? Will I get some problems?
Vielen Dank im Voraus!
Viele Grüße

Hi @wangzizhe,
unfortunately, the TransiEnt library is not yet compatible with OpenModelica, although we would like to work on that in the longterm. The TransiEnt library is based on the ClaRa library which is currently also not working with OM. There is a team working on the OM compatibility of the ClaRa library and the remaining issues will hopefully be solved soon. Once this is done, we will start working on the TransiEnt to make it OM compatible but we cannot give an estimation as to when it will be achieved, since we don’t know yet the extend of the changes that will need to be undertaken.

Is this still up-to-date?

Hi @JakeI,
the current version of the TransiEnt is based on the ClaRa 1.8.1, on the release note I saw that only since Clara 1.8.2 it supports most of model for OpenModelica. So I guess still until today TransiEnt is not compatible with OpenModelica.

Hi @wangzizhe
thank you for the clarification. That's precisely what I thought.
However, what are the future planes for TransiEnt as a project?

Is there any plan to upgrade to ClaRa 1.8.2?
If so, when? Given that ClaRa just changed its minor version, it sounds like an upgrade might be fairly quick and straight forward.
Would an upgrade of ClaRa make TransiEnt OpenModelica compatible? Or does any part of TransiEnt itself rely on Dymola specific features?

From my perspective, requiring a Dymola license is the one big issue with this library. That's why I'm very excited about potential OpenModelica compatibility.

Hi @JakeI,
there is a "OpenModelica Library Coverage Testing" repo. From the result (https://libraries.openmodelica.org/branches/overview.html) I can see that the coverage testing's result of ClaRa and TransiEnt is different.
(Ideally the number of Parsing, Frontend, Backend, SimCode, Templates, Compilation, Simulation, Verification should be the same and there is no yellow color in the result. It means that library is totally compatible with OpenModelica).
So I assume even though ClaRa becomes compatible with OM, there should be some modifications down to TransiEnt. But I'm not sure, I have the same question as you. Now I reopen this issue and wait for the development team's answer.
@annehage Could you please provide some information?
Thanks!


feat(scope): make Abstractions layer

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
Fixes #130  (issue)
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

[x] I have performed a self-review of my code
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have made corresponding changes to the documentation
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] New and existing unit tests pass locally with my changes

Hi @mo3in,
Thank you for your contribution 💐,
I was planning to create an abstraction layer and you made my life easier 🙌.
Soon I'll review your PR, meanwhile, it would be nice if you could also update the documentation,
(one small page about the abstraction layer)

Just one more thing I can't decide on yet is that typically, abstraction packages also include extension methods. However, in the current version, we don't have them. I'd like to know your opinion on this. Do you think relocating interfaces and DTO classes would be sufficient? 🤔

Yes, you are right, but I believe there are additional issues. Upon reviewing GridifyExtensions, it appears that nearly all the extensions have dependencies which prevent them from being moved to Abstractions. However, there are some exceptions, such as:

ApplyPaging
FixPagingData

On the other hand, the abstraction layer should enable developers to provide custom implementations for Gridify.
A question for me is how i can provide custom CursorPaginate (KeySetPaginate) functionality outside of the main Gridify package...
As of now, one approach could be to move DTOs/Interfaces to a project named Gridify.Contract, and await the final decision regarding the abstraction layer to better support customization.
Your assistance would be greatly appreciated...

One approach we can take for now is to create the Abstraction package with a minimum set of features but plan to refactor the underlying services/extensions for v3 to support different implementations, (obviously with some breaking changes). As you mentioned I also want to support CursorPagination soon, so I think some refactoring is needed for sure.
I believe having another Gridify.Contract package makes it more complicated.


Remove references to slsa/v2alpha1 in top level docs.

Changes

This effectively darklaunches it while we're still working on it. Docs on v2alpha1 still exist in pkg/chains/formats/slsa/README.md.
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[x] Has Docs included if any changes are user facing
[ ] Has Tests included if any functionality added or changed
[ ] Follows the commit message standard
[ ] Meets the Tekton contributor standards (including
functionality, content, code)
[ ] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
NONE

/cc @chitrangpatel

ping @lcarva


Follow 302 redirects when fetching /alerts.json

I use Karma with alertmanager.proxy: true and authentication/authorization handled by a reverse proxy (Cloudflare Access to be precise).
The issue is that the session tokens set by Cloudflare Access expire after a while, so when Karma tries to fetch from /alerts.json, it gets an HTTP 302 reply and stops there. If it followed the redirects instead, it would be able to renew its token (through the magic of SSO), and then it could fetch the alerts as if nothing had happened.
Does that make sense?

That should be easy to add, redirect: 'follow' is needed on fetch() calls as per https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch
Will look into that, thanks!

Raised #1158 that enforced redirects, but looks like that's already default for at least Google Chrome 47+ so I'm not 100% sure if that's enough

Hmm, interesting. I'm using Firefox, but I think a colleague had the same issue on Chrome 69; so maybe you're right and that's not enough.
In any case, thanks for the very quick reply, and more generally, thanks for this awesome tool!
I'll try and find the time to test your patch, see if it really fixes this issue.

Latest docker image should have this change if you wait 30-60 minutes.
I suspect that fetch requests are not triggering Access revalidation, but need to look a little more into it

If I go via Access and remove the auth cookie the next fetch errors with:
Access to fetch at 'XXX' from origin 'XXX' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

So I think the issue is that when Access gets into the middle of request it doesn't have same CORS headers (since it's not part of karma service, so lacks same setup). This can be fixed by passing mode: same-origin or mode: no-cors to fetch().
It feels that same-origin would be best here, given that karma UI & API are intended to be run as a single service on the same domain. I'll look a bit more into that, changing that is another trivial PR so it just needs a bit of testing.

You're too fast, I haven't even had the time to do a Docker pull :)
Good to know that there's a simple fix, I'm looking forward to it!

Well, there's a redirect involved going through Access, so it won't be same origin.
Need to test if no-cors really fixes it and whenever it doesn't break anything else.
This might be more tricky than I expected

I think the solution is to use no-cors for /alerts.json, as it's a simple GET and that will allow it to follow the redirect, but for anything that does A POST we need to use "full fetch" with CORS mode.

Another option is to set redirect: "manual" and if the response is type: "opaqueredirect" then we could reload the page to let use re-authenticate

#1159 splits fetch call so GET requests can use different options from POST or DELETE, hopefully this will play better with Access

Please try :latest docker image in 30-60 minutes

no-cors — JavaScript may not access any properties of the resulting Response
Missed that, with no-cors we won't get the body, which kinda makes the whole effort of making a request pointless.
That leaves us with  redirect: manual and reloading the page whenever we start getting 302 instead of 200 from fetch

We run into this issue as well. Instead of "Cloudflare Access" we have keycloak and an oauth2-proxy next to karma.
At the moment I'm manually reloading the page whenever the fetch fails.

I don't belive that there's anything we can do to refresh auth sessions while doing fetch (by following redirects or something else), especially that some auth implementations might ask for use input (input 2nd factor auth, choose the account to use, etc) so the best solution I can think of right now is:

have retries for failed fetch requests (don't wait the usual 30s or so but retry faster)
after a few consecutive failures reload the page

Will try to add that soon

I've added fetch retries, which will cover temporary issues, so now if we bubble a fetch error it's only if we tried 6 times (first + 5 retires). This should make it easier to add failure handling.
I'll try to experiment with popup windows - if we detect that fetches are failing we could open a browser popup window that will make a request and close, that should refresh the session, and if it asks for any user input it will happen without affecting main window. Not sure if that will work well since I browser might be blocking pop up windows that are open without user click.
If that doesn't work reliably implementing a full page reload is easy.

Popups seem to be blocked unless they're raised after user click, so don't think I can use that.
Raised #1285 that should help with this problem - it will add more retries to fetch and if it keeps failing it will eventually switch from mode: cors to mode: no-cors, which (at least with cf access) should prevent requests from being blocked at the early stage and allow to inspect the response.
The idea is that if cors requests are failing, but a non-cors goes through then there's something in between the browser and the backend, so we should reload the page.
This might need a few tweaks but seems like might work without too many false positives.

Please try v0.53 release that I've tagged recently. I did test it with Cloudflare Acces and it does correctly reload the page when I delete session cookies while karma is open.
Not sure if that will work for all auth proxies and/or brower so please report if it doesn't work. Thanks!

For keycloak it seems to work also. No manual reload anymore needed...
Thank you very much. You earn your Christmas gifts very well!

Sorry to drag up an old issue however im seeing a similar problem using Apereo CAS as the SSO identity provider and mod_auth_cas to provide authentication to the karma UI.  The error i see in the chrome console is
Access to fetch at 'https://alerts.wikimedia.org/alerts.json?&gridLabel=severity&gridSortReverse=0&sortOrder=&sortLabel=&sortReverse=&q=%40state%3Dactive&ticket=ST-136--Em-qym3ReaeP6u1QS48sh2Ks8o-idp2001' (redirected from 'https://alerts.wikimedia.org/alerts.json?&gridLabel=severity&gridSortReverse=0&sortOrder=&sortLabel=&sortReverse=&q=%40state%3Dactive') from origin 'https://alerts.wikimedia.org' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

From my own testing im still able to use the UI correctly and wonder if the error in the google chrome console is an expected artefact or if we need to configure something else?
and more details can be seen in our bug tracker

Yes, errors in console are expected. Here's what happens:
You open karma.example.com with some proxy service between your browser and karma itself
Auth proxy in between will redirect your browser on the / request to authenticate and issue some form of an token
If you leave browser running eventually that auth token expires
Next request for /alerts.json will be intercepted by proxy auth, instead of passing it to karma backend auth proxy will try to redirect it some auth page - this often breaks CORS since that auth page will likely be under different origin than karma.example.com (like sso.example.com) or is missing CORS headers, which means that the browser will block such request, /alerts.json requests will fail, plus it's a XHR request so it won't render that auth page in the browser anyway
The result is that karma will keep failing to fetch data from /alerts.json until you reload the page, which triggers in-browser redirect to the auth page again, the loop restarts
This is really poor UX you'll get from auth proxies, any web service routed via one will behave in a similar way.
To minimise the disruption karma tries to detect this situation and help those auth requests complete by disabling CORS headers after N number of failures. Disabling CORS headers means that browser won't block the request, it will get redirected to the auth proxy and there's a good chance it will trigger token refresh, which will then allow subsequent requests to work.
But since we first need to observe some failures to know that things ain't right and it might be time to try without CORS you will see some failing requests in the console.

Thank you for the super quick and detailed response


config.proto changes too frequently to move to the service config

And I don’t think the changes will settle. If a client adds a feature we didn’t anticipate, the gapic config will change again. We need to decide if merging the two configs is still a goal. If it is, we need the gapic config not to change when we add new features somehow.

I don't quite understand what the problem is. Changes can either be additive (new config entries) or breaking (deletes, renames, type changes). If we restricted ourselves to additive changes when adding new features, I don't think there would be a problem.

I was of the understanding that merging with the service config required more stability than that

@ethanbao do you have more insight on that?

I recommend that we merge the two configs once gapic config surface becomes stable, and will only have non-breaking changes.

What if the gapic config doesn't stabilize?

Then we need to figure out why it cannot be, even after the toolkit becomes GA quality.

Back to my point about different types of changes: We can decide at some point to make the config.proto surface "GA", since that we only do additive changes after that point. I think it's a separate question of the rate of change: we could still have a lot of additive changes even after we declare that there are no new breaking changes. So at which point do we merge: after we declare the config surface frozen, or after the rate of change slows down?

Closing because gapic config should not be merged with service config

Reopening because the issue of whether the gapic config and the service config should be merged is not resolved.

Is this question still relevant, @eoogbe , given the proto annotations?


push global RCT to groups if possible, parallelize otherwise

Makes global RCTs a bit faster (as e.g. used at fast modular speed settings like thunder and falcon).
Introduces a list of global transforms that are applied immediately after decoding each group instead of during finalization. For now only RCTs are moved from the global image to that list, could later add non-delta-palette transforms (channel palettes or simple color palettes) too. @veluca93 : this might be a start for a memory reduction by skipping the global image in case there are no global transforms left at that point.
In case there are still global RCTs left (e.g. when the global RCT is followed by a global squeeze or delta palette), use threads.
About 5% speedup of modular thunder decoding:
before: 4064 x 2704, geomean: 47.28 MP/s [38.06, 49.52], 30 reps, 4 threads.
after: 4064 x 2704, geomean: 49.72 MP/s [40.32, 52.68], 30 reps, 4 threads.

@jonsneyers : it seems that this PR causes benchmark pipeline failures: jxl:m:cheetah:P6 failed to decompress encoded image. Original source: /tmp/tmp.dvO3hD76wF/bench2.png and other errors

Please revert then for now, I will look at it on Monday.

The issue was with the case where first a channel palette is done, then an RCT. It would then push that RCT to the groups, which could be done in principle but it's a bit more complicated since the channel indices in the groups are just the usual 0-2 while in the full_image the RCT is done on channels 2-4 (it did channel palette on two channels, which adds two metachannels with the palettes, but in the groups those are not visible).
In general the channel indices will always be messed up with any non-RCT transform that gets done before the RCT, so for now I"ll just only do it when there's only a global RCT and no other global transforms.


nana::msgbox does not take wide char string

There are some inconsistency in the API call.
msgbox msg("訊息");
msg << L"儲存至test.png";
msg.show();

"The text is not encoded in UTF8: 訊息"
However, I cannot pass L"訊息" to the constructor.

msgbox msg(to_utf8(L"訊息"));
or
msgbox msg(u8"訊息");

yes, but other widgets like button do not need thee explicit conversion.

u8 prefix is preferred.  wstring version overloading is not necessary, it's just for convenience on Windows.


chore: bump to angular 4 final

Bumps the version to Angular 4 final. The release is pretty safe and doesn't need any code changes.

Does the @angular/material version needs to be bumped and released as well?

We're working on cutting a release as soon as possible.

Hi @jelbourn, is this a matter of hours/days or weeks?
Just that we know what to expect.
thanks!

Hopefully just a few days- we've done some work to update our package structure to match angular/angular, but we're still tuning it so that it will better support tree-shaking as it exists in Webpack.

Hi @jelbourn I came across the UNMET PEER DEPENDENCY at #1512 and I would like to know if there any advice, maybe use angular 2.0 instead of 4.0? or should I wait the next release?
PS: Keep doing great!


Codeunit 12 - "Gen. Jnl.-Post Line" - Change CalcLCYToAddCurr() to global

Could we change CalcLCYToAddCurr from local to global?

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Application Skin configuration not working.

Application skin configuration as written within config/admin.php :
/*
|--------------------------------------------------------------------------
| Application Skin
|--------------------------------------------------------------------------
|
| This value is the skin of admin pages.
| @see https://adminlte.io/docs/2.4/layout
|
| Supported:
|    "skin-blue", "skin-blue-light", "skin-yellow", "skin-yellow-light",
|    "skin-green", "skin-green-light", "skin-purple", "skin-purple-light",
|    "skin-red", "skin-red-light", "skin-black", "skin-black-light".
|
*/
'skin' => 'skin-blue-light',
this configuration is not working as I can't see any changes with the color in admin panel pages, while I'm changing the value with a different one as described within comment.
System

Open-admin version : latest as I tried with a fresh installation
PHP version : 8.1
Laravel Version 9.41.0
OS: Linux
Browser: Google Chrome, Firefox

Hi there, I had a look into this, i saw a line commented out but the class is being echo'd out in the view in the body tag, however i find its not changing the navbar and sidebar, so I'm gonna dig a bit deeper and see where I can assist and get everything to change together, also to the ones who started this project THANK YOU!!! I really liked encore/z-song and sad to see it no longer maintained, but this is amazing well played!!

@saptakchatterjee Thank for reporting
@NealRyanCruickshank Thank you.
This feature is indeed removed. Maybe in a later stage we'll add more theming options. You could overwrite style with the custom skin class.
Open is based on bootstrap5, ff you are interested in changing these styles you could take a look at
/vendor/open-admin-org/open-admin/resources/assets/open-admin/scss
These are compiled with sass via composer. You can run "composer sass" inside /vendor/open-admin-org/open-admin to create a new version of the css. Be aware you need to push you news styles to public root as well. Y
ou can use:
php artisan vendor:publish --provider="OpenAdmin\Admin\AdminServiceProvider"
To make development more easy i created myself a symbolic link between my asset folder in vendor dir and the vendor/open-admin dir in the public root.
Happy coding!


python27Packages.azure-mgmt-network: 8.0.0 -> 9.0.0

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/python2.7-azure-mgmt-network/versions.
meta.description for python27Packages.azure-mgmt-network is: '"Microsoft Azure SDK for Python"'.
meta.homepage for python27Packages.azure-mgmt-network is: '"https://github.com/Azure/azure-sdk-for-python"

Checks done (click to expand)

built on NixOS

Warning: a test defined in passthru.tests did not pass

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 9.0.0 with grep in /nix/store/2075yk04fcjmfiy2fjqql10dblh7hwjn-python2.7-azure-mgmt-network-9.0.0

directory tree listing: https://gist.github.com/42a7c14cf8f1b3108574e9ea7467dd98

du listing: https://gist.github.com/ce297fd5da67d9a96c1059fd9fa18031

Rebuild report (if merged into master) (click to expand)

20 total rebuild path(s)
5 package rebuild(s)
5 x86_64-linux rebuild(s)
5 i686-linux rebuild(s)
5 x86_64-darwin rebuild(s)
5 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
azure-cli
fluidasserts
python27Packages.azure-mgmt-network
python37Packages.azure-mgmt-network
python37Packages.fluidasserts

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/2075yk04fcjmfiy2fjqql10dblh7hwjn-python2.7-azure-mgmt-network-9.0.0 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python27Packages.azure-mgmt-network https://github.com/r-ryantm/nixpkgs/archive/007d5f1b193b45cffed19bbd0276d4540e0a9b85.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/2075yk04fcjmfiy2fjqql10dblh7hwjn-python2.7-azure-mgmt-network-9.0.0
ls -la /nix/store/2075yk04fcjmfiy2fjqql10dblh7hwjn-python2.7-azure-mgmt-network-9.0.0/bin

cc @olcai @mwilsoninsight for testing.

@kamadorueda Since fluidasserts has tests being ran, you could also remove the version constraints. The package should still fail if the api becomes invalidated, for example, the breaking changes in this release are:
    Operation ExpressRouteCircuitAuthorizationsOperations.create_or_update has a new signature
    Model ConnectionMonitorParameters has a new signature

which I highly doubt your using, seeing as your tool is cross-cloud

@jonringer working on it, had a bussy weak


read clipboard not work on safari

hi
i used safari open the demo https://charlesstover.github.io/use-clippy/
it can't read my clipboard , but chrome can do

I've heard mixed feedback on Safari support. More information, such as the specific Safari version, may be necessary to assist in debugging.
As a Windows developer, I am unable to test this hook in Safari, but I do welcome PRs to address this.

Hi there,
If I understand correctly, there was no support for the clipboard API on Safari.
Which is now supported:
https://webkit.org/blog/10855/
When I tested, it worked well so I think you can safely close this issue.


Functorise vertex header on page header

For now the dependency of the vertex header on the page header is implicit.

Long version : Store.Page.kind relies on the fact that the beginning of the header of any page (node or leaf) follows the same format -- and in particular contains a Kind field. This invariant is implicit, as Store.Page and Vertex_header separately implements a deserialisation protocol, which happens to be mutually consistent.


Spelling mistake in winstonLogger.js

System information

OS: Mac OS X Sierra
TypeScript Hero version: 1.7.0
VSCode version: 1.18.1

Description
Spelling mistake in "There was an uncought exception, do you want to see the logfile?"
uncought => uncaught

fixed in #338 I believe. I just don't think there's been a release since then.


rework the css for the tree of sub menu categories

This PR updates the css for the left sidebar menu:

reducing the font size to better accomodate the amount of text for the large index of contents
improves styling to increase distinction between parent/child categories - fixes #13
improves styling active and hover states to try and make things clearer

Screenshot:

A Note on Directories:
This task was fairly complex, as the nesting of subfolders has gotten pretty deep in the docs tree of contents. In some cases things are nested three or even four levels deep. Two sections in particular were rendering weirdly (I think the underlying Docsy theme is unable to handle that many layers of nesting) so I had to change two pages to not be inside additional directories:
content/docs/design_concepts/osm_components_interactions/_index.md
content/docs/install/manual_demo/_index.md

changed to:
content/docs/design_concepts/osm_components_interactions.md
content/docs/install/manual_demo.md

Signed-off-by: flynnduism<EMAIL_ADDRESS>

Awesome! Some notes...

Two sections in particular were rendering weirdly (I think the underlying Docsy theme is unable to handle that many layers of nesting) so I had to change two pages to not be inside additional directories:

This is actually great and I intend to do this to everything possible. This style of naming only was necessary when we were also trying to make the links work on github also.
Three things remain:

some merge conflicts on this PR, so we can't merge it quite yet

is there any way we could have space or horizontal rules or something in the sidebar? Looking at this, it is hard to tell if "High-level Software Architecture Components & Descriptions" is one link or two. Hovering over doesn't make it obvious to me either, in terms of visual changes:

could you maybe reclaim and use more space on the left sidebar if we didn't use this space on the right for a fairly unneeded right sidebar?

Thanks @bridgetkromhout

Rebased
I've updated the styling with more differentiation, horizontal breaks to denote sections, arrows to denote further children and more dramatic hover states
I think that would be a good improvement - two columns would be a lot less squishy. by default, the underlying Docsy framework is laid out like this, but it's certainly doable to change it to a two column layout. I will issue a seperate PR for that as it's a broader layout change from the text styling here.

Screenshots:

The current menu is humongous!


How to Clear Git Repository Activity Inside Visual Studio Code

Hello,
Always received a message popping up at the bottom of a screen 's right corner said that," Too many active git activities", so how to clear them out, thank you for helping out to solve this problem and thank you so much.

Hello,
The following languages are shown inside Visual Studio Code   and caused to unable to do code shortcut/reference selection automatically. Example: html 5 Boilerplate/html5 with i class shown on selection code list .CSS unable to function, neither. Thank you for helping me out to such a thing.

/gifPlease


Update named-views.md

Demo has superfluous

Opening issue instead of PR, because jsfiddle needs to be updated.


Onboarding Devtron over OSS Port

Added Devtron in the projects of OSS Port.

Thank you @Abhinav-26 for your great contribution!
I left some minor change requests. Could you please take a look?
Thank you in advance!

Hey, the suggested changes has been incorporated.

Looks perfect, thank you, @Abhinav-26!
Welcome to Port 🚢

Sounds Great 🎉,
Ready to sail the ocean of open source ⛴


Fix: Update alias name used in where

alternate the related table alias name causing the test to fail as it used the original name instead of the alias
 SortableTest::it_can_sort_by_has_one_relationship with data set #0 ('asc')

I am checking this now


"Create Deck" Button missing?

I can't find a "Create Deck" button... not sure if I simply don't see it or if I don't have the rights to create a Deck. Nevertheless, I guess in both cases it would be possible to improve the UI.
If I don't have the rights, it might be nice to see a deactivated button, since this would tell me that I simply don't have the permissions and I could stop searching for the button.

Currently we do not provide the user with a UI to create decks. This feature is on the roadmap though:
https://unikube.io/docs/roadmap/dynamic-deck-creation.html

This might be a high priority feature. I personally got completely stuck when I tried to create a Deck (obviously because its not possible yet). Nevertheless, I just had the expectation that after creating a Project, I would be able to create a Deck and get started. I guess many people who just want to try the tool could stop using it at this point. They would now have to got into the Docs or watch the explanation video to see how to process.
Maybe a temporary fix would be a comment: "SOPS missing", or a link to some explanation. Long-term, however, I would expect the "Create Deck" Button ;D

Maybe a temporary fix would be a comment: "SOPS missing", or a link to some explanation.

Well that really depends on the project here. Not every project needs SOPS, even it would be better if all projects made use of it. But maybe we can create some kind of detection which allows us to display the kind of hint your are writing about.
The creation of decks is kind of a complex thing, since you need to create some kind of understanding of the helm charts and k8s manifest to enable the use to kind of drag'n drop k8s resources and manifests into a Deck.

Maybe the issue is/was that I used an empty git repo to create the project. Now I filled it with some helm charts, but maybe I do something wrong, since I see no deck after syncing again. Maybe it would be sufficient to indicate that no deck could be extracted from the provided repo?

We just released a new version (1.3.0) for the frontend. It includes an indicator for the project status. This should at least help to find common problems with the project setup.

I close this for now. We open an issue as soon as we start working on the dynamic deck creation feature.


add Damian's session on 2023-05-12

Hey @DamianReeves, please have a look at this draft for your session on May 12th and let us know what you would like to change or add. Maybe some details about the analyzer you want to implement in the session?
Thanks.

This is great. I think I want to start with implementing an analyzer that doesn't allow types to be defined outside of a module. I'm interested in porting code from Elm to F#, and that sort of thing seems achievable in the session and also useful.


[Addition]: hotspots from Heltec Automation that only connect to each others

Hotspot b58 Addresses
11pqQLfuNSfgkEPCjS4t61SxxCKsncYjarkgCEVmQzuqju2q4ok
112Gft6GerEdFHhv9VVrpJDxYsBoYGtT94cp6TKT7ekdWwN7r77
112e7m3fH9H2PWNwtggBH28bVGsYN8EFH3EQnvYKdwRZWLfisvny
11VX1yF9BQEzLHjSY6cAggDjiiJJzqCQ2aU86tJfMdZUiY5LKbt
11KFAqz2T3iXfuXAi4zqxkiTAXt8gk1CtaqJFvhHTJpxvMFy3cw
11pqQLfuNSfgkEPCjS4t61SxxCKsncYjarkgCEVmQzuqju2q4ok
11VX1yF9BQEzLHjSY6cAggDjiiJJzqCQ2aU86tJfMdZUiY5LKbt
11KFAqz2T3iXfuXAi4zqxkiTAXt8gk1CtaqJFvhHTJpxvMFy3cw
11pqQLfuNSfgkEPCjS4t61SxxCKsncYjarkgCEVmQzuqju2q4ok
112Gft6GerEdFHhv9VVrpJDxYsBoYGtT94cp6TKT7ekdWwN7r77
Discord Handle
cybergogo#9171
Reason(s)
Hello, I saw this group of hotspots by Heltec Automation that only connect to each others and make big amounts of HNT. I've checked it on crowdspot and the graphs show abnormal behaviours: https://www.crowdspot.io/hotspots/112VbP17AF7YVATvJCCBHW2k12yKMgPCvJveAzQgjMb5svbm1n8C
Here are the concerned adresses in the b58 field.
Can you confirm it's illegitimate?
Thank you so much.
Best regards.

Based on
11pqQLfuNSfgkEPCjS4t61SxxCKsncYjarkgCEVmQzuqju2q4ok

High clustering co, same onboarding day.

Your evidence is obviously not enough to support. I looked at some machines next to these machines, which are more like cheating. At almost the same time, dozens of machines covered 20 kilometers. You know, this is impossible. Please look
113jjoMPn5HnQJ4jNFubJ7t8MhRbFVj7XM7GT6prBBKbNowRWw7

Man, these mining machines have practical applications, but there are some mining machines around them that have no practical applications and have high profits. How can they have such high profits? Very suspicious.

Man, these mining machines have practical applications, but there are some mining machines around them that have no practical applications and have high profits. How can they have such high profits? Very suspicious.
As follow：
14Zhr4tnfwLCKHDGAh2T4Ge3HjBmfxaxa29itjdFRKjMk28G5f1


Update data-serialization-formats.md

Public-Facing Changes

None
Description

Fixing a dead link to json-schema.org

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Crash when using randomname in application packed with PyInstaller

Overview
Operating System: Windows 11 64 bit
PyInstaller Version: 4.8
Python Version:  3.8.5
Version of randomname:
> pip show randomname
Name: randomname
Version: 0.1.5
Summary: Generate random adj-noun names like docker and github.
Home-page: https://github.com/beasteers/randomname
Author: Bea Steers
Author-email<EMAIL_ADDRESS>License: MIT License
Location: c:\users\sebastian\.conda\envs\editor\lib\site-packages
Requires: fire
Required-by:

Issue description
I have an application which uses randomname to create random file names.
The application is packaged into a .exe in Windows using PyInstaller.
When I run the application (as exe) I observe an application crash.
Steps to reproduce:
I have described the steps to reproduce the issue in this this git repository together with the source code, environment and pyinstaller packaging script.
What's the expected result?
Using the application I provided in this git repository I would expect the output to be something like this:
The name is natural-kitchen

If I run the file main.py using python like this
python main.py

I do in fact get the desired output.
What's the actual result?
When I run the application (as exe) I observe an application crash. The console output is:
  File "site-packages\randomname\core.py", line 13, in get_name
  File "site-packages\randomname\core.py", line 7, in generate
  File "site-packages\randomname\core.py", line 8, in <genexpr>
  File "site-packages\randomname\util.py", line 208, in choose
  File "random.py", line 290, in choice

Additional details / screenshot

Additional Info:
The stack trace at the very end reads:
File "random.py", line 290, in choice
IndexError: Cannot choose from an empty sequence

I suspect the names are read from files which are not included in the package when using PyInstaller.

Update:
commit 0c7ccb6656df668c43b0d258db835bbbe19b4fe9 and
52eebf8b0bbbd7e1141b077e7328cf3a281ea60b at this branch in my repo fixes the issue but has two disadvantages:

an extra step is needed in the package script
it doesn't work with the --onefile option of PyInstaller

This is rather a workaround than a solution.
Suggestion for fixing the issue
I would suggest adding the wordlists not as extra .txt files in the source tree but store the wordlists in .py source files as dictionaries. The dictionaries could be auto-generated from the .txt files in the wordlists folder by a script.


Graphviz groups not working on windows

@patrickkusebauch after a painfull rebase I realized the newly added "groups" feature seems to not work on windows, see
1) Tests\Qossmic\Deptrac\OutputFormatter\GraphVizOutputFormatterTest::testFinish
Failed asserting that two strings are equal.
 With #586 all tests are green on windows. Or is here a really bug in graphviz groups on windows?

Fixed by #1029


visionOS Compatibility

Expected behavior
The package should build with a visionOS target
Actual behavior
Some compiler errors are produced, due to some missing #if os()
Steps to reproduce

Open the package with Xcode 15 beta 3
Target a visionOS device (simulator for now)
Try to build

If possible, minimal yet complete reproducer code (or URL to code)
just swift-log code
SwiftLog version/commit hash
Tested with version 1.5.3
Swift & OS version (output of swift --version && uname -a)
swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-<IP_ADDRESS>.108 clang-15<IP_ADDRESS>)
Darwin  22.6.0 Darwin Kernel Version 22.6.0: Wed Jul  5 22:22:05 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6000 arm64

Already working thanks to commit 5996c7a. Not sure why I didn't have this fix in my dependencies


expose uni_app::now

as a standard way to get time whether you're on native of web target

Great, and would you mind to add a new method in World::now() to redirect this function ? As end user would not be able to use uni_app crate directly.

Nice, thanks


[alphanumeric_viewer] Keep terminal alive

ROS2 Version
humble
Version
main
What happened?
Should have a launcher, that launch node on a new gnome. When kill terminal where launch was launch, show close gnome terminal whith the node
Relevant log output
No response

Hi @javilinos, what is the status of this?


Add detailed logs for a received, transmitted, or dropped IPv6 message

This commit adds detailed logs (at info level) when an IPv6
message is received, transmitted or dropped (due to tx failures).
The log contains the following information about the IPv6 message:

The message length (in bytes),
The protocol (TCP/UDP/ICMP6, etc.),
Checksum value for UDP or TCP,
MAC address of source/dest (short or extended address),
Message security (enabled/disabled),
Message's source and destination IPv6 addresses,
In case of tx failure, error causing the drop (CCA, No Ack, ...).

@nibanks , can you please help me with this windows/appvoyer build error:
..\..\src\core\mac\mac_frame.cpp(56): error C3861: 'snprintf': identifier not found [C:\projects\openthread\etc\visual-studio\libopenthread_k.vcxproj]

Is there a trick to get snprint()? Thanks.

Looks like you are missing an include for #include <stdio.h>.

We have #include <stdio.h> in mac_frame.cpp here do we need it somewhere else?

Then you probably need to include the config file. Take a look at link_quality.cpp.

Codecov Report

Merging #1515 into master will increase coverage by 0.55%.
The diff coverage is 8.1%.

@@            Coverage Diff             @@
##           master    #1515      +/-   ##
==========================================
+ Coverage   69.28%   69.84%   +0.55%
==========================================
  Files         150      147       -3
  Lines       20009    20009
  Branches     2446     2437       -9
==========================================
+ Hits        13863    13975     +112
+ Misses       5230     5121     -109
+ Partials      916      913       -3

Impacted Files
Coverage Δ

src/core/thread/mesh_forwarder.hpp
0% <ø> (ø)
:arrow_up:

src/core/net/ip6_address.cpp
83.33% <ø> (ø)
:arrow_up:

src/core/net/ip6.hpp
80% <ø> (ø)
:arrow_up:

src/core/mac/mac_frame.hpp
66.66% <ø> (ø)
:arrow_up:

src/core/mac/mac_frame.cpp
85.71% <0%> (-2.1%)
:arrow_down:

src/core/net/ip6.cpp
65.89% <0%> (+9.45%)
:arrow_up:

src/core/thread/mesh_forwarder.cpp
74.37% <75%> (ø)
:arrow_up:

src/core/mac/mac.cpp
79.47% <0%> (-0.44%)
:arrow_down:

src/core/thread/mle_router.cpp
79.76% <0%> (-0.27%)
:arrow_down:

src/missing/strlcpy/strlcpy_test.c

... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 73564c3...593936f. Read the comment docs.

Ok. Thanks... Just pushed a new commit and added the config file as well...


Training adapters

Python library that allows

[ ] developing ML Model classes by implementing the interface
[ ] Mechanism of ML Model plugins – hooks that are invoked before and after prediction
[ ] Tool to pack python directory to the python library
[ ] Tool to install the python model as a python library

Check with D.S.


I2c optimization and features

Hi, i do some work on i2c.
Currently done:
Add custom delay to get true 400kHz at 160Mhz and  320KHz at 80Mhz ( cant be faster with 80 MHz for now )
Add status flag check ( usefull when more than one task use i2c ) ( Level 1 API dont start if bus busy)
Rewrote comment (Update them)
Rework printf debug
My plans (need opinion):
-Selectable frequency. (100kHz / 400kHz )
-Add force option. ( if bus busy, stop com and start a new one  for level1 API)
-Add possibility to write and read 16 bit data.
-Change data from i2c_slave_read(uint8_t slave_addr, uint8_t data, uint8_t *buf, uint32_t len) to a data pointer. Function dont send register address when NULL.
-Optional (if time permit): Add multiple i2c setting support.
What is better:
Add new functions to include 16 bits possibility or add  data format to current function ?
Write a new "read" function without register data or change him to a pointer  ?   (other idea ?)

I think using volatile variable for delay may produce different actual delay with different optimization levels. Even though it doesn't get any simpler than addi and s32i in a loop, compiler may still unroll the loop.
Something like this might produce more consistent results:
uint32_t delay;
__asm volatile (
	"loop: movi %0, %1" "\n"
	"addi %0, %0, -1" "\n"
	"bnez %0, loop" "\n"
: "=a" (delay) : "i" (5));

You can fine-tune the speed by putting nop instructions after the loop.

thank you for your help @resetnow , i had hesitate to use asm because i got problem when trying to replace fix value 5 by a variable :(.
I think, asm delay can be applied to ws2812 driver too for same reason?

i got problem when trying to replace fix value 5 by a variable :(.

That's because this snippet is made to explicitly accept immediate arguments. To avoid magic numbers, maybe it makes sense to make 5 a macro. Using a variable is also possible (the same movi instruction is now generated by the compiler) :
uint32_t delay = 5;
__asm volatile (
	"1: addi %0, %0, -1" "\n"
	"bnez %0, 1b" "\n"
: "=a" (delay));

asm delay can be applied to ws2812 driver too for same reason

Likely yes.

Update WIP:
-Added ASM delay.
-Added 100KHz, 400Khz or 500 KHz selection  ( measured ok but its Approximate)
-Added 16 bits data transfert I2C ( tested with ina3221 ok!)
Need feedback for these points:
Rework transfert:
I had just created new api function for 16bits to show  prototype format and know if it is good for all to update 8bits functions like them and for all lib. I think this allows more choice of transfer for the library.
Added Improvement of error handling ( EBUSY and EIO ) : make sens or keep old system ?
bool i2c_slave_write(uint8_t slave_addr, uint8_t *buf, uint8_t len);
bool i2c_slave_read(uint8_t slave_addr, uint8_t data, uint8_t *buf, uint32_t len);

to
int i2c_slave_write(uint8_t slave_addr, uint8_t *data, uint8_t *buf, uint8_t len);
int i2c_slave_read(uint8_t slave_addr, uint8_t *data, uint8_t *buf, uint32_t len);

Rework of flag system:
For test, i created a new flag variable, Level 0 API dont check it right now. reason: dont include transfert timing when i prevent task change here:
static int i2c_bus_test(bool force)
{
    taskENTER_CRITICAL(); // To prevent task swaping after checking flag and before set it!
    bool status = flag ; // get current status
    if(force)
    {
        flag = true ; // force bus on
        taskEXIT_CRITICAL();
        if(status)
           i2c_stop(); //Bus was busy, stop it.
    }
    else
    {
        if (status)
        {
            taskEXIT_CRITICAL();
            debug("busy");
            taskYIELD(); // If bus busy, change task to try finish last com.
            return -EBUSY ;  // If bus busy, dont read
        }
        else
        {
            flag = true ; // Set Bus busy
            taskEXIT_CRITICAL();
        }
    }
    return 0 ;
}

I use this prevent to be sure than task don't change between a check and a set.I I dont want include time from " i2c_stop()", that why i created new flag. Everything is ok ? idea to improve this ? or maybe it is useless ?
only 16 bits transfert got the busy protection, i will update everything after feedback.

I had doing some test with ina3221 example reworked.
One task read flag every 10 ms ;
Second task read value if flag good ;
No EIO error,  bus busy detected

I will remove 16 bits api after update 8 bits function. Is not really usefull.

The new api will look like this :
int i2c_slave_write(uint8_t slave_addr, uint8_t *data, uint8_t *buf, uint32_t len, bool force);
int i2c_slave_read(uint8_t slave_addr, uint8_t *data, uint8_t *buf, uint32_t len, bool force);

I think that all lib can be update with Level 1 API .  Level 0 API dont implement flag security. Even if we got i2c_status(), Level 1 API is more secure.

It is better that each lib do special treatment when I2C bus busy or that level 1 API  retry each time busy line detected ? Can be infinite loop if I2C bus permanently used by other task.

Usually multiple tasks share a semaphore that restricts access to a single task. I don't think this should be implemented inside the library.

hi, i have done few fixes and test.  I think core I2C work well now. Need tests for some library to confirm that changes on  i2c components lib is good. i will do some test again on i2c frequency. Already tested with example:
-ssd1306  ok
-ds3231 lok
-ina3221 ok
-bmp280 ok

I looked over this yesterday and here are my thoughts:

Do we really need to pass register address as a pointer? 90% of devices with i2c I worked follow the pattern slave_address — reg_address — data. For the few of those doing it differently the read/write sequence can be implemented separately.
Do we really need to pass force flag with every read/write? It seems to me that forcefully stopping existing communication isn't something you would want to do often.

Hi @resetnow , thank you for your feedback. Interesting thing.
1- I have choice to create one function because i was thinking that is not so much less used like you said. I use many sensors that need raw reads and some components that use register can be read without specify a register to be faster when reading a lot of data ( when start from 0 or when you know already where is the current register pointer  for example ). I'm not against the option of having two functions because I hesitated to do it. If it is better, i can do it.
2- you are right,  we will not use this often. In my case i want do some priority reading on a specific sensor with a timer.  Just thinking to add 1 function like
i2c_force_bus();  => This function will force the next transaction level 1 api. you need rearm it or
i2c_force_bus(bool state); => This function will force all i2c until he is setting off
What did you think ?

I'll wait for some feedback from others.

Ad 1) - I vote for having the pointer. Just make it (const uint_8*). Same for the write.

Is there some blocker regarding this? Or is it simply not going to get merged?

@sheinz @UncleRus what do you think?

I use this PR in my project. No problem detected with library i use (ds3231, ina3221, bmp280, ...). I have done some test with multi-task I2C without using semaphore. Reading and writing work well. Busy line properly detected.
I have done some test with forced option too. Didn't see problems. ( I mostly read device than write into them)
Some frequency bus need to be adjusted but it is fine.
I got not time right now but i was thinking to add bus system like SPI lib or Softuart and remove all I2C init from some library. It can be done in an other PR.

@resetnow For me, this PR is good enough.

Going to merge it if no objections.


DOC: add a note in poisson distribution when mu=0 and k=0 in pmf method

Reference issue

Fixes #12113
What does this implement/fix?

I have added a note in poisson distribution docs to warn that the pmf will return 1.0 at quantile k = 0. when rate = 0.

it says

returns 1.0 instead of the expected value 0.0.

why is 0. the expected value? if this behavior is intended and won't be broken in future, it can just say returns 1. No need to say anything about what was expected.

Thanks @andrewfowlie. Changed it!

I think this is helpful and should be merged

The fail in travis is the ppc that is happening frequently (on other PRs too) and is unrelated.
I can confirm the docstring change describes current behavior on master:
>>> from scipy.stats import poisson
>>> poisson.pmf(1, 0.)
0.0
>>> poisson.pmf(0, 0.)
1.0
>>> poisson.pmf(0.1, 0.)
0.0
>>> import scipy
>>> scipy.__version__
'1.6.0.dev0+c5e0ca5'
>>>
(scipy-dev) Lucass-MacBook:scipy rlucas$ git log -n 2 --oneline
c5e0ca5be (HEAD -> master, origin/master, origin/HEAD) MAINT: Cleanup unneeded void* cast in setlist.pxd
ec8763fe0 Merge pull request #12186 from pmla/spherical-voronoi-generalize-tests
(scipy-dev) Lucass-MacBook:scipy rlucas$

Thanks @tirthasheshpatel can you rebase this PR off master and remove the [MRG] part from the commit message as that isn't a convention we follow in SciPy?
Feel free to either ping me once complete or select the "request review" and request me here in the PR, and I'll merge.

@rlucas7  Seems like I messed up the rebase. Sorry! Should I open a new PR?

@rlucas7 Seems like I messed up the rebase. Sorry! Should I open a new PR?

No worries, sorry that happened. Yes, go ahead and open a new PR.
I used to think I understood why this happens but now I'm no longer sure, don't get discouraged by it, it happened to me several times while learning the git rebase process.

Thanks @rlucas7! Closing this in favor of #12486


Vector search returning suspect results

Describe the bug
I'm experimenting with vector similarity search in meilisearch and running into some odd behavior on a small test dataset. When I first added openai text-embedding-ada-002 (dimension size: 1536) embeddings as a _vectors field to a few documents, vector search seemed to be working fine (searching by document x's _vectors as vector against /search  would return document x with a similarity score of something like 0.996 or so).
However, after updating all documents in the store to include the _vectors, searching by the exact vector of document x returns 20 results that don't include vector x, and they somehow each have a similarity score slightly greater than 1.0:
    [0] => stdClass Object
        (
            [id] => 33
            [_semanticScore] => 1.0000012
        )

    [1] => stdClass Object
        (
            [id] => 32
            [_semanticScore] => 1.0000012
        )

    [2] => stdClass Object
        (
            [id] => 139
            [_semanticScore] => 1.0000012
        )

    [3] => stdClass Object
        (
            [id] => 106
            [_semanticScore] => 1.0000012
        )

    [4] => stdClass Object
        (
            [id] => 289
            [_semanticScore] => 1.0000012
        )

    [5] => stdClass Object
        (
            [id] => 3451
            [_semanticScore] => 1.0000012
        )

    [6] => stdClass Object
        (
            [id] => 8
            [_semanticScore] => 1.0000012
        )

    [7] => stdClass Object
        (
            [id] => 221
            [_semanticScore] => 1.0000012
        )

    [8] => stdClass Object
        (
            [id] => 433
            [_semanticScore] => 1.0000012
        )

    [9] => stdClass Object
        (
            [id] => 16
            [_semanticScore] => 1.0000012
        )

    [10] => stdClass Object
        (
            [id] => 49
            [_semanticScore] => 1.0000012
        )

    [11] => stdClass Object
        (
            [id] => 273
            [_semanticScore] => 1.0000012
        )

    [12] => stdClass Object
        (
            [id] => 200
            [_semanticScore] => 1.0000012
        )

    [13] => stdClass Object
        (
            [id] => 3411
            [_semanticScore] => 1.0000012
        )

    [14] => stdClass Object
        (
            [id] => 560
            [_semanticScore] => 1.0000012
        )

    [15] => stdClass Object
        (
            [id] => 241
            [_semanticScore] => 1.0000012
        )

    [16] => stdClass Object
        (
            [id] => 58
            [_semanticScore] => 1.0000012
        )

    [17] => stdClass Object
        (
            [id] => 528
            [_semanticScore] => 1.0000012
        )

    [18] => stdClass Object
        (
            [id] => 1447
            [_semanticScore] => 1.0000012
        )

    [19] => stdClass Object
        (
            [id] => 369
            [_semanticScore] => 1.0000005
        )

To Reproduce
Steps to reproduce the behavior:
Create an index with some long text fields, each unique with some shared words (the corpus was blog posts on our site), and use the openai  text-embedding-ada-002 embedding as the _vectors value when adding to the store.
After adding enough documents (~650), run a search against the exact vectors of the one of the documents.
Expected behavior
Searching with the vector embedings of one document in the index should return that document with highest semantic score (at or just below 1.0), followed by other documents that don't perfectly match. No document should return a semantic score greater than 1.0.
Screenshots
If applicable, add screenshots to help explain your problem.
Meilisearch version:
1.4.0 from git 705878ff5939a68bd9469afdb8a7f79669b2a06f
Additional context
I can share the entire database with you; it does not contain anything confidential.

Hi! I have not been able to reproduce your issue. Meilisearch v1.4 returns the desired document in the first place. However, it has returned it with a score of 1.0000001.
Could you confirm @Kerollmops if this is expected behavior?

I can share the database if you need help reproducing, but I think the semantic score greater than 1 already indicates that there is something wrong going on and I'd love to hear back from @Kerollmops or someone else on that.

Ok, there is definitely a bug:
First, a sanity check:
> http post <IP_ADDRESS>:7700/indexes/site_1_posts/documents/fetch filter="id = 5060"  fields:='["id", "title"]' | jq 'del(.vector)'
{
  "results": [
    {
      "title": "tcpproxy 0.4 released",
      "id": 5060
    }
  ],
  "offset": 0,
  "limit": 20,
  "total": 1
}

Store the vectors for id 5060 in vec_5060.json:
> http post <IP_ADDRESS>:7700/indexes/site_1_posts/documents/fetch filter="id = 5060"  fields:='["id", "title", "_vectors"]' | jq -r '.results[0]._vectors' > vec_5060.json

Search with the vectors from vec_5060.json; note that the results do NOT contain document 5060:
> http post <IP_ADDRESS>:7700/indexes/site_1_posts/search<EMAIL_ADDRESS>attributesToRetrieve:='["id"]' | jq
{
  "hits": [
    {
      "id": 480,
      "_semanticScore": 1.0000012
    },
    {
      "id": 571,
      "_semanticScore": 1.0000012
    },
    {
      "id": 230,
      "_semanticScore": 1.0000012
    },
    {
      "id": 2789,
      "_semanticScore": 1.0000012
    },
    {
      "id": 468,
      "_semanticScore": 1.0000012
    },
    {
      "id": 542,
      "_semanticScore": 1.0000012
    },
    {
      "id": 1011,
      "_semanticScore": 1.0000012
    },
    {
      "id": 360,
      "_semanticScore": 1.0000012
    },
    {
      "id": 4722,
      "_semanticScore": 1.0000012
    },
    {
      "id": 310,
      "_semanticScore": 1.0000012
    },
    {
      "id": 311,
      "_semanticScore": 1.0000012
    },
    {
      "id": 4940,
      "_semanticScore": 1.0000012
    },
    {
      "id": 402,
      "_semanticScore": 1.0000012
    },
    {
      "id": 23,
      "_semanticScore": 1.0000012
    },
    {
      "id": 1182,
      "_semanticScore": 1.0000012
    },
    {
      "id": 4034,
      "_semanticScore": 1.0000012
    },
    {
      "id": 4477,
      "_semanticScore": 1.0000012
    },
    {
      "id": 4609,
      "_semanticScore": 1.0000012
    },
    {
      "id": 3038,
      "_semanticScore": 1.0000012
    },
    {
      "id": 3920,
      "_semanticScore": 1.0000012
    }
  ],
  "query": "",
  "processingTimeMs": 24,
  "limit": 20,
  "offset": 0,
  "estimatedTotalHits": 641
}

Verify that the returned .vector from the /search request above matched our vec_5060.json:
> http post <IP_ADDRESS>:7700/indexes/site_1_posts/search<EMAIL_ADDRESS>attributesToRetrieve:='["id"]' | jq -r '.vector' > vec_search.json
> diff vec_5060.json vec_search.json && echo "matched"
matched

Ok, let's see why document 5060 wasn't included in the results by making the same /search but filtering on id to restrict it to only compare against id = 5060 to see what the semantic score was, to see why it didn't rank above the other documents:
> http post <IP_ADDRESS>:7700/indexes/site_1_posts/search<EMAIL_ADDRESS>filter="id = 5060" attributesToRetrieve:='["id"]' | jq 'del(.vector)'
{
  "hits": [],
  "query": "",
  "processingTimeMs": 13,
  "limit": 20,
  "offset": 0,
  "estimatedTotalHits": 1
}

Vector searching by vector 5060's own _vectors value against only that same document (via vector=... and filter="id = 5060") shows that post 5060 is not considered to be a match for its own vector at all!

Hey @mqudsi 👋
Thank you for diving into this and finding this issue about distance functions returning higher than 1.0 distances. This is kind of expected as we use a dot product as the distance function and not an Euclidean or an actual distance function that follows the Triangle Inequality. The reason is that it is recommended to use a dot product by most of the searchers.
About the last bug you found, I would like to take time and try producing it by using our internal dependency called Instant-Distance. By using a dot product, inserting the vector, and searching for the same vector, we must find the original one.

Thanks for replying, @Kerollmops. I think dot product scales better than euclidean search, right?

Indeed, that's the main reason, and it is explained somewhere in the FAISS documentation.

Can you confirm that the steps in my last post above exhibit incorrect or undesired behavior in terms of the match results?

If I am not missing something, I can't reproduce it as I don't have the original documents with the embeddings. Would it be possible to provide them to me? Even by email (in my bio)?
However, the documents you are seeing in your message are not all displayed here. Only the first twenty are shown. There are approximately 641 total results.
But you are right, searching for a vector that is exactly the same as one in the dataset should technically return the exact document. However, the main reason why you can't see it with a distance of zero is probably due to the HNSW library that is kind of randomly searching the nearest neighbors documents but I need to investigate more with your dataset.

I close this issue due to a lack of answers. Nothing is definitive of course, feel free to ask to re-open if needed 😊


Externally, internally, adjacently and un-tagged enums

All of those need to be properly tested, I don't think they are working currently. Behavior wrt. Value needs to be documented.
Also see: https://serde.rs/enum-representations.html

@torkleyy I just ran into this, a normal enum like this is serialized as ():
#[derive(Serialize, Deserialize)]
pub enum Foo {
	A,
	B,
	C,
}
Any idea when enums will be supported?

@Boscop enums in general are (of course!) supported.
I just tried to add your enum to the encode.rs example:
diff --git a/examples/encode.rs b/examples/encode.rs
index 3d3dc25..bf3b1e1 100644
--- a/examples/encode.rs
+++ b/examples/encode.rs
@@ -10,6 +10,7 @@ struct Config {
     nested: Nested,
     var: Variant,
     array: Vec<()>,
+    foo: Foo,
 }

 #[derive(Serialize)]
@@ -26,6 +27,13 @@ struct Nested {
     b: char,
 }

+#[derive(Serialize)]
+enum Foo {
+    A,
+    B,
+    C,
+}
+
 fn main() {
     let data = Config {
         float: (2.18, -1.1),
@@ -37,6 +45,7 @@ fn main() {
         },
         var: Variant::A(!0, ""),
         array: vec![(); 3],
+        foo: Foo::C,
     };

     let pretty = PrettyConfig::new()

And it serializes just fine:
(
    float: (2.18,-1.1,),
    tuple: ((),false,),
    map: {1:'2',3:'5',0:'1',8:'1',},
    nested: (a:"Hello from \"RON\"",b:'b',),
    var: A(255,"",),
    array: [(),(),(),],
    foo: C,
)

This issue is about funky custom-configured enums that Serde supports, not regular stuff.

@kvark Hm, but for me it serialized it as (), any idea why?

@Boscop you are likely talking about a different issue. Could you file one, provide the repro steps, and we'll go from there?

Might be related to this, but I've tried each of the 3 enum types.  Basically I have an array of an enum, and I'm trying to get something like this:
[
    DrawSprite( sprite_name: "blah", rect: Rect(x: 1.0, y: 2.0, w: 3.0, h: 4.0) ),
]

But it seems the best I can do is with 'external' serde enum's, to get:
[
    {"DrawSprite": ( sprite_name: "blah", rect: (x: 1.0, y: 2.0, w: 3.0, h: 4.0) )},
]

Is it planned to have a way to use the enum head names as the top level 'object' like in the first code-fence?

And for note, trying the prior example just has it report 1: 3:5: Expected map, I'm unsure why it needs a map for a single enum value per array cell though...

Is this supposed to be closed?

I don't think this is still an issue @OvermindDL1


comparing arrays (ALL() or ANY())

suggested by @urbanjost

Some ammo for a tweet from my old FAQ:
How do I compare arrays?
You cannot use a simple compare of two arrays in an IF(3f), because a
comparison of two arrays returns a logical array, not a single scalar
logical. So this IF(3f) statement will return a compiler error:
    integer :: A(3)=[1,2,3], B(3)=[1,2,3]
    write(*,*)A.eq.B  ! This returns an array
    if(A.eq.B)then    ! SO THIS WILL NOT WORK
       write(*,*) "A and B are equal"
    endif
    end

ANY(3f) and ALL(3f) are probably what you are looking for
There is not an specific intrinsic function to compare arrays in Fortran.
but you can use the very flexible and generic ALL(3f) and ANY(3f)
functions:
    integer :: A(3)=[1,2,3], B(3)=[1,2,3]
    write(*,*)A==B    ! Note this returns an array, not a scalar
    if(all(A.eq.B)) then
       write(*,*) "A and B are equal"
    else
       write(*,*) "A and B are NOT equal"
    endif

    write(*,*) all(A.eq.B)
    write(*,*) all(A.eq.B+2)

    end

Results:
    T T T
    A and B are equal
    T
    F

which works for all arrays as long as they have the same type
and length.
DO-ing it yourself
Of course, you can loop through the elements with a DO(3f):
    integer :: A(3)=[1,2,3], B(3)=[1,2,3]
    logical :: answer
    COMPARE: block
       integer :: i
       answer=.false.
       if(size(a).ne.size(b)) exit COMPARE
       do i=1,size(a)
          if(A(i).ne.B(i)) exit COMPARE
       enddo
       answer=.true.
    endblock COMPARE
    write(*,*)'equality of A and B is ',answer
    end

Results:
    equality of A and B is  T

Writing a function and returning .TRUE. or. .FALSE. is straight-forward,
but for each type of array there has to be another function or you
have to use CLASS(*).
As an example, an alternative lacking the generic character of ALL(3f)
or ANY(3f) is:
    integer :: A(3)=[1,2,3], B(3)=[1,2,3]

    if(equal(A,B))then
        write(*,*) "A and B are equal"
     else
        write(*,*) "A and B are NOT equal"
    endif

    contains
    pure logical function equal( array1, array2 )
    integer,dimension(:),intent(in) :: array1, array2
    integer                         :: i

    equal=size(array1)==size(array2)

    if(equal) then
       do i=1,size(array1)
          equal=array1(i) == array2(i)
          if(.not.equal)exit
       enddo
    endif

    end function equal

    end

Results:
    A and B are equal

Be careful when comparing floating-point values
If the arrays are INTEGER or CHARACTER, then the comparison can be exact.
However, if the arrays contain floating-points values such as  REAL,
DOUBLEPRECISION or COMPLEX variables, you should consider using a suitably
small tolerance when comparing values. For example:
    !real :: A(3)=[1.0,2.0,3.0], B(3)=[1.0,2.0,2.9999999999999]  ! this might test as equal
    real :: A(3)=[1.0,2.0,3.0], B(3)=[1.0,2.0,2.999999]         ! this should be close enough
    real :: tolerance=0.00001  ! just a sample tolerance

    if(all(A==B))then  ! testing for exact matches can be problematic
       write(*,*) "A and B are equal"
    elseif (all( abs(A - B) < tolerance) )then
       write(*,*) "A and B are close enough to equal"
    else
       write(*,*) "A and B are NOT equal"
    endif
    end

Most modern compilers do a good job at allowing programmers to compare
floating point values, but there are several good sources on why you want
to compare using a tolerance and how to determine what that tolerance
should be.
ANY(3f) and ALL(3f) may not be the most efficient method
The ANY(3f) and ALL(3f) functions may generate a logical array the size of
the input arrays or always test all elements; depending on how they are
implemented.  This could cause comparisons of large arrays to require a
significant amount of memory or do unneeded tests. The functions may or
may not take advantage of parallel or vector processing when available. So
if you are doing many array comparisons of very large arrays you might want
to create your own functions, but I suspect most ANY(3f) and ALL(3f) functions
will perform as well or better than your own routines.
If anyone has examples using Coarrays, OpenMP, or MPI that would be useful.
Timing information on various methods for large arrays would also be very
interesting. If I get the time I will try to add that.

Tweeted here and in following posts, thanks.


Navigation bar sub links possible?

Is it possible to have a sublink under the navigation bar? For example., clicking on Posts, shows a sub link?

Hy,
Not for the moment but it's a feature we can eventually work on (I will maybe need it for myself) @RatanShreshtha what do you think about it?

Sure @Guileas

Good to know. Any pointers on how to get it implemented?

@joesan not sure, I will try to look at it soon


Bookkeeping

The bookkeeping overhaul is complete and I've started on unrelated plotting tasks, so it's time for a PR!

I understood what was going on. The error was on my part. I am approving this since everything seems to be ok.


Client - add "Last seen"

If open client, would be good to know device is offline and since when (if > 30 seconds since advert)

[ ] Maybe add by storing the last-seen time (of advert) on each node.
[ ] then putting under an "i" info icon


Xcode 10.2 support for nullability

Problem description
With the migration to Xcode 10.2 and due to the fact that we treat all our warnings as errors, we're not able to compile our app anymore because Optimizely's SDK throws a lot of Pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) warnings that are treated as errors by Xcode.
Steps to reproduce

Download Xcode 10.2
Open a Swift project with OptimizelySDKiOS as a dependency
Build
Open the issues navigator
Check there are nullability warnings

Is it possible to have at least _Null_unspecified set in those properties?
Cheers

It seems this issue is solved when updating to the latest version of your pod, will close the issue for now and resolve the breaking changes on my side.


Can't sync execution client

Describe the bug
The execution client shows
execution-client  | 2023-01-28 11:49:18.8819|Waiting for peers... 58s
execution-client  | 2023-01-28 11:49:19.8805|Waiting for peers... 59s
execution-client  | 2023-01-28 11:49:20.5728|Request starting HTTP/1.1 POST http://execution:8552/ application/json 1972
execution-client  | 2023-01-28 11:49:20.5728|Received a new payload: 26190518 (0x7c64b79c530b37428ff25ef577d4c8a867ce9bc9ff18c0d0234e3b56e1f65a21)
execution-client  | 2023-01-28 11:49:20.5728|Syncing... Parent wasn't processed. Inserting block 26190518 (0x7c64b7...f65a21).
execution-client  | 2023-01-28 11:49:20.5728|Request finished HTTP/1.1 POST http://execution:8552/ application/json 1972 - 200 - application/json 1.0603ms
execution-client  | 2023-01-28 11:49:20.5728|Request starting HTTP/1.1 POST http://execution:8552/ application/json 59
execution-client  | 2023-01-28 11:49:20.5728|Executing JSON RPC call eth_syncing with params []
execution-client  | 2023-01-28 11:49:20.5728|Request finished HTTP/1.1 POST http://execution:8552/ application/json 59 - 200 - application/json 0.2847ms
execution-client  | 2023-01-28 11:49:20.5942|Request starting HTTP/1.1 POST http://execution:8552/ application/json 340
execution-client  | 2023-01-28 11:49:20.5942|Received: ForkchoiceState: (HeadBlockHash: 0x7c64b79c530b37428ff25ef577d4c8a867ce9bc9ff18c0d0234e3b56e1f65a21, SafeBlockHash: 0xa1a17b8bec26f83dad79afbd286884ffdd4e06624bd734cfdcb195fbb988855d, FinalizedBlockHash: 0xafbce30e2ef4af18451b65ad3b565d2affb807dd4cc6d7a12c38f7baa9b09390)
execution-client  | 2023-01-28 11:49:20.5942|Syncing beacon headers... Request: ForkchoiceState: (HeadBlockHash: 0x7c64b79c530b37428ff25ef577d4c8a867ce9bc9ff18c0d0234e3b56e1f65a21, SafeBlockHash: 0xa1a17b8bec26f83dad79afbd286884ffdd4e06624bd734cfdcb195fbb988855d, FinalizedBlockHash: 0xafbce30e2ef4af18451b65ad3b565d2affb807dd4cc6d7a12c38f7baa9b09390) .
execution-client  | 2023-01-28 11:49:20.5942|Request finished HTTP/1.1 POST http://execution:8552/ application/json 340 - 200 - application/json 0.7190ms
execution-client  | 2023-01-28 11:49:20.5942|Request starting HTTP/1.1 POST http://execution:8552/ application/json 59
execution-client  | 2023-01-28 11:49:20.5942|Executing JSON RPC call eth_syncing with params []
execution-client  | 2023-01-28 11:49:20.5942|Request finished HTTP/1.1 POST http://execution:8552/ application/json 59 - 200 - application/json 0.4291ms
execution-client  | 2023-01-28 11:49:20.8820|Waiting for peers... 0s
execution-client  | 2023-01-28 11:49:21.8821|Peers | with known best block: 0 | all: 0 |
execution-client  | 2023-01-28 11:49:21.8821|Waiting for peers... 1s
execution-client  | 2023-01-28 11:49:22.8806|Waiting for peers... 2s

and it has not synced in a few days. The execution_data folder is around 200MB
To Reproduce
I have the following env configuration
# --- Global configuration ---
EL_NETWORK=xdai
CL_NETWORK=gnosis
# --- Execution Layer - Execution Node - configuration ---
EC_IMAGE_VERSION=nethermind/nethermind:1.14.7
NETHERMIND_LOG_LEVEL=INFO
EC_ENABLED_MODULES=[Web3,Eth,Subscribe,Net,]
EC_NODENAME=Nethermind
NETHERMIND_METRICS_PUSH_GATEWAY_URL=http://localhost:9090/metrics
NETHERMIND_PRUNING_CACHEMB=2048
EC_DATA_DIR=./execution-data
EC_SNAP_SYNC_ENABLED=true
EC_JWT_SECRET_PATH=/media/vault/siedge/jwtsecret
# --- Consensus Layer - Beacon Node - configuration ---
CC_PEER_COUNT=50
CC_LOG_LEVEL=info
EC_API_URL=http://execution:8546
EC_AUTH_URL=http://execution:8552
CC_INSTANCE_NAME=Lighthouse
CC_IMAGE_VERSION=sigp/lighthouse:v3.3.0
CC_DATA_DIR=./consensus-data
CC_JWT_SECRET_PATH=/media/vault/siedge/jwtsecret
CL_FEE_RECIPIENT=REDACTED
CHECKPOINT_SYNC_URL=https://checkpoint.gnosischain.com
CL_BOOTNODES="REDACTED"
# --- Consensus Layer - Validator Node - configuration ---
CC_API_URL=http://consensus:4000
GRAFFITI=nethermind-lighthouse
VL_LOG_LEVEL=info
VL_INSTANCE_NAME=LighthouseValidator
VL_IMAGE_VERSION=sigp/lighthouse:v3.3.0
KEYSTORE_DIR=./keystore
VL_DATA_DIR=./validator-data

In case anyone needs it for me it was that the bootnodes were not configured by default and it was failing to connect to any node https://docs.gnosischain.com/updates/2022/12/08/temporary-bootnodes/


Recognize End Points for Cue Markers

This is a feature request or enhancement, but I'm wondering if it is possible for wavefile to recognize cue markers that have both a start AND end point—markers like this can be created in Adobe Audition, for example. Note the "start" and "end" times in this screenshot:

And here is a link to the file that the screenshot was taken with:
https://www.dropbox.com/transfer/AAAAACWjc7db0I3xtIeZ8l9CdhEH-kgKOCyCfrkJFS1nwrv-vYQ3Nu8
Would love to know your thoughts on if this is possible.

Thanks for the file!
The bug that corrupted files like this is solved in version 9.0.2 - the files can be saved and the regions are kept (they are showing up in Reaper). Some of the values are not displaying correctly, still.
This file uses some features that the current version of wavefile do not support, but im working to add support for them in future releases. I included the file in the test suite. Thanks!
Cheers!
Rafael

Version 9.1.1 should handle the regions correctly.
Cheers!
Rafael

Amazing! Read and write?

In version 9.x you can only read and write regions that already existed in the file. In v10.x will be possible to use the setCuePoint method to create both cue points and regions.
Cheers!

Version 10.0 is out; now you can add regions like this:
// create a cue region with a label:
wav.setCuePoint({position: 1500, end: 2500, label: 'some label'});

To create a standard cue point, ommit the 'end' attribute:
// create a cue point with a label:
wav.setCuePoint({position: 1500, label: 'some label'});
// create a cue point with no label:
wav.setCuePoint({position: 1500});

Cheers!
Rafael

This is awesome. I have no idea the magic that makes this script work, but I would be interested to know if it can parse files from Pro Tools, which writes "clips" into its files in a way that I have never been able to get another app to read. But that's a different issue for a different day. Thanks again!

Could you provide a file with some clips like that?
Cheers!
Rafael


Fix word wrapping in copy to clipboard

Should break long strings like that, but otherwise only break between words.
cc @dgutride

:tada: This issue has been resolved in version 2.6.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


🐞[Bug]: the login button in the footer is not working

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug

Expected behavior
it is taking to this page i think this happning because of the link is not proper
Add ScreenShots
No response
What browsers are you seeing the problem on?
Chrome
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

@varshith257 please assign me this issue


Client Allocation Request for: Ewesion

Client Allocation Questions
Core Information

Name: Ewesion
Website / Social Media: http://www.ewesion.com/
Region: Greater China Region
DataCap Requested: 100TiB
Addresses to be Notarized: f1a5qmgwasifom7lnvidz7gk6k2k2j6qmjta45apa
Notary Requested: UnionLabs2020

@Yiweishi-tec Please subscribe to notifications for this Issue to be aware of updates. Notaries may request additional information on the Issue.

This user’s identity has been verified through filplus.storage


Training becomes increasingly slower when creating models in a loop

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): NO
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
TensorFlow installed from (source or binary): BINARY
TensorFlow version (use command below): v2.4.0-49-g85c8b2a817f 2.4.1
Python version: 3.7.6
CUDA/cuDNN version: NO GPU
GPU model and memory: NO GPU

Describe the current behavior
When creating and training a model multiple times in a loop, the training process gets increasingly slower with each model.
Describe the expected behavior
I expected the training times to be roughly the same.
Standalone code to reproduce the issue
import numpy as np
import tensorflow as tf
import time

n_samples = 300000
n_features = 100
n_targets = 5
batch_size = 100
n_inducing_points = 100
x = np.array(range(n_samples * n_features), dtype=np.float64).reshape((n_samples, n_features))
y = np.array(range(n_samples * n_targets), dtype=np.float64).reshape((n_samples, n_targets))
for t_idx in range(10):
    tf.keras.backend.clear_session()
    dataset = [x, y]
    dataset = tf.data.Dataset.from_tensor_slices(tuple(dataset)).shuffle(n_samples).repeat().batch(batch_size=batch_size).prefetch(tf.data.experimental.AUTOTUNE)
    data_iterator = iter(dataset)

    inputs = tf.keras.Input(shape=(n_features,), name='input')
    outputs = tf.keras.layers.Dense(n_features, name='dense_1', activation=tf.keras.activations.relu)(inputs)
    outputs = tf.keras.layers.Dense(n_features, name='dense_2', activation=tf.keras.activations.relu)(outputs)
    outputs = tf.keras.layers.Dense(n_features, name='dense_3', activation=tf.keras.activations.relu)(outputs)
    outputs = tf.keras.layers.Dense(n_features, name='dense_4', activation=tf.keras.activations.relu)(outputs)
    outputs = tf.keras.layers.Dense(n_targets, name='output', activation=tf.keras.activations.linear)(outputs)
    model = tf.keras.Model(inputs=inputs, outputs=outputs)

    trainable_variables = list(model.trainable_variables)

    adam_opt = tf.optimizers.Adam(learning_rate=0.001)

    @tf.function
    def loss(batch):
        x_, y_ = batch
        y_pred_ = model(x_)
        return tf.keras.losses.MSE(y_pred_, y_)

    @tf.function
    def optimization_step():
        batch = next(data_iterator)
        def f(): return loss(batch)
        adam_opt.minimize(f, var_list=trainable_variables)

    iterations = 50000
    loop_start = time.time()
    optimization_times = []
    for idx in range(iterations):
        optimization_step()

    loop_end = time.time()
    print(f'Elapsed: {loop_end - loop_start}')

Using model.fit() is not an option, since this code was obtained by stripping down a much more complicated code, where the call to model.fit() is not possible.
Other info / logs
Output obtained with the script above:
Elapsed: 49.798316955566406
Elapsed: 55.18571472167969
Elapsed: 58.57510209083557
Elapsed: 64.41855955123901
Elapsed: 66.76858448982239
Elapsed: 68.3305652141571
Elapsed: 67.73438382148743
Elapsed: 69.73751258850098
Elapsed: 73.59102845191956
Elapsed: 73.14124798774719

As you can see, training times have a clear upward trend. Moreover, this does not happen on my Linux machine (same python version, same tensorflow version, on a clean virtualenv; Ubuntu 18.04).

@mattia-,
Could you please restart your machine, close all the programs running in the background before running the script and let us know if you are still facing the same issue? Thanks!

@amahendrakar,
Thanks for taking the time to reply!
I can confirm the issue still stands after a clean reboot. Can you reproduce the same behaviour on your machine or is it just me?

@rmothukuru,
I did not face any issues while running the code on a Linux machine with TF v2.3, TF v2.4 and TF-nightly. Please find the gist of it here. Thanks!

@rmothukuru,
Just to confirm that this looks like a Windows-only problem. I did not face any issues running my script on Linux either (and neither on Mac or on WSL, for that matter). Thanks!

@mattia- I ran your code on Win10 with tf-nightly and I don't see the issue of slowing down. Please check the output below
Elapsed: 51.30133318901062
Elapsed: 57.163376331329346
Elapsed: 57.61692833900452
Elapsed: 55.73898267745972
Elapsed: 58.43379831314087
Elapsed: 56.97764277458191
Elapsed: 57.768574237823486
Elapsed: 57.758583784103394
Elapsed: 58.231318950653076
Elapsed: 58.51157069206238

Process finished with exit code 0

Can you please verify with tf-nightly and let us know what you see. Thanks!

Hi @jvishnuvardhan
I tried to test what you suggested, but I'm having some problems with the use of tf-nightly:
Traceback (most recent call last):
File ".\tmp.py", line 16, in <module>
dataset = tf.data.Dataset.from_tensor_slices(tuple(dataset)).shuffle(n_samples).repeat().batch(batch_size=batch_size).prefetch(tf.data.experimental.AUTOTUNE)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 685, in from_tensor_slices
return TensorSliceDataset(tensors)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 3844, in __init__
element = structure.normalize_element(element)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\data\util\structure.py", line 129, in normalize_element
ops.convert_to_tensor(t, name="component_%d" % i, dtype=dtype))
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\profiler\trace.py", line 163, in wrapped
return func(*args, **kwargs)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\framework\ops.py", line 1566, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\framework\tensor_conversion_registry.py", line 52, in _default_conversion_function
return constant_op.constant(value, dtype, name=name)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\framework\constant_op.py", line 272, in constant
allow_broadcast=True)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\framework\constant_op.py", line 283, in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\framework\constant_op.py", line 308, in _constant_eager_impl
t = convert_to_eager_tensor(value, ctx, dtype)
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\framework\constant_op.py", line 105, in convert_to_eager_tensor
ctx.ensure_initialized()
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\eager\context.py", line 536, in ensure_initialized
config_str = self.config.SerializeToString()
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\eager\context.py", line 964, in config
self._initialize_physical_devices()
File "C:\Users\XXX\tmp\venv_tf_nightly\lib\site-packages\tensorflow\python\eager\context.py", line 1293, in _initialize_physical_devices
devs = pywrap_tfe.TF_ListPhysicalDevices()
tensorflow.python.framework.errors_impl.InternalError: Cannot dlopen all CUDA libraries.

tf-nightly was installed with
pip install tf-nightly

Any suggestions? Thanks!

@mattia- which version was installed? can you do print(tf.__version__) after import tensorflow as tf. thanks!

@jvishnuvardhan

which version was installed? can you do print(tf.__version__) after import tensorflow as tf

>>> import tensorflow as tf
>>> print(tf.__version__)
2.7.0-dev20210630

Another question. Are you trying to use TF with GPU?

Nope, I'm using (or, at least, trying to use) the plain CPU version.
Thanks!

@mattia- Do you have GPU in your system? can you try reinstalling TF as !pip install tf-nightly-cpu. If you just use !pip install tf-nightly, it looks for GPU. If you have GPU, then automatically it installs GPU enabled TF.
As far as I know, this is not an issue with TF code but something with the system. So, may be unistall and reinstall TF-cpu and test again. You could also check it in google colab. Thanks

@jvishnuvardhan
I managed to install it and test it. Here are the results on my machine:
>>> tf.__version__
'2.7.0-dev20210704'

Elapsed: 49.9630708694458
Elapsed: 53.02399563789368
Elapsed: 53.265995502471924
Elapsed: 53.51599383354187
Elapsed: 54.79951238632202
Elapsed: 55.34299564361572
Elapsed: 56.96799683570862
Elapsed: 57.81099009513855
Elapsed: 59.2099826335907
Elapsed: 59.92298483848572

It looks better than before, but to be honest I can still see the upwards trend in the training times.
Thanks!

My recent runtime (without any other programs running) show that there is a small increase but that is expected. Please check the runtime below
Elapsed: 54.97995686531067
Elapsed: 52.393901348114014
Elapsed: 53.56126165390015
Elapsed: 53.747889280319214
Elapsed: 54.081284284591675
Elapsed: 55.353312253952026
Elapsed: 56.64463949203491
Elapsed: 55.35281229019165
Elapsed: 55.846272230148315
Elapsed: 56.97290635108948


app crashes when adb is not connected

Godot version:
3.1 beta 10
OS/device including version:
X96 (S905x) tv box (android 6.0.1)
Mali 450 MP gpu
followed from #26424 but not related
beta 10 solved the compatibility issue but for some reason it only works when adb is connected
since it does works when i connect adb i can't provide a logcat

since it does works when i connect adb i can't provide a logcat

adb logcat gives you the system's log since the device last booted, so you should be able to make it crash without adb connected, then connect adb and use adb logcat to retrieve logs include that of said crash.

It's not clear what we can do for this issue, but I haven't seen other similar reports so it must have been quite specific.
I'll close for now. If you can still reproduce this issue in latest 3.2 builds, you can retrieve logs since the last boot with adb logcat, which should include any Godot crash that happened in between.


Does not recognize c++20 modules

I got a project in Sublime with CMake which produces compile_commands.json
basically main.cpp is main file, and foo.cpp is a module exported and imported and used in main.cpp
all compiles OK with GCC and LSP works but shows:
clang(module_not_found) on "import foo;" in main.cpp
here is the compile_commands.json
[
{
  "directory": "/home/r2com/projects/prog/cpp/build",
  "command": "/usr/bin/c++    -std=c++2b -fmodules-ts -std=c++23 -o CMakeFiles/hello.dir/main.cpp.o -c /home/r2com/projects/prog/cpp/src/main.cpp",
  "file": "/home/r2com/projects/prog/cpp/src/main.cpp"
},
{
  "directory": "/home/r2com/projects/prog/cpp/build",
  "command": "/usr/bin/c++    -std=c++2b -fmodules-ts -std=c++23 -o CMakeFiles/hello.dir/foo.cpp.o -c /home/r2com/projects/prog/cpp/src/foo.cpp",
  "file": "/home/r2com/projects/prog/cpp/src/foo.cpp"
}
]

and here is Cmake:
cmake_minimum_required(VERSION 3.25)
project(std_module_example CXX)

# Default to C++ extensions being off. Clang's modules support has trouble
# with extensions right now and it is not required for any other compiler
set(CMAKE_CXX_EXTENSIONS OFF)

# Set the C++ standard to C++20
set(CMAKE_CXX_STANDARD 23)

# Enable C++ modules by setting the appropriate compiler flag
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++2b")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fmodules-ts")

# Create the foo module
add_library(foo MODULE foo.cpp)

# Create an interface library for the foo module
add_library(foo_interface INTERFACE)
target_link_libraries(foo_interface INTERFACE foo)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
# Create the hello executable
add_executable(hello main.cpp foo.cpp)  # <-- Include foo.cpp here

# Link the foo_interface to the hello executable
target_link_libraries(hello PRIVATE foo_interface)

# Set the RPATH for the hello executable when building
set_target_properties(hello PROPERTIES
    BUILD_RPATH "$ORIGIN"
)

why LSP-Clangd cannot figure it out?

The plugin only integrates clangd, so you might get better help on the clangd github.
Maybe you have an older version of clangd installed? In this case that version would be preferred.

well, nah, clangd people say that modules are not supported yet. so bummer. time to get on Clion ;)


shellcheck fixable recommendation

Applied "fixable recommendation" from shellcheck:
shellcheck -f diff bashdot |git apply

@vincentqb Thanks for the contribution!


Fix nodemon reboots

Careen is exploding when the app reboots. The error is:
Unhandled rejection error: relation "category" already exists

It's like it's not keeping track of which migrations are being run, or something, because this shouldn't be an issue. I'll have to hop into the DB to see what's being stored.
This may be related to #16

As of #48, this project no longer reboots with nodemon.


prune backing and journal after commit block

prune backing and journal after commit block, keep latest 2 history block
clean unused crates

stable_holiday 也需要提交PR