IntelliSense for Django models is not aware of all expected attributes.

It seems like IntelliSense does not know about a default model manager - 'objects', and it is not aware of model exceptions such as 'DoesNotExist'.

It's not even aware of things like the model attributes, or anything to do with django models, really.

I've got the same problem, see screenshots. In runtime, the methods work like they should, no problems.
Diagnostic info: https://gist.github.com/volkit/a9db61f7f506c480026f89b734f8e68f

Any ideas?

Belongs to #3046


[CORE] Upgrade spark 34 version to 342

What changes were proposed in this pull request?
Upgrade spark 34 version to 342.
How was this patch tested?
Existing unit tests.

Does it impact UT?

Does it impact UT?

Yes.  The TPC-H and TPC-DS can passed. However the unit tests failed.

@FelixYBW @PHILO-HE Thanks for your review. I will merge this PR firstly and record the four failed https://github.com/oap-project/gluten/issues/3559.

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_4238_time.csv
log/native_master_01_03_2024_38de952fe_time.csv
difference
percentage

q1
35.78
32.78
-3.002
91.61%

q2
25.26
23.75
-1.516
94.00%

q3
38.00
37.47
-0.531
98.60%

q4
39.13
40.67
1.537
103.93%

q5
72.12
73.42
1.301
101.80%

q6
6.93
6.78
-0.153
97.79%

q7
87.89
85.99
-1.902
97.84%

q8
86.13
88.32
2.191
102.54%

q9
124.25
126.96
2.716
102.19%

q10
45.08
44.95
-0.128
99.72%

q11
20.48
19.97
-0.503
97.54%

q12
28.06
28.06
0.007
100.02%

q13
46.07
46.25
0.174
100.38%

q14
15.83
17.13
1.297
108.19%

q15
27.36
28.28
0.925
103.38%

q16
15.73
15.84
0.114
100.72%

q17
102.55
102.58
0.027
100.03%

q18
152.08
148.96
-3.118
97.95%

q19
12.69
14.29
1.602
112.63%

q20
27.37
28.11
0.738
102.70%

q21
224.84
232.01
7.167
103.19%

q22
14.22
13.97
-0.244
98.28%

total
1247.85
1256.55
8.699
100.70%


Remove unused language resources

So as to try and keep the amount of space of our builds to a minimum. Lingua uses a lot of space if we have all languages present.
This current build is 25mb while the previous build was over 150mb with all dependencies included.
Since Lingua is only to be used on migration of the projects to aid message templates, it should be dropped in the future. Hence, generating the Jar with the deleted language resources and having it as an unmanaged dependency for now on the engine should prove sufficient. It will avoid us having to deal with package deploy in Kotlin (currently working locally).
Again, this will cut build space by a very large margin (I believe building the engine with the original would add almost 300mb instead of the now 25mb).
Signed-off-by: Ruben Aguiar<EMAIL_ADDRESS>

Hello Rúben,
I just saw your obviously accidental pull request in my repository. Can I help you somehow? I don't quite understand what you are trying to achieve. You want to reduce disk space by removing all language models from the library? Are you aware that Lingua will not work anymore after doing this? The rule engine might still work but it will certainly crash when it tries to access the language models. Even if the library does not crash (what I seriously doubt), the detection accuracy will drop to nearly 0%.
Feel free to open an issue in my repository if you can need some help. Thanks.

Hello Rúben,
I just saw your obviously accidental pull request in my repository. Can I help you somehow? I don't quite understand what you are trying to achieve. You want to reduce disk space by removing all language models from the library? Are you aware that Lingua will not work anymore after doing this? The rule engine might still work but it will certainly crash when it tries to access the language models. Even if the library does not crash (what I seriously doubt), the detection accuracy will drop to nearly 0%.
Feel free to open an issue in my repository if you can need some help. Thanks.

Hello Peter,
First of all, apologies for spamming you, I erratically opened the PR with the wrong destination. Thank for your offer to help, it's truly appreciated.
As you guessed, I was trying to make some 'hackish' attempt to remove some languages and then test it to see if it would work 🙈 . The main idea was to remove unused languages in lingua since for our purposes we only need to detect 5 languages. The purpose is to reduce the space of our builds since we're starting to see some issues due to this.
Is this not possible at all or it would require more changes in the engine side to be working? I expected the engine to definitely crash if I tried to access the removed languages but hoped for the non removed ones to continue to work. At this point I haven't fully tested it.
Since this is a very particular use case tied to our use of languages I felt opening a PR on your side wouldn't really help anyone but ourselves hence our fork.
Thanks in advance for the help

Is this not possible at all or would it require more changes in the engine side to be working?

No, this is currently not possible. I have already thought about making each language a separate dependency so that people can decide themselves which languages they want to add. I'm just not sure yet how to approach this because I don't want to break existing code of the users of my library. Nevertheless, I will make each language optional some time in the future. I just can't tell you when exactly.
I think the library will still work as long as you don't try to access any of the removed languages. For this matter, you should also remove the respective languages from the Language enum. Otherwise, I'm afraid, the code will be buggy.

Thanks in advance for the help.

You are welcome. (-:


Updated GLOBUS_APP_URL

The old Globus app URL was deprecated and ending April 2019.  Updated to new URL.


Further protect against XSS and CSRF

Look into (adding or extra) protection against;

[ ] XSS: Cross-Site Scripting
[ ] CSRF: Cross-Site Request Forgery
More?

For starters see this excelent blog in dutch.

No longer relevant, closing now since repository is being archived.


Api client functions

These functions allow us to download questions, and their answers, from an API.
When run for the first time, it will download ALL the questions (about 6-7 thousand at the moment).  The data will be saved as new_questions.csv
When a file called archived_pqs.csv is present, these functions will attempt to download only the questions posed since the last question in that file.
What is lacking here, at present, is something that takes new_questions.csv and appends them to archived_questions.csv.  You may or may not feel that is part of this work.  I await your comments!
If you would like me to explain any of the code, I'll happily do that.
There are a few bits and bobs I'd like to add for completeness but perhaps this is OK for now..?

@andrewPoulton - you may also be interested to see this and I'd be interested in your feedback.

This is driving me a little mad.  Any insight into what might cause one of these two builds to fail, but not the other, would be greatly appreciated.  The problem appears to be in the way dependencies are managed but, I would have thought, that should effect both builds equally.

Looks good for my point of view.
With regards to the appending the new qs to the archived ones, I would have thought this should be part of this piece of work.

@RossKen you're right - I will add that in along with some tests for parsing and the changes Sam asked for (constituency and party).  You'll need to re-review afterwards but the stuff you've already seen shouldn't change much.

@RossKen – I've tweaked this so that the archive is updated at the end of fetching the questions.  There's also some new tests since you last saw this, some helpful (optional) feedback and... that might be it.
See what you think.
Tomorrow I will try and get to grips with the DataCreator to make sure the archive created here is compatible.  Let me know if you have any insight into that issue.

@RossKen - please could you have another quick look at this before I push the button and break everything... I mean merge! ;)

I think this is good to go


Forgot Password Link Not Working

Forgot Password Link Not Working
Issue:
When trying to reset the password through the "Forgot Password" feature, the link provided in the reset email is leading to a 404 Page Not Found error.
Steps to Reproduce:

Navigate to the Login page.
Click on the Forgot Password option.
Enter your registered email address.
Receive an email with a password reset link.
Click on the link in the email.
The page that loads shows a 404 Page Not Found error instead of the password reset page.

Expected Behavior:

Upon clicking the password reset link, the user should be redirected to a functional password reset page to create a new password.

Actual Behavior:

The user is instead redirected to a 404 Page Not Found error page.

Possible Causes:

The reset link might be broken or not pointing to the correct URL.
The backend might be misconfigured, leading to the missing reset page.
The route for the password reset page might not exist or might have been deleted.

Suggested Fix:

Ensure that the reset link generated in the email points to the correct URL.
Check the routing and server configuration for handling password resets.
Ensure the reset page exists and is properly accessible on the server.

Screenshot:

@anuj-pal27 please assign this issue to me .

@mohit-1710 assigned


Separate IN PR

Split the IN&CONCAT pr by deleting all CONCAT changes

Great, can you resolve the conflicts, and then we can merge it?


label: add text_align option for multi-line text

instead of messing with multiple labels with its' own alignments and shadow FBs, let pango do the job
there are more fancy stuff that could be added, like justify and ellipsize with explicit width and height, but I don't think it is relevant in context of a locker
or is it?

I think there is not
not in here https://docs.gtk.org/Pango/pango_markup.html
and not a single align word in src
I guess cause it is layout attr


Investigate sluggish local dev performance

Local dev performance has gotten dramatically slower, I dont think it used to be this bad. Something changed (maybe thirdweb upgrade?).
We should checkout older branches to try and pinpoint the time this issue propped up and see if we can mitigate with better next.js configs

Might be solved by https://github.com/Stand-With-Crypto/swc-web/issues/1293

When I try to use turbo on dev, I usually see this error when I try to save a file.

Error: Module [project]/node_modules/next/dist/esm/client/components/error-boundary.js [app-ssr] (ecmascript) was instantiated because it was required from module [project]/node_modules/next/dist/esm/build/templates/app-page.js?page=/[locale]/events/page { COMPONENT_0 => "[project]/src/app/layout.tsx [app-rsc] (ecmascript, Next.js server component)", COMPONENT_1 => "[project]/src/app/not-found.tsx [app-rsc] (ecmascript, Next.js server component)", COMPONENT_2 => "[project]/src/app/[locale]/layout.tsx [app-rsc] (ecmascript, Next.js server component) <facade>", COMPONENT_3 => "[project]/src/app/[locale]/error.tsx [app-rsc] (ecmascript, Next.js server component)", COMPONENT_4 => "[project]/src/app/[locale]/not-found.tsx [app-rsc] (ecmascript, Next.js server component)", COMPONENT_5 => "[project]/src/app/[locale]/events/page.tsx [app-rsc] (ecmascript, Next.js server component)" } [app-rsc] (ecmascript) <locals>, but the module factory is not available. It might have been deleted in an HMR update.

I tried updating code in a route that doesn't have localization and it worked perfectly. I updated code in /embedded/map and hot reload worked 100%
@travisbloom-cb @mhamendes @lucasrmp-cb

TODOs:
[ ] - Upgrade to V15 canary(with React 19) ignoring conflicts to verify that the error is fixed on V15 | IN PROGRESS
[ ] - If it is, create an upgrade plan to React 19 first and then an upgrade plan to Next V15>
[ ] - If it doesn't fix, try to find a solution online around it and patch the necessary modules for it to work
(use https://www.npmjs.com/package/patch-package)

Tried new attempts while investigating this problem:

Replaced the homepage with a new version containing only a div with some headings, also removed all layout.tsx code, middleware, sentry and so one. The objective on this was to restore each piece one by one to try to find which code or lib could be causing this issue. Result: Didn't make any difference. When I tried to change the headings text to check how long the fast refresh would take, it usually took 8/10 seconds to process.

Tried running the project in another computer, but the results were the same

Next action

Continue investigating NextJS issues on Github to check if something that could give us a hint about something that could be causing this issue.

After further investigation, I saw that creating issues on the NextJS repo about performance problems usually has replies. Vercel members help investigate the problem using the generated trace file inside .next.
Something that I saw that is very common is Vercel members asking the issue owner to update next to the latest version to make sure that the problem that they are facing is not fixed in the next version.
Because of that, I think that a clear next step before creating an issue would be to upgrade our project libs versions to make sure that we will have the latest version of everything. This will also benefit us when we upgrade to React 19 and NextJS 15.

As we are close to the election, we decided to postpone this upgrade work until after the election to prevent facing new bugs from the libs upgrade during this important time for our team.
C.C.: @travisbloom-cb

Adding scope to this:
Let's audit queries with lots of reads and a few number of indexes (excluding aggregations) and check if we can add indexes to any columns to increase performance.
Let's focus on queries we use on login here


Where exactly do the KBs live? Databases? not enough details

There aren't enough details about where the KBs live or how to import and manage KBs. Can they live in a database? In SharePoint? In a spreadsheet? Are they indexed somewhere? What's the process that refreshes them? I can't find this information.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: dce931c1-ae7e-14a4-707c-dd4b0daeaeaa
Version Independent ID: 451c94ba-5a34-23fa-5f12-ffea66944f70
Content: Manage knowledge bases - QnA Maker - Azure Cognitive Services
Content Source: articles/cognitive-services/QnAMaker/How-To/manage-knowledge-bases.md
Service: cognitive-services
Sub-service: qna-maker
GitHub Login: @nerajput1607
Microsoft Alias: nerajput

@gcsfred2 we'll review your feedback and get back to you shortly. Thanks!

Where the KBs live or how to import and manage KBs: Please review KB data sources https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/concepts/plan?tabs=v1#ingest-data-sources
Are they indexed somewhere? What's the process that refreshes them?: Please review the development lifecycle https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/concepts/development-lifecycle-knowledge-base. Furthermore, here's an approach to automate the process.

Hope this helps!


⚠ Bofrost.be has degraded performance

In 1efaecb, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1091 ms

Resolved: Bofrost.be performance has improved in 9e27f7b after 9 minutes.


Fix the svc name of webhook to avoid breaking istio

This addresses the issue https://github.com/kedacore/charts/issues/500
Only webhook's svc name is http, others are fine
Checklist

[x] I have verified that my change is according to the deprecations & breaking changes policy
[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)
[x] README is updated with new configuration values (if applicable) learn more
[x] A PR is opened to update KEDA core (repo) (if applicable, ie. when deployment manifests are modified)

Fixes #

❤


Long path issue hit in Insiders build

We were probably very close to it already with the Alpha release.
ping @egamma @jrieken

One idea is to use npm@3 on the build machines to try to reduce node module nesting.

tslint should have not been in there to begin with - I have updated vsce in the hope it doesn't contain dev-dependencies

tslint doesn't even seem to be a dev dependency as per Omnisharp/Omnisharp-vscode. Maybe clean your node_modules folder and run npm install from scratch?

yeah - it seems like vsce skips dev dependencies instead of taking only dependencies

Microsoft/vscode-vsce#58

Seems to be fixed.


Upload Image & File

Where is the upload function in C#?
https://getstream.io/docs/c_sharp/#files_upload

Any news on this?

Slipped a bit, we can add this week.

Hi ferhatelmas,
Please grant me access so I may push my branch. I have implemented Image.Upload
Chris

Is there any chance you could get me a -beta for this week? It would be appreciated

Thanks for the PR, I should be able to release it today.

Thanks so much!!

I still dont see it in nuget. Did you release?

I seen that you merged my Pull request. Thanks, Do you think you could get me a '-beta' deployment on nuget?

Can't wait to see that feature released. Thanks

Thank you @Christopherjl for adding this feature

I need this feature too thank you

Any news on this?

Delayed a bit, coming today.

Thanks for the info :-)

Do you think it will be out today?

Released as 4.1.1. Thanks for your patience and collaboration.


Append to ImplGenerics & TypeGenerics

(Similar to #732)
Consider the following trait and implementation:
trait Trait<'a> {}

struct Struct;

impl<'a> Trait<'a> for Struct {}

It would be very handy to be able to add to the ImplGenerics and TypeGenerics structs after they've been created. They could contain an extra field (something like extra_params: Option<Punctuated<GenericParam, Token![,]>>) to allow for an api like this:
// `generics` exists here, if it's mutated here then `impl_generics` and `ty_generics` both see the modifications

// would probably want this to be `Punctuated<GenericParam, Token![,]>`
let trait_params = [parse_quote!('q)];

let (mut impl_generics, ty_generics, where_clause) = generics.split_for_impl();

// add a lifetime required by the trait
// name is bikeshedding ofc
impl_generics.with_additional_generics(&trait_params);

// here, #impl_generics contains #trait_params
quote! {
    impl #impl_generics Trait<#trait_params> for #ident #ty_generics #where_clause {
        // etc
    }
}

Rather than needing to clone and do this:
let (_, ty_generics, where_clause) = generics.split_for_impl();

let (impl_generics, _, _) = generics.split_for_impl();

Note that #1092 asked for something similar, but that approach would require split_for_impl  to  take &mut self, which may or may not be desirable.

I think that use case is covered well enough by the approach described in the issue you linked.

While #732 explains how to extend the impl_generics how would one extend the ty_generics with a non ident type, e.g., a tuple (A, B, C).
gen.params.push_value(parse_quote!((A, B, C)));

Panics, because a tuple is not a valid generic param. But it is valid in the ty_generics position of an impl block.

One idea I had was to have a function to get generic_arguments() from a Generic, that would return GenericArguments.


Add option to insert blank page between pages in Multi-tool (#2194)

Description
The added functionality was to insert blank pages between existing pages. The following files were modified:

modified: src/main/resources/static/css/pdfActions.css
modified: src/main/resources/static/js/multitool/PdfActionsManager.js
modified: src/main/resources/static/js/multitool/PdfContainer.js
modified: src/main/resources/templates/multi-tool.html

File: PdfContainer.js

The function addFiles() was modified to accept a second boolean parameter to indicate whether the page should be blank.

A new function, addFilesBlank(), was also added. It generates a blank page and adds it to the PDF.

Another new function, addFilesBlankAll(), was added to insert blank pages between all existing pages at once.

File: PdfActionsManager.js

The function insertFileBlankButtonCallback() was added to allow the insertion of a blank page between specified pages, one at a time.

Other modifications included updates to the UI, where two buttons were added to enable this functionality.

Closes #(2194)
Checklist

[x] I have read the Contribution Guidelines
[x] I have performed a self-review of my own code
[x] I have attached images of the change if it is UI based
[ ] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[ ] I have read the section Add New Translation Tags (for new translation tags only)

Good work!


Ideas For a High-Performance System Design

Hey there,
I've recently done a bit of work on physics-related tasks and in the process spent some thoughts on how Farseer performance could be improved.
Performance Observations
First, some assumptions on certain performance aspects that hopefully are uncontroversial enough to agree on as a precondition:

The fastest way to process data is to iterate over an array of structs.
It tends to be faster to bulk-process batches of data in a tight loop than to process one item at a time.
Parallelization has a synchronization overhead, except where no synchronization is necessary.
Executing user code has a guard overhead, except where no context-specific guard is necessary.
There is a non-zero overhead to abstractions like List<T> vs. T[] when it comes to bulk-processing large amounts of data.

Potential Performance Sinks
Next, here is a list of Farseer system design decisions that potentially drag performance below the theoretical maximum:

Shape, Joint and Body are all classes stored in lists or similar data structures.
Users can subscribe to events that are invoked from the middle of the physics simulation.
User events are invoked callback style, item-by-item.
Parallelizing physics simulation internally (i.e. not by putting Farseer into its own thread) is complicated because every object may alter the state of every other.

Design Decision Draft for Maximizing Efficiency
Finally, I'll just throw in some rather radical ideas on how to restructure the overall system to address the above issues:

Make Body a struct and store all bodies in a World-global (potentially sparse) array.
Make Joint a struct (generalized) and store all joints in a World-global (potentially sparse) array.
Make Shape a struct (discriminated union) and store all shapes in a Body-local (dense) array.
Double-buffer all of the above data within World, so a physics step / update can strictly read-only from buffer A (current frame) and write-only to buffer B (next frame). This enables internal parallelization without synchronization using Parallel.For and similar.
Grant users direct access to all of the above World-global arrays so they can perform efficient bulk operations.
References between Body, Joint and Shape take the form of int-based handles that represent the access index in their world.
Instead of event callbacks, gather all occurrences (structs!) to "subscribed" events in an array over an update step and deliver them in bulk once the update step is finished. This enables users to do efficient batch-processing and frees Velcro from the need for state guards and their overhead. See issue #8.
User code that absolutely requires to be executed within the physics frame (such as collision filters, but be really strict here) is required to consist of a pure function that gets all required data read-only via parameter and returns a value.

As a side effect to improved efficiency, less overhead and internal parallel execution, the above changes could also make serialization (see issue #6) a lot easier. It would also make pooling (see issue #5) obsolete.
On the downside, this would require users to adopt a new API and potentially be more careful due to less safety options being around. There will be a bigger need for good documentation so users are primed for certain aspects of the system. This might also generate a large amount of issues that need to be resolved in order to adapt to the new design, but if any of these changes should make it into Velcro, now would likely be a better opportunity to tackle them than later.
I'm totally aware that the above are quite radical suggestions and that I don't have any idea about most of the internal structures and design decisions of Farseer, so please read it as a collection of ideas by an interested observer :)

Something like this could be awesome and may open up the possibility of first- and even second-order speedup (N10x or N100x) factors using multicore , SIMD,  or GPGPU, and maybe even future reentrant server-side execution for multi-player or cloud use.  It would be best to do this on the implementation side without changing existing Body, Shape, Fixture, Joint classes so much as to alienate existing Farseer clients.
For a maintainable component, Callbacks such as CollidesWith and other are not dependent on the nuances of being called from within a tight loop and that this previous frame->next frame state model sounds like a good approach.
The old adage is "Restrain from premature optimizing",  but this effort would not be like that as its going for big gains by reducing algorithmic complexity.
GPU acceleration using OpenCL.Net or similar as is done by the BulletPhysics engine which has very rigid and incredibly fast piling allowing for impressive building falling simulations.
With .net we value rapid prototyping, safe code, and steady APIs, and component software.   Open source has the tendency to branch and explode in versions unless it is contained.  I think for existential reasons Velcro physics should be positioned away from PhysX, Bullet in 2d constrained mode, Jitter-2d, and other physics engines.  BulletPhysics piling performance has been shown to be much higher with GPU acceleration and there is a server component, but the cost of integration of any different physics engine to an existing client is high and the inclusion of 3d increases the system complexity by a whole dimension.
From what I have seen, Piling of bullet objects performance in Farseer is not good, issues seen with only 20 items, on a phone, less.
Not using CCD and allowing sticking, tunneling to happen can make physics games buggy enough to not be ship-able.  As a physics engine client most of the heavy work I think I have to do is trying to work without CCD when possible, to flip the IsBullet.
As a heavy api user of Farseer, I would forward that Body and Joint and Fixture classes not be overly changed for this if practical.  The state could be copied from the vectors.  But since we did not upgrade to the latest very version of Farseer because the changes where not compelling enough vs the difficulty of merging,  a different data stucture or api would not be a major deterrent.   My team decided to put  DataContract persistence on the properties of Body that are not derivatives of other state so if body were a struct then that fits with this model.
If this is pursued I can produce some links of prior work if that might help.

Great ideas @ilexp
So far I've tried to be really close to Box2D and only provide some much-needed tools and APIs to make it easier for beginners to use a physics engine. For a clean Box2D, there used to be Box2DX. I will be going in another direction with Velcro Physics and your proposal sounds interesting.

structs are definitely faster than classes for many reasons. However, an engine that uses only structs is really hard to design. Velcro could definitely use structs a lot of places where it currently uses classes, simply due to the fact that they are read-only "data-holding" classes.
Bulk operations on large arrays of structs is fast when we talk about a warm cache, instruction prefetching and memory-copying; but it is not always the way to go. Cache coherency is a big factor and having small structs in the inner loops can greatly speed up things due to cache hits on the CPU.
Branch prediction is yet another big factor. Branches should be moved outside of inner loops as much as possible.
 The CPU can automatically detect prefetch if calculations can be done in parallel

In an unrelated project, I've done some research into RyuJIT64 as well as the CLR runtime and optimizations. Let's just say that I'm deeply troubled by my findings, as it seems even trivial optimizations are not performed. One example is int i = 64 % 8 is slower than int i = 64 & (8 - 1). Usually, for hard coded constants that are power of 2, we can do this trick. The compiler should do it automatically, but it doesn't! Well, at least not according to my benchmarks. I still need to dive down into the actual IL code and perhaps the machine instructions to see what is going on. I tried searching both corefx and coreclr on Github to see if anybody mentioned these issues, but it seems nobody cares.

@damian-666
I looked into using SIMD to speed up the calculations in the collision resolution code, as it does either 1, 2 or 3 points at the same time. The results were less than impressive on PC. Note that this is based on outdated SIMD instruction sets as I have an Intel 4th generation CPU - it might be very different on newer hardware due to larger registers.
I also tried to build a multi-threaded version of the engine which failed miserably. The engine restricts the number of points in polygons to 8, which keeps the workload low, so any optimization to the collision resolution code did not benefit in the inner loops. MT to the outer loops were also unsuccessful due read/write constraints. @ilexp's proposal to make a step read-only will help a lot with that though.
The CCD has always been very expensive as it substeps inside each step. This is really an algorithm problem rather than an optimization problem.
As for API backwards compatibility, I don't think I will have that with Velcro. Farseer Physics Engine is stable and if it does the job, you should stay with it. I might have to move shapes, fixtures etc. out of the API for safety reasons, and there is just no way I can keep it compatible with FPE's API while doing so.

This proposal with require a redesign of most of the engine. Louis over at his fork has done a great job at simplifying the engine to accommodate refactorings like this, so I will be copying some of his changes first.

In an unrelated project, I've done some research into RyuJIT64 as well as the CLR runtime and optimizations. Let's just say that I'm deeply troubled by my findings, as it seems even trivial optimizations are not performed.
[...]
I tried searching both corefx and coreclr on Github to see if anybody mentioned these issues, but it seems nobody cares.

There's the tiered JIT issue in coreclr. To me, it seems like the biggest blocker for more JIT optimizations is the requirement to be fast at compiling vs. fast at executing. A tiered JIT has the potential to work around this in a way. But in any case, that's kind of future / hypothetical stuff. The other part is extending C# to allow writing more efficient code in the first place.
Anyway - really glad you think some of the above ideas are worth pursuing. I'm currently a bit short on time, but I'll take an occasional look at this issue (and project) to see how things develop, or whether there's opportunity for me to chime in with an occasional comment. And from my side, as a long-term Farseer user, feel free to break as much as you want with Velcro as long as it's for the better. 👍

Seems like if significant  redesign  is going to be considered, I see that the batches that must be organised are not the same as existing Islands, which were originally designed to be done in parallel, according to its original designer Erin Catto.     But that does not give incredible single pile performance like bullet shows .
From what I gather the  key is to batch contacts that do not share a body as described here on slide 6
https://www.slideshare.net/takahiroharada/solver-34909157
There are other tricks to parallelizing in there.
That way no sync is needed since the position or velocity state of a body is not being pushed by another contact on a parallel thread,  so no race condition or lock is needed.
here they discuss the parallelizing issues around box2d / Farseer Islands , some of which may be relevant even in the many-batch-in-one-pile scheme:
http://box2d.org/forum/viewtopic.php?t=9350
Looks like the box2d-MT has the box2d islands but its not super-impressive on the tests.  Erin chimed in somewhere else on this that the Islands  would need to have separate piles for testing.
Using AABB in world coordinates from the last frame I am guessing.
The reference code in C++ I think is in Bullet by Coumans he has made incredible huge piles of boxes and very stiff joints and contacts for robot and AI researchers who might also be interested in 2d for certain problems.
If parallelizing with Parallel.For or something works then maybe GPU could  using OpenCL.Net or something could be looked at.  For 2d render without 3d shaders loaded, and for  2d physics the most GPUs would have quite alot of spare memory and could allow all the data to go into the GPU memory which is super important for that.   If all of the bodies data, the lists of vertices in Fixtures, the AABBs in world ( i guess from the last frame), the tree, and the positions and velocities fit into GPU memory  then they can be batched in there.
I know this is a tall order but parallel physics is almost standard as even $50 phones have 4 cores now and cheap AMD processors now use 6 or  12.
One must be wary of little-gain optimizations if they are not thread-safe or away from the functional programming methodology of passing in the parameters that might not be the same for all callers in a frame, as they may turn up as issues in making it threadsafe.  Not to sweat the little stuff if 4, 12,  or even 60-factor gains are possible

Very interesting read in relation to performance:
https://blogs.msdn.microsoft.com/dotnet/2017/06/29/performance-improvements-in-ryujit-in-net-core-and-net-framework/

its interesting but seems like bounds-checking type optimization is never
going to give the possible performance boosts of 2x, 3x, 8x  when
algorithmic complexity is pretty high and there is an opportunity for
parallel or batch processing.   given 4 core is on budget phones and xbox
one is 8 core, and now AMD is selling 8 core CPUs for 118$, isnt there any
sort of parallel processing that can be pursued without doing a massive
overhaul, in the CCD or the collision detection and response?
Although we havent had much improvement with Parallel.For in spots here and
there, My team had wild success with running ray casts ( uses broad and
narrow phase)  in parallel with farseer , with just one or two fixes in our
branch .  With .net it was easy to find the race condition as it was
crashing in a static variable which was a negligible optimization, and just
making that a member viable was all that was needed to make our
pseudocollier's ray casts just about 4 x faster on a quad.  I know that
collision and ccd is more complex that just collision detection.
according to Erin Cattos comment in one newsgroup, the existing box2d
islands were originally meant to be run in parallel and i believe iforce2d
gave it a try in c++ where threading conflicts are more mysterious to
diagnose.   In my experience a functional-like programming  ( passing in
the parameters you need) and elimination of static vars that are both read
and write is usually enough to fix the threading conflicts.
I am not sure that c# is great for xbox one or UWP development at the
moment since there are graphics performance issues, but allot of us,
including unity developers, own  allot of c# code and might want to put our
physics  games on xbox and not have 4 or even 6 idle cores
On Thu, Jun 29, 2017 at 3:39 PM, Ian Qvist<EMAIL_ADDRESS>wrote:

Very interesting read in relation to performance:
https://blogs.msdn.microsoft.com/dotnet/2017/06/29/performan
ce-improvements-in-ryujit-in-net-core-and-net-framework/
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/VelcroPhysics/VelcroPhysics/issues/29#issuecomment-312127642,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AP_LmQAIYnbOkFbD0it9-s2Am79_wJeXks5sJCefgaJpZM4NZqKu
.

this is a big job, and is certainly not a pullrequest.  Maybe we can establish a Git Bounty then fork this to try and assemble a pair of engineers who have done this before, to try and do this using a "try an work through the runtime issues" approach
A ton of work in the solid library ImageSharp has parrallel.For  here and they just run it in parallel then fix the exceptions found.  That is how my team did parallel Raycasts in Farseer.
C# is great for parallel for as it allows easy "try and see" approaches for batching.
i suggest this with all the naiveté mabye Ian can stop me but,  , before any massive rewrite, to just try it in the simplest case of two vertical piles of boxes ( box2d island),  and then one vertical pile with two threads (Bullet lib- type batch, or "sets of contacts that do not share a body")  collect the issues.
box2d islands means disparate piles, and in real game use this is not going to be as useful as the other batch of body states, those" sets of contacts that do not share a body"

Try a rough  approach on boxes piling in 2d with a simple 4 body test.   that will have piling issues to see if it runs without crashing.   make two arbitrary lists of bodies and run them in parallel.
piling will likely have also sorts of issues but its just to see if it doesnt crash.

sort the bodies by batches that dont share a body as in the Bullet lib and the paper.
From what I gather the key is to batch contacts that do not share a body as described here on slide 6
https://www.slideshare.net/takahiroharada/solver-34909157

This way no two threads are fighting over any  body state since no two batches share a body

break the collision handlers outside the loop.

And / or
I would recommend to copy the method by ecoumans used in bullet if there are some parallels.  Their c++ lib can pile boxes with orders of magnitude faster and does have a 2d mode.  But pure .net has all the deployment and development advantages Bullet will never have.
In parallel with this, try to run CCD in parallel, that is of equal or more importance.
Another thing is to research parallel techniques for the iterative constraint solver.    Again, this was all achieved in Bullet and with GPU, with  two orders magnitude faster piling in the case GPU is used, in the bullet lib.  Once CPU parallelisation is done, a GPU method is about memory management and might be explorer with Alea Parallel.For
if this is done then a LiquidFun SPH type extension to Velcro and water with boats can be explorer from there.  on 6 or 8 cores thats could be interesting and playable water.

I've had the multi-threading discussion more times than I care to count :)
First of all, It is certainly possible to multi-thread the engine in specific scenarios, but years of experiments have taught us that a general solution hurts the common case. The things that have made a difference was cache-coherency and modern CPU features such as prefetch and pipelining. Keep small structures in CPU cache and you are suddenly running 2x speed. If I add just a bit of complexity such as keeping track of locks and have state objects for threads, then performance drops.
The most common case for this engine is not large games with incredibly advanced physics - it is small to medium simulations with 10-500 objects, and if I add on multithreading to speed up scenarios where you have 5000 objects, then we slow down the 10-500 objects case by x2. Not a great trade off.
In the 5000 objects case, you can certainly find a hotspot like ray-casts and multi-thread it in order to juggle more. We just have to remember that more threads do not equal faster simulation, it equals more simultaneous objects. In my tests, I had a ~20% increase in throughput by multi-threading with 4 cores; a terrible waste of resources gone to locking, state copying etc.
However, if you create a world and put that into a thread by itself, you suddenly have a 100% increase in throughput, and that scales linearly! So in larger games that actually need threading, it is much easier to copy state manually between the worlds on the macro level, than trying to squeeze anything out on the micro level.
With regards to islands - I did try to multithread (MT) them but keeping track of state across islands again degraded performance for the common case, while increasing performance for large simulations. I ended up with a version with A LOT of compiler conditions in order to get around it, and I was about to make a "MT edition" and a single thread edition, but I figured most people would choose the MT and then disregard the engine due to its bad performance in PoCs.
That being said, I'm very interested in what people can come up with. I'd love a multi-threaded version of the engine :)

Thats an interesting idea running entirely separate world in parallels, it can help for some ideas such as the current concept of "Islands", which are not the same as the batches or islands used in a successful MT piling implementation.  Two piles  could be solved MT by putting the bodies all in separate farseer worlds as you mentioned and running the whole worlds in parallel.  But for a pile, it would all be in one island.
I wasn't proposing introducing any synchronization or locks, just something like presorting the workloads and batching them all without sync,  as proposed by the thread originator.    I haven't have much success with complex locks either, and  I think ilexp was  proposing batching and running it all in parallel without sync inside a loop.
From what i gather, Those batches are sets of constraints  that do not share a body as described here. https://www.slideshare.net/takahiroharada/solver-34909157
I got a 3.9x faster performance for 4 cores when i simply ran cay casts in parallel over sets of bodies big or small,  in a real game scenaro, to implement a wind effect with shadowing.   For raycasts, its an embarrassingly parallel case since the fixtures are organized in a tree which does not change inside the loop, but I know the contact and constraint solver is another matter.

Ray-casts has always been the low-hanging fruit since they are pretty much stateless. I'm worried that if it is multi-threaded, it will result in thread starvation or too much overhead due to crappy schedulers on mobile devices etc. It could definitely be configurable with low overhead, but as you said, the solvers are a completely different matter and much more interesting.

i looked deeper into the velcro code and have some promising things to try.  I have added info below on the the engine fix we put for parallel ray casts in for farseer client in section 1) ,   then a partial proposition for implementing parallel collision and constraint solving work in 2) .
The parallel raycast really is an obvious trouble-free 4x speedup we tested on quad MacOSX and PC that required on or two engine changes .  For the client code that calls the raycast,   the outer sync was just break the body list in to N sections and use  ThreadPool.QueueUserWorkItem, then WaitHandle.WaitAll(waitHandles) after all the raycasts for all batches were done;    using processorCount.
We didn't try Parallel.For but that is the new and simple to do it , and the way ImageSharp iterates nearly all of its batches of work.  For my ray code the speedup is obvious but for the constraint loops in Velco, benchmarking timers arount each place would be needed.  Then the power of managed .net is exploited  that any runtime issues are easy to find.
Now that i look at some of the heavy loops, with that i'm an optimist to the absurd,  I think we can do speed up Velcro incrementally and practially without any noticable tradeoffs or ifdefs,.  To realize  the full benefit of multicore ,  by using Parallel.For on lists of Bodies, we would next need to look at how Bullet sorts contacts and or constraints by those batches that do not share a body. Then, all the loops that apply changes to body.AngularVelocityInternal.  and body.LinearVelocityInternal, and the positions, will not be having conflicting write to the same body, since no two batches share a body.  This genius idea is described  by Harada Takahiro in many of his publications but he goes much further to the GPU methods.    I see that GPU in .net is something much harder to be deferred for future,  and would need this constraint and contact sorting step done as a prerequisite.   No #ifdef would be required for Parallel.For, but the BatchSize could be a runtime parameter for testing, setting to 1 means don't bother.   This sorting would be done on the bodyLists inside an Island.   As for Running the currept box2d Island concept in parallel , I do not recommend to even bother with that,  since with the Takahiro method, each island will be sped up and the cores will all be loaded.
1.To make raycast multithreaded there are several changes to the RayCast method implementation, and make it use the parameters input and less of the tree state.
A crash was caused byissue with using the stack as a shared member in DynamicTree.RayCast method.
For my team it was a one or two line change in older Farseer to allocate a new 256 item stack per Raycast,  . I see the similar problem in Velcro, latest version.
rather than do try to pull request and then merge/ commit i am setting you the three important changes, i am using my own branch not on GitHub.  but i'd be happy to commit them if someone will piont me to the unittest .
The change to _stack was to allocate a new stack as box2d  did.   Reusing this tiny 256-item stack in the tree really is negligible impact in the context of nearly any usecase.  This is an example of a small optimization making a big one crash.   Then i have shown a client can call  RayCasts in parallel, reusing a big or small  dynamictree.
the offending code is in dynamictreeboadphase   (and  other  Raycast implemenations ) line 341.
_raycastStack.Clear();    <---dont reuse a raycastStack member, remove it and make on on the stack.
a)  Stack stack = new Stack(256);  <-- use that as box2d did.
then    b)     RayCastInput subInput;
subInput.Point1 = input.Point1;
subInput.Point2 = input.Point2;
subInput.MaxFraction = maxFraction;
subInput.Callback = input.Callback;//dh added this to pass the shape callback without using member var.. (thread safe)
subInput.TestInside = input.TestInside;
                float value = callback(ref subInput, nodeId);

then c)  This might not crash but is faster and causes less GC
AABB segmentAABB;
//shadowplay Mod use stack instead of heap for parallelism and less garbage
{
Vector2 t = p1 + maxFraction*(p2 - p1);
Vector2.Min(ref p1, ref t, out segmentAABB.LowerBound);
Vector2.Max(ref p1, ref t, out segmentAABB.UpperBound);
}

For MT implementation inside Velco, BodyLists could be done in parallel if the bodies were batched, per Island, by those not sharing a body.  I dont have time now but i think Bullet does this type of sort  and probably very quickly

for example here is a heavy , expensive process that writes to body state, velocity and position.
internal void SolveTOI(ref TimeStep subStep)
//add benchmark code here..
for (int i = 0; i < BodyCount; ++i)   <--- after bodies are sorted to batches that do not share a body, then just try parallel.for here and hope for a
Parallel.Foreach batch...  do this instead..
{
for (int i = 0; i < batch.BodyCount; ++i)
}
SynchoniseFixtures is all very expensive and trying that in parallel could be worth a try.

I didn't read the entire conversation in detail since my last comment, but please be aware that replacing a for loop with a Parallel.For is not a trivial drop-in replacement.
If you're writing a game library, you need to be aware that one of the contexts in which it is used will be editor applications, which often have a special SynchronizationContext due to their strict requirements regarding message queue processing and blocking UI code. This means that Parallel.For and other TPL classes may actually process window messages while waiting for its threads to join / finish, which can lead to reentrancy problems in the editor application that is hosting this code, i.e. crashes and undefined behavior. There is nothing that the author of that editor can do to prevent this without also introducing potential other problems or severely increasing application complexity.
With this in mind and also regarding what @Genbox mentioned, it seems preferable to not do multithreading on a micro-level like individual for loops, but instead provide the facilities to allow users of the library to do multithreading on a macro-level, e.g. a defined API that allows to perform certain operations in a thread-safe way. Give users the tools to multi-thread if they need or care to instead.

Thanks for raising the possible issue of  "This means that Parallel.For and other TPL classes may actually process window messages"   .  I have an editor and we put the physics outside of the UI thread, not sure if my setup would seen issues like this .  So in my system  it is  graphics and , UI in one thread,  and physics on the other.  We used means of trying to load the other  cores that are normally there in a quadcore,  but sometimes CCD or piles  is the bottleneck.     Any code proposed could use parallel for and batching only if proposed Settings.MaxBatches was >1.. , or some boolean to prevent it. so clients using UI thread also for physics could just set that or it be the default.
ImageSharp is a very solid general-purpose .net imaging engine with many professional level contributors around the clock.  They use parallel.For throughout since raster processes are parallelizable , and I never saw mention that clients didn't want it or how to get around it.  But to shut it off clients can set paralleloptions.maxdegreeofparallelism to 1 i believe, it will be passed to the .net method parallel.For
Here is an example of a parallel.For inside a general purpose engine implementation: https://github.com/SixLabors/ImageSharp/blob/master/src/ImageSharp/Processing/Processors/Convolution/Convolution2DProcessor.cs
ImageSharp may well could be used in editors with UI, not sure if the window message issue would come up for that.  For my level editor, in my setup,  visible objects are drawn while physics updates bodies,  then the body positions are copied to the visible objects , then repeat.
btw batching as in Haradas paper  is really non- trivial, so im not sure if this will happen.  The batchin could be the existing "Islands" though, or maybe a parallel batching scheme testing the bodies in  new constraints to those in existing batches, all in parallel.   Again in MaxBatches was ==1 , then it would not need to incur this overhead

I used the TumblerTest with 400 bodies to stress the engine and see if it's possible to multithread some methods.  SolveVelocityConstrains() took most of the time, about 23.4% overall.
My approach was to lock the two Velocity structs with Interlocked.CompareExchange().
I found that  Parallel.For() had way too much overhead. Even the documentation says that you will see benefit only when your Parallel loop has an inner loop. In other words Parallel.For() can't be uses as an in-place replacement for a for loop. So I split the loop in batches and had Parallel.For() dispatching batches to a regular for-loop. At the end I replaced Parallel.For() with tasks.
The result was to drop SolveVelocityConstrains() down to ~7.6% (on a 4-core Athlon) and a design that can easily switch to regular for-loops at some body threshold. Still need to verify if it's 100% correct and maybe switch back to Parallel or use threads directly instead of Task.

SolvePositionConstrains() will be harder to optimize , mainly because it removes Contacts from the list.
I know that @Genbox plans to replace the List<> with a LinkedList<>,  so here's an interesting article about  LinkedLists and Multithreading:
http://www.drdobbs.com/parallel/linked-lists-are-like-so-last-century/232400466?pgno=2

I prefer lock-free algorithm myself by in this case I don't think it's possible.
What you describe is  the Graph coloring problem](en.wikipedia.org/wiki/Graph_coloring). You can either re-color a body whenever a new contact is created or use a greedy algorithm to recolor the entire graph on every step. Both approaches use a lot of CPU.
So my reasoning goes like this: At some point you will think about using multithreading to speed up this step as well. Then you will have to use some form of barriers or sync mechanism between bodies to make it work. Unfortunately you can't avoid it. So let's skip the extra step and just use barriers inside the existing algorithms.
In my test I tried to find if it's possible to sync threads on bodies with low overhead.  CompareExchange seems to work pretty well. (Initially I tried Increment/Decrement but that couldn't go anywhere..)
If you look at the benchmark you will see that the code runs x3 or more on a 4core cpu, which I think it's pretty good. Most of the lost pref is related to thread setup/sync and not the actual locks. If you have 4 or 8 threads and hundreds of bodies you will rarely get a thread blocked on a body.

Oh 3x performance for 4 cores is very compelling, if you have a github branch with this work it would be interesting especially to try on a pile of boxes all marked bullet, with the the threading in the Island.SolveTOI.   But see, the compelling thing about the "upfront batching"  is that it has been implemented that way and tested years ago for bullet3, albeit for GPU, and  there are a few loops that all could reuse the per frame body batches . Yes, the" graph coloring" batching is illustrated in the  https://www.slideshare.net/takahiroharada/solver-34909157 with chess pieces , i am still trying to download the paper.
As you note it should be , batching itself is a parallel process, one that Bullet does on the GPU also.  Here  might be the batching code if anyone can understand it, in openCL, its considers static bodys as readonly and so batches can share those.   this is just an idea to explore and might not be as practical on CPU as for GPU.
.https://github.com/bulletphysics/bullet3/blob/master/src/Bullet3OpenCL/RigidBody/kernels/batchingKernelsNew.cl
So ,while bullet3 does stack 100,000 bodies in amazing videos, its hard to deploy gpu code and just a simple multi - threaded stack solver of 2d "bulleted" boxes would be useful.  Even stacking 50 items with CCD can get bogged...

Hi all. Hope my dropping in on this conversation is okay.
I really appreciate @Genbox 's mention, back on May 14, of my C++ work with Box2D. Incidentally, I'm still working on it but I've shifted my work over to a new repository with a new project name of PlayRho in case anyone is interested in following it.
As for speed optimizations, so far the thing I've seen in my experimenting that's most impressed me is what happened when I compared using various different data structures vs. the existing Box2D structure for detecting when a contact already existed in code for finding new contacts. I got inspired to look into the Add Pair Stress Test Testbed demo where related sources in Erin's Box2D code had commented about considering using a hash table (see b2ContactManager::AddPair ) . What I saw suggested to me that using an array of pointers (std::vector) was more performant than the existing intrusive linked-list data structure usage and significantly more performant than the hash table (std::unordered_map). At least in regards to the demo. I don't know for certain why the array of pointers was fastest but presumably it was because it was cache friendlier at least for the 400 or so small bodies that get impacted in the demo.
In regards to multi-threading, I haven't seen any performance improvement with the multi-threading that I've tried so far, at least on a pyramid like stacking test. Given that a pyramid is all one island, I wouldn't expect solving separate islands in separate threads to speed up a test with just one island but it was discouraging that multi-threading the contact updating didn't help. Admittedly, I haven't played around with this as much as I think it needs and realize that there's probably ways I could optimize what I've done. OTOH, I'm worried that cache misses may have more to do with performance losses than what multi-threading can gain and so more of my focus has been on refactoring code towards becoming cache friendlier.
In any case, this continues to be a very interesting problem to me.

Hi @louis-langholtz - thanks for dropping by and writing about your experience. I suspect you are right regarding the cache coherence versus code running time.

.NET Core is soon going to have this: https://github.com/dotnet/csharplang/blob/master/proposals/csharp-7.2/readonly-ref.md
I remember back 10 years ago when they argued that Java/C's const parameters were too much for most developers and it should be something handled by the JIT compiler. Well, they seem to have reversed that decision completely and are now going with const parameters. In any case, I expect the compiler to be able to inline much more aggressively and do a lot more tricks for arithmetic reductions.

I finally manage to replace Tasks with a ThreadPool and a simple wait mechanism using Interlocked.Decrement() (I guess a Semaphore that sets the initial number of entries to the number of batches will work as well). ThreadPool was +20% faster than Tasks.
I run Island.SolveVelocityConstrains() on multiple threads. That's all contact pairs on a single island.

What was really disappointing was:

The number of Garbage generated by ThreadPool. It triggers a Generation1 GC every minute or more.
Garbage was the main reason I tried to move away from Tasks.
Differences between newer implementation of .Net framework(s). Portable, WinRT, UAP, all lack thread APIs or implement them differently from each other. (Thread, ThreadPool, even Interlocked)

I could write my own thread pool to solve #1 but that wont work on WinRT that lacks a Thread constructor.  So, my implementation now works only on Net 4.0 & Net 4.5 and decided to make it disabled by default,  until I find a common ground, wait and see where Core 2.0 is going and solve the Garbage problem.
I am sure C++ don't have any of those problems and can see a benefit from a similar implementation without compromises. @louis-langholtz here are my changes in case you want to give it a try https://github.com/tainicom/Aether.Physics2D/pull/24

@louis-langholtz

What I saw suggested to me that using an array of pointers (std::vector) was more performant than the existing intrusive linked-list data structure usage and significantly more performant than the hash table (std::unordered_map).

An Array is fast to iterate and Add but very slow at removing contacts. You can measure this with TumblerTest where contact pairs are destroyed and created.

A quick update on SIMD: https://github.com/dotnet/coreclr/pull/13576
It looks like we finally have a full implementation of platform-based instructions!

As far as SIMD hardware acceleration goes, would it be possible to use System.Numerics.Vectors? Already out there since 2015, no need to wait for cutting edge CLR / FX changes to hit mainstream .NET.

@ilexp sure, that's not really the news though :)
The Vector classes have several shortcomings that I've hit in trying to use them and they are far from a full implementation of the SIMD instruction set. One of the largest limitations with the current Vector based SIMD is that you are locked into using their Vector classes. Not we can augment our own Vector classes and keep backwards compatability or optimize without the overhead of copying into different structs.
I've implemented part of the engine solver using System.Numerics.Vector and ran some tests - the results were not really what I expected. Much of it had to do with boxing, conversions and memory copying. With this API, it does not seem to be an issue.


MailForwardingRulesToExternalDomains

This CSV report needs a lot of work.  The description column is cut off so all it displays is "If the message:', there is no field to tell whose rule/mailbox it is, the forwardto column includes users in the same domain (not external)...

I am seeing this same issue however I do have an answer to part of your issue. Where you state "the forwardto column includes users in the same domain (not external)...", this is half correct. What you are seeing here are forwards that were setup when your user(s) were on-prem. When a user is migrated to EXO, the LegacyExchangeDN is converted to an X500 address however the rules are not updated to reflect this during the migration. Anywhere you see the following: /o=Contoso/ou=Exchange Administrative Group in the .csv can essentially be ignored as in a hybrid environment (which you were in or are in) treats those addresses as external as they are essentially external to O365/EXO.
I am planning to dig into the PoSH script above in order to see if I can get the information you require as I require it as well.

Thanks, that makes sense since there were migrated years ago.  And the description column is actually still in there, just needed to tell Excel it was a text column and it started wrapping it correctly.  To get the mailbox name in there I just added an 'Identity,' field in the Select statement before Name which worked fine.

Not sure if this is a hybrid environment thing, but all of our rules that forward to internal users are using the X500 address, like [EX:/o=ExchangeLabs/ou=Exchange Administrative Group...., and they all show up in the report.
So I just changed the Where-Object {($_.ForwardTo -ne $null) -or ($_.ForwardAsAttachmentTo -ne $null) -or ($_.RedirectsTo -ne $null)}
to be
Where-Object {($_.ForwardTo -imatch 'SMTP:') -or ($_.ForwardAsAttachmentTo -imatch 'SMTP:') -or ($_.RedirectsTo -imatch 'SMTP:')}

And to fix the new lines in the Description field issue, you can make a calculated property in the Select, so this line:
$UserInboxRules += Get-InboxRule -Mailbox $User.UserPrincipalname | Select Name, Description, Enabled, Priority, ForwardTo, ForwardAsAttachmentTo, RedirectTo, DeleteMessage .....
Becomes:
$UserInboxRules += Get-InboxRule -Mailbox $User.UserPrincipalname | Select Name, @{Name="Rule";Expression ={$_.Description -Replace '\r+\s+', ' '}}, Enabled, Priority, ForwardTo, ForwardAsAttachmentTo, RedirectTo, DeleteMessage | Where...... etc


 MY EYES ARE BLEEDING!

Thanks for your consideration.

Ha! Happy to change it if you have a suggestion. It's @timehop yellow


Fix activated conda env vars from PythonApi (#15526)

release port for https://github.com/microsoft/vscode-jupyter/issues/15524

These issues are still present, and these issues seem to be causing slow connections to the jupyter kernel in conda, so I have to temporarily use a connection to a remote jupyter server instead of directly connecting to conda.


sendgrid transport fix

Sending mail via Sendgrid fails as sgTransport is initialized wrongly & self.mailer is not initialized as a function in lib/emailservice.js when using sendgrid mailer

fixed by https://github.com/bitpay/bitcore-wallet-service/pull/813
thanks.


Discover rate

Hi,
im using bleacon on a Raspberry Pi with following code:
var Bleacon = require('bleacon');
var uuid = 'e2c56db5dffb48d2b060d0f5a71096e0';
var major = 0; // 0 - 65535
var minor = 0; // 0 - 65535
var measuredPower = -59; // -128 - 127 (measured RSSI at 1 meter)
Bleacon.startAdvertising(uuid, major, minor, measuredPower);

And on my MacBook (Yosemite 10.10.5) following code to discover the beacon:
var noble = require('noble');
noble.on('discover', function(peripheral) {
  var d = new Date();
  console.log(d.toLocaleString() + '.' + d.getMilliseconds() + ' Found peripheral');
});
noble.startScanning([], true);

I expected to get a log about every 100ms, but I get 3-4 logs every 2-3 seconds.
Whats wrong here ?
Greetings Jan

@Lock-up nothing can really be done for this unfortunately. The OS X CoreBluetooth framework / blued doesn't pass on all advertisement events.
This was also discussed in https://github.com/sandeepmistry/noble/issues/138


Per Category Permission

Hello,
Could you add Per Category Permission ? :D
For example : The Category Pokémon, permission : decoheads.pokemon.
Thanks ! :D

Ah, not a bad idea! I can implement this easily. Should the icon not even show in the categories menu if you don't have access?

Yes, or, under the categories (and maybe) the heads, there is a variable : "Status".
When you have the permission : Status : Owned. Or Status : Not Permission if you don't have the permission. :)
Tell me if it's good.

Hmmm, a system for buying / earning the heads is also a good idea, but it's completely a different feature. Anyways, I'll get back to you with a beta-build that has per-category permissions soon.

Okay, perfect.
Yeah, I wanted to create an issue with this system. xD
Please add my head. ;( xD

Haha next goal: In-game head creator.
Maybe I'll put your head in the Misc. category as an easter egg :3

Yeah, so fantastic. :D
Yeah, pleeeaaase. xD

The ability to purchase heads will have to be a future feature, but per category permissions were added actually quite a while ago. (e1ca684f5a7375019cef4f1b58332ddffbfb7b2c, I forgot to mention it in here :p)
If you would still like the ability to "unlock" heads via money, then please open a separate issue. I'm going to close this issue.


self built executable throws error

I have set a a dev environment for osxphotos based on python 3.8.2.
Everything is working if I invoke cli.py
But if I build an executable with help of make_cli_exe.sh and try to run it I get the following error:
[84939] Error loading Python lib '/var/folders/jw/2t6k1dy963xd344v_xh9fr8w0000gn/T/_MEIF7TZXn/Python': dlopen: dlopen(/var/folders/jw/2t6k1dy963xd344v_xh9fr8w0000gn/T/_MEIF7TZXn/Python, 10): no suitable image found.  Did find:
/var/folders/jw/2t6k1dy963xd344v_xh9fr8w0000gn/T/_MEIF7TZXn/Python: code signature invalid for '/var/folders/jw/2t6k1dy963xd344v_xh9fr8w0000gn/T/_MEIF7TZXn/Python'

How did you install python? (For example, anaconda, home brew, directly from python.org)?
Are you using a virtual environment and if so, how did you create it?

Direct download from python.org. No virtual environment. (At least not intentionally set up by me.)

OK, I've not tested with that version (I use Anaconda).  I do recommend you learn about virtual environments because it'll save you lot of headache as you do more python development.  In summary, virtual environments create a separate copy of python and any needed libraries/packages for each project to avoid conflicts, for example, if one project requires a different version of a python package than another project.
I'm traveling and don't have access to my Mac so it'll be a week or two before I can look at this specific issue.  Ideally, the exe should build on the official python distro from python.org.  My primary intent with the single-file exe is to make it easy for people who don't have python installed to use.  It uses pyinstaller to bundle the python executable and all needed code and packages into a single self-extracting zip.  Because of this it can fairly slow to execute as the zip has to unzip each time you run it.  I will try to debug this though when I get back home.
Since you already have a python development environment, I recommend you install with python3 setup.py install which will create the osxphotos command line tool.  You can also install with pipx

Gave it a try with a virtual environment. But result remains the same.
Nevertheless, please close the issue. python3 setup.py install works fine.
I did not recognize that it also creates an executable.
Thought it was only for retrieving required dependencies.

I've verified that as of 0.42.29 with Python.org 3.9.5, pyinstaller builds without error. The requirements.txt and dev_requirements.txt are now streamlined and use latest package versions which seems to have solved the problem.


Fixed bug new release check

What was the bug?

What was the bug?

I think is just wrong "arithmetic" for versions... If we use Version class, greater/lower operators should work correctly

What was the bug?

For example solana installed release was 1.11.1 while latest was 1.10.29 so for how it was before the parsed number were 1111 and 11029 and so 1111 < 11029 returned True even if the version was older, now with the padding with the 0, all should be fine (11110 and 11029 and so 11110 < 11029 return False)

We could use Version class from Packaging

I didn't know that exists a library for this, we can also use it... even if know all should be fine I guess

Packaging

@AlessandroPaparella  is it in standard library? Btw if it's just two lines of code I prefer @Lu191 implementation.
We need at least one test case, I'll write it now


[Symfony\Component\Debug\Exception\FatalErrorException]  Call to a member function filter() on null

I am using L 5.1 version,
Following error comes when I fire "composer update" command
[Symfony\Component\Debug\Exception\FatalErrorException]
Call to a member function filter() on null
There is no any compiled.php file in vendor folder
Please suggest me about this.

The compiled.php file is located in bootstrap/cache.


Improper implementation of Light Contrast Theme

Type: Bug
The Light Contrast Theme is not being implemented properly.
 Unfortunately, it's hard for us to understand what issue you're seeing. Please help us out by providing a screen recording showing exactly what isn't working as expected. While we can work with most standard formats, .gif files are preferred as they are displayed inline on GitHub. You may find https://gifcap.dev helpful as a browser-based gif recording tool.
If the issue depends on keyboard input, you can help us by enabling screencast mode for the recording (Developer: Toggle Screencast Mode in the command palette). Lastly, please attach this file via the GitHub web interface as emailed responses will strip files out from the issue.
Happy coding!

Thanks for reporting this issue! Unfortunately, it's hard for us to understand what issue you're seeing. Please help us out by providing a screen recording showing exactly what isn't working as expected. While we can work with most standard formats, .gif files are preferred as they are displayed inline on GitHub. You may find https://gifcap.dev helpful as a browser-based gif recording tool.
If the issue depends on keyboard input, you can help us by enabling screencast mode for the recording (Developer: Toggle Screencast Mode in the command palette). Lastly, please attach this file via the GitHub web interface as emailed responses will strip files out from the issue.
Happy coding!

That's very strange, here's what I see:

Can you try with a fresh install of VS Code?

That's very strange, here's what I see:
Can you try with a fresh install of VS Code?

Sadly, uninstallation of VSCode from my device is disabled by my employer.
But is there any way of updating or resetting VSCode?

Please ask your employer to get your VS Code updated to the latest version. Then you can use a Command Palette command to use a temporary profile, which is like resetting.

Please ask your employer to get your VS Code updated to the latest version. Then you can use a Command Palette command to use a temporary profile, which is like resetting.

okay so i got it done but i still face the same issue

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


get_absolute_size should raise if it can't find anything

can't directly test PermissionError, but it's easy enough to see it will happen. Tested with FileNotFoundError, which exercises the same case (get_absolute_size returns 0 if the path doesn't exist).
closes #124

Nice, verified to error running pytest not using root!

I wanted to solve the problem of not failing in any way if this was setup without sufficient permissions. I think the best way of failing for this situation is a very hard failure making it show up as restarts of a container etc in k8s as compared to error/warning messages shown for the admin starting to suspect something is wrong and inspecting logs.
Do you think its better to fail with clear error messages, without crashing? I think that would be preferred if file system access is expected to be intermittent, but I figure its fine to crash if its just a race condition of some other system first having setup the folder.


Add an example for event handlers ?

Hi Thank you for this great library!
I couldn't find a way to do event handlers on markers.
Something like
  <Marker {latLng} on:click={() => {
    console.log("click")
  }}>

will not do anything.
I briefly looked through the code of the library and saw that you have an EventBridge specifically for this. This would tend to tell me event handlers would work, however, I couldn't find how. Maybe it would be good to have an event handler example in the doc ?
I'm happy to write documentation if you show me how to make it work.

Hi, have a look at: https://ngyewch.github.io/svelte-leaflet/event-handling
You'll have to add events={['click']}.
Try:
<Marker {latLng} events={['click']} on:click={() => { console.log("click") }}>

I can't believe, I didn't see that!
Thank you for taking the time!

I am trying to do this for the LeafletMap component. I am interested in load event which I don't seem to be able to configure.
<LeafletMap
    bind:this={leafletMap}
    bind:options={mapOptions}
    events={["load"]}
    on:load={() => {
      console.log("load");
    }}
  >
....
</LeafletMap>

Note that, I tested with other events like zoom, click etc. Those work just fine.

@shubhendumadhukar check this out https://github.com/ngyewch/svelte-leaflet/issues/31#issue-1946253693


Build with windows OS: TypeError: path must be a string & Icon issue. (V2.2.0)

When I build the nw app with v2.2.0(v2.1.0 works fine) there is a "TypeError: path must be a string" issue occurs, the build configuration is as following:
var NwBuilder = require('nw-builder');
var nw = new NwBuilder({
files: './src/*', // use the glob format
platforms: ['osx32', 'osx64', 'win32', 'win64'],
version: "0.12.3",
appName: "landpy",
appVersion: "<IP_ADDRESS>",
buildType: "versioned",
winIco: "launch.ico"
});
//Log stuff you want
nw.on('log', console.log);
// Build returns a promise
nw.build().then(function () {
console.log('Done!');
}).catch(function (error) {
console.error('Error: '+error);
});
And I found that the "winIco" value of options is only effective with small icon, if you change to "Medium Ico"+, the exe file will show the NW build-in icon.

The ico issue is not the NW.js issue. When I resize the the ico sometimes it display as nw.js defualt icon. So I think when I generate the ico I should generate all the size ico, maybe I miss some size.

I also faced this error when building by v2.2.0. Didn't try with v2.1.0 yet.

Joining the issue.
build.js code:
var NwBuilder = require('nw-builder');
var nw = new NwBuilder({
    files: './app-source/**/**', // use the glob format
    platforms: ['osx32', 'osx64', 'win32', 'win64'],
    version: '0.12.3'
});

//Log stuff you want

nw.on('log',  console.log);

// Build returns a promise
nw.build().then(function () {
   console.log('all done!');
}).catch(function (error) {
    console.error(error);
});

@dariuszsikorski thank you mate, your solution helped me to solve same problem. i just added zip: false to NwBuilder settings and it works now.

On the Mac this does not solve the issue. I am using zip: false but still receive an error saying
WARNING!!
macZip is deprecated. Use the zip option instead.

in addition to the
Fatal error: path must be a string

Falling back to<EMAIL_ADDRESS>helps ...

Fixed today. Please try the latest version. Thanks.


Add a schedule trigger for MITx Online deploys to production

We are ready to start deploying Open edX for MITx Online on a weekly basis on Wednesdays. We need to add a schedule trigger to the production stage of the relevant pipelines. This includes the edx-platform instance, the MFEs being used (learn and gradebook at the moment), and Forum.

@blarghmatey is this still relevant, given that we want the Wednesday deployment to production to be discretionary, based on the MITx EdTech teams's input?


missing algeria flag

Missing algeria flag

Hi @muminkahraman 👋 This issue is known in our flagpack-core repo under #40. This will be part of our next release.


Stateful Service life-cycle isn't the same as described in documentation.

Good day,
This issue has the same origin as microsoft/service-fabric-issues#1257
Description
In the documentation there is a statement:

Stateful services have a similar pattern to stateless services, with a few changes. For starting up a stateful service, the order of events is as follows:

The service is constructed.
StatefulServiceBase.OnOpenAsync() is called. This call is not commonly overridden in the service.
The following things happen in parallel:

StatefulServiceBase.CreateServiceReplicaListeners() is invoked.

If the service is a Primary service, all returned listeners are opened. ICommunicationListener.OpenAsync() is called on each listener.
If the service is a Secondary service, only those listeners marked as ListenOnSecondary = true are opened. Having listeners that are open on secondaries is less common.

If the service is currently a Primary, the service's StatefulServiceBase.RunAsync() method is called.

After all the replica listener's OpenAsync() calls finish and RunAsync() is called, StatefulServiceBase.OnChangeRoleAsync() is called. This call is not commonly overridden in the service.

Similar to stateless services, there's no coordination between the order in which the listeners are created and opened and when RunAsync is called. If you need coordination, the solutions are much the same. There is one additional case for stateful service.

Then in the service code files add a class WebHostWrapper that would emulate infinite execution of ICommunicationListener.OpenAsync method:
public class WebHostWrapper : IWebHost
{
    private IWebHost webHostImplementation;

    public IFeatureCollection ServerFeatures => this.webHostImplementation.ServerFeatures;

    public IServiceProvider Services => this.webHostImplementation.Services;

    public WebHostWrapper(
        IWebHost webHostImplementation)
    {
        this.webHostImplementation = webHostImplementation;
    }

    public void Dispose()
    {
        this.webHostImplementation.Dispose();
    }

    public void Start()
    {
        // This method is used by Kestrel server.
        Thread.Sleep(Timeout.Infinite);
    }

    public Task StartAsync(
        CancellationToken cancellationToken = new CancellationToken())
    {
        return this.webHostImplementation.StartAsync(cancellationToken);
    }

    public Task StopAsync(
        CancellationToken cancellationToken = new CancellationToken())
    {
        return this.webHostImplementation.StopAsync(cancellationToken);
    }
}

No use this class to wrap IWebHost pre-configured by project template.
internal sealed class Web1 : StatefulService
{
    public Web1(
        StatefulServiceContext context)
        : base(context)
    {
    }

    protected override IEnumerable<ServiceReplicaListener> CreateServiceReplicaListeners()
    {
        return new ServiceReplicaListener[]
        {
            new ServiceReplicaListener(
                serviceContext =>
                    new KestrelCommunicationListener(
                        serviceContext,
                        (
                            url,
                            listener) =>
                        {
                            var w = new WebHostBuilder()
                               .UseKestrel()
                               .ConfigureServices(
                                    services => services
                                       .AddSingleton<StatefulServiceContext>(serviceContext)
                                       .AddSingleton<IReliableStateManager>(this.StateManager))
                               .UseContentRoot(Directory.GetCurrentDirectory())
                               .UseStartup<Startup>()
                               .UseServiceFabricIntegration(
                                    listener,
                                    ServiceFabricIntegrationOptions.UseUniqueServiceUrl)
                               .UseUrls(url)
                               .Build();

                            // comment to test change role scenario
                            return new WebHostWrapper(w);
                            // uncomment to test change role scenario
                            // return w;
                        }))
        };
    }

    protected override Task OnOpenAsync(
        ReplicaOpenMode openMode,
        CancellationToken cancellationToken)
    {
        return base.OnOpenAsync(openMode, cancellationToken);
    }

    protected override Task OnChangeRoleAsync(
        ReplicaRole newRole,
        CancellationToken cancellationToken)
    {
        // uncomment to test change role scenario.
        // Thread.Sleep(Timeout.Infinite);
        return base.OnChangeRoleAsync(newRole, cancellationToken);
    }

    protected override Task RunAsync(
        CancellationToken cancellationToken)
    {
        return base.RunAsync(cancellationToken);
    }
}

According to the documentation ICommunicationListener.OpenAsync and RunAsync are happening in parallel. But if we set a breakpoint in RunAsync it will never be hit.
In the documentation there are also mentioning that OnChangeRoleAsync is called when the all listeners are opened and after the RunAsync.

After all the replica listener's OpenAsync() calls finish and RunAsync() is called, StatefulServiceBase.OnChangeRoleAsync() is called.

But currently it is not. This can be easily verified by commenting / uncommenting things in the example above (change role scenario). The RunAsync simply wouldn't be invoked until the ChangeRoleAsync is completed.
Update 2018/10/26
I have analyzed the shutdown lifecycle and found that it doesn't correspond to the documentation too.

In parallel:

Any open listeners are closed. ICommunicationListener.CloseAsync() is called on each listener.
The cancellation token passed to RunAsync() is canceled. A check of the cancellation token's IsCancellationRequested property returns true, and if called, the token's ThrowIfCancellationRequested method throws an OperationCanceledException.
After CloseAsync() finishes on each listener and RunAsync() also finishes, the service's StatefulServiceBase.OnChangeRoleAsync() is called. This call is not commonly overridden in the service.
After the StatefulServiceBase.OnChangeRoleAsync() method finishes, the StatefulServiceBase.OnCloseAsync() method is called. This call is an uncommon override, but it is available.
After StatefulServiceBase.OnCloseAsync() finishes, the service object is destructed.

According to the documentation ICommunicationListener.CloseAsync and cancellation of cancellationToken in RunAsync are happening in parallel.
public class WebHostWrapper : IWebHost
{
    private IWebHost webHostImplementation;

    public IFeatureCollection ServerFeatures => this.webHostImplementation.ServerFeatures;

    public IServiceProvider Services => this.webHostImplementation.Services;

    public WebHostWrapper(
        IWebHost webHostImplementation)
    {
        this.webHostImplementation = webHostImplementation;
    }

    public void Dispose()
    {
        // The Dispose method is called from KestrelCommunicationListener.CloseAsync method.
        // comment in order to test ChangeRoleAsync scenario.
        Thread.Sleep(Timeout.Infinite);

        this.webHostImplementation.Dispose();
    }

    public void Start()
    {
        this.webHostImplementation.Start();
    }

    public Task StartAsync(
        CancellationToken cancellationToken = new CancellationToken())
    {
        return this.webHostImplementation.StartAsync(cancellationToken);
    }

    public Task StopAsync(
        CancellationToken cancellationToken = new CancellationToken())
    {
        return this.webHostImplementation.StopAsync(cancellationToken);
    }
}

internal sealed class Web1 : StatefulService
{
    public Web1(
        StatefulServiceContext context)
        : base(context)
    {
    }

    protected override IEnumerable<ServiceReplicaListener> CreateServiceReplicaListeners()
    {
        return new ServiceReplicaListener[]
        {
            new ServiceReplicaListener(
                serviceContext =>
                    new KestrelCommunicationListener(
                        serviceContext,
                        (
                            url,
                            listener) =>
                        {
                            var w = new WebHostBuilder()
                               .UseKestrel()
                               .ConfigureServices(
                                    services => services
                                       .AddSingleton<StatefulServiceContext>(serviceContext)
                                       .AddSingleton<IReliableStateManager>(this.StateManager))
                               .UseContentRoot(Directory.GetCurrentDirectory())
                               .UseStartup<Startup>()
                               .UseServiceFabricIntegration(
                                    listener,
                                    ServiceFabricIntegrationOptions.UseUniqueServiceUrl)
                               .UseUrls(url)
                               .Build();

                            return new WebHostWrapper(w);
                        }))
        };
    }

    protected override Task OnChangeRoleAsync(
        ReplicaRole newRole,
        CancellationToken cancellationToken)
    {
        return base.OnChangeRoleAsync(newRole, cancellationToken);
    }

    protected override Task RunAsync(
        CancellationToken cancellationToken)
    {
        for (;;)
        {
            if (cancellationToken.IsCancellationRequested)
            {
                break;
            }

            Thread.Sleep(100);
        }
        return base.RunAsync(cancellationToken);
    }

    protected override Task OnCloseAsync(
        CancellationToken cancellationToken)
    {
        return base.OnCloseAsync(cancellationToken);
    }
}

If we set a breakpoint in RunAsync -> break it will never be hit. The second important note here is that ChangeRoleAsync method isn't called.
Update: 2018/10/29
I also did some more investigation and found that promotion and demotion cycles are also different from what is described in the documentation.
In the documentation for promotion cycle we have:

In parallel:

StatefulServiceBase.CreateServiceReplicaListeners() is invoked and any returned listeners are opened. ICommunicationListener.OpenAsync() is called on each listener.
The service's StatefulServiceBase.RunAsync() method is called.
After all the replica listener's OpenAsync() calls finish and RunAsync() is called, StatefulServiceBase.OnChangeRoleAsync() is called. This call is not commonly overridden in the service.

and demotion cycle is:

In parallel:

Any open listeners are closed. ICommunicationListener.CloseAsync() is called on each listener.
The cancellation token passed to RunAsync() is canceled. A check of the cancellation token's IsCancellationRequested property returns true, and if called, the token's ThrowIfCancellationRequested method throws an OperationCanceledException.
After CloseAsync() finishes on each listener and RunAsync() also finishes, the service's StatefulServiceBase.OnChangeRoleAsync() is called. This call is not commonly overridden in the service.

But there is no parallelism involed in both cycles and some of the methods aren't called (the current cycle is described in current state).
I also created a gist to simplify the testing.
Update: 2018/12/05
I have added information about abort cycle.
Current state
Startup life-cycle is the following (primary replica):

Service's OpenAsync is called.
Service's CreateServiceReplicaListeners is called.
All listeners have their OpenAsync called and awaited.
Service's ChangeRoleAsync is called and awaited.
Service's RunAsync is called and awaited.

Shutdown life-cycle is the following:

All listeners have their CloseAsync called and awaited.
CancellationToken passed to Service's RunAsync is canceled and awaited.
Service's CloseAsync is called and awaited.

Promotion cycle:

All previously created ICommunicationListener have their CloseAsync method called and awaited. The methods are called and awaited in sequence.
All ServiceReplicaListener's returned from CreateServiceReplicaListeners (during secondary replica initialization) are used to create instances of ICommunicationListener and have their OpenAsync methods called and awaited. The methods are called and awaited in sequence.
Service's ChangeRoleAsync method is called (with newRole = Primary) and awaited.
Service's RunAsync method is called.

Demotion cycle:

All previously created ICommunicationListener have their CloseAsync method called and awaited. The methods are called and awaited in sequence.
CancellationToken passed to RunAsync method is canceled.
RunAsync method is awaited.
All ServiceReplicaListener's returned from CreateServiceReplicaListeners (during primary replica initialization) where ServiceReplicaListener.ListenOnSecondary == true are used to create instances of ICommunicationListener and have their OpenAsync methods called and awaited. The methods are called and awaited in sequence.
Service's ChangeRoleAsync method is called (with newRole = ActiveSecondary) and awaited.

Please note that there is no parallelism invoked at all.
Abort cycle:

CancellationToken passed to Service’s RunAsync method is canceled.
All previously created ICommunicationListener have their Abort method executed. The methods are executed in sequence.
Service’s Abort method is executed.
Service’s implementation object is destroyed.

I too think stateful service lifecycle is still wrong in documentation or there are bugs in the system.
My observation is that CreateServiceReplicaListeners() needs to finish before RunAsync() is called which is not what the documentation says.
There might be other issues but I haven't gone through the scenarios in detail unlike @OlegKarasik probably has done.

I just want to add that proving these kind of issues with breakpoints is probably not good enough.
I've run into this trap as well, when parallel flows didn't hit breakpoints as expected. But this was always more of an VS issue. It seems that when hitting a breakpoint, VS stops executing anything, even stuff you expect to run parallel until you continue.
Usually when I add logging to both methods/tasks and a delay to one of them, I can see in the log output that they are indeed running in parallel/independently.

Hi @tuhland,
The breakpoints here are used to prove things which get never called i.e. breakpoints aren't hit at all. There is no ordering proves made with breakpoints.

I fail to get any parallelism. See screenshots.

Yeah, OK, that does indeed look strange. Didn't doubt that there are issues with the lifecycle, just wanted to make sure the proof is sound. ;)
The whole lifecycle stuff doesn't seem to be very reliable. I recently I tried to clean up managed resources on service "shutdown".
But there's no single way to do this, since depending on how the shutdown happens (Tested with restarting code package, partition or node) different lifecycle methods (Close, Abort, RoleChange and the cancellation of RunAsync token) were triggered, but not a single one for all cases.


Make it possible to call RemoteAction to ask for the next slot in FormAction

Proposed changes:

I implemented a TODO to make it possible to use a custom action to ask for the next slot.
These custom actions should be prefixed with action_ask_, and can be action_ask_{form_name}_{slot_name} or action_ask_{slot_name} shortly, just as in the utter_ask_ counterpart.

Status (please check what you already did):

[ ] added some tests for the functionality
[ ] updated the documentation
[ ] updated the changelog (please check changelog for instructions)
[x] reformat files using black (please check Readme for instructions)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.M. Yusuf Sarıgöz seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

I think you might be right behind us 😄 this is being implemented on master for rasa 2.0: https://github.com/RasaHQ/rasa/pull/6384/files#diff-263e4240a19c4cc6db32118bdb0f47d1R443

Hahha @erohmensing 😂 I was planning to review 2.0@master but then I decided to make this PR first. So should I cancel it?

Unfortunately I think so! Appreciate the PR though. How did you discover this, did you see the issue, or the to-do in the code?

@erohmensing Well I was in need of this functionality in one of our projects and I reviewed the source code to better understand how things go on with each other. And just happened to see the TODO and decided to implement it. Thanks anyway, good luck with 2.0 😄


[#1332451] Add an error for return outside of function.

Problem
See: https://bugs.launchpad.net/pyflakes/+bug/1332451
The code on master was already ignoring this error and no longer crashing.
Changes
Create a dedicated errror but not sure if this is needed as it is already raised by python as a syntax error.
Check that changes make sense of just close this pull request and the Launchpad ticket.
Thanks!

Looks good, thanks.
The reason these errors aren't already caught is that pyflakes parses, but does not compile the code. See  #8.


add frc-rapid-react

Please confirm this pull request meets the following requirements:

[x] I followed the contributing guidelines: https://github.com/github/explore/blob/main/CONTRIBUTING.md.
[x] I have no affiliation with the project I am suggesting (as a maintainer, creator, contractor, or employee).

Which change are you proposing?

[ ] Suggesting edits to an existing topic or collection
[x] Curating a new topic or collection
[ ] Something that does not neatly fit into the binary options above

Curating a new topic or collection

[x] I've formatted my changes as a new folder directory, named for the topic or collection as it appears in the URL on GitHub (e.g. https://github.com/topics/[NAME] or https://github.com/collections/[NAME])
[x] My folder contains a *.png image (if applicable) and index.md
[x] All required fields in my index.md conform to the Style Guide and API docs: https://github.com/github/explore/tree/main/docs

I think that, while not super widely used (~30 repositories between the aliases), it would be helpful to add context for these topics as FRC repositories are commonly used as references in future years.

Please note: we will close your PR without comment if you do not check the boxes above and provide ALL requested information.

Thank you for the PR, however, according to our contribution guidelines:

We are likely to consider suggestions to curate a topic or collection that is valuable to GitHub's community. Valuable topics, for example, include those that are already widely used by repositories and could benefit from the addition of important information. When suggesting content, please consider how to make your contribution broadly useful and relevant to others, rather than serving a specific use case.

This topic is only used on 16 repositories, therefore it does not qualify as "widely used".