Support scripting for composite aggs in concurrent segment search

Description
Fix race condition related to SourceLookup when using _source scripts in concurrent segment search.
SearchLookup is created per-QueryShardContext so it is shared across threads. SearchLookup::getLeafSearchLookup is called as a part of getLeafCollector so it creates a new LeafSearchLookup for each thread. However, the SourceLookup passed into each LeafSearchLookup is the single instance created in SearchLookup which will be shared across threads but is not thread safe. To resolve this we initialize a new SourceLookup and pass it into each new LeafSearchLookup instance.
Ideally we would create a new SourceLookup only for each slice instead of for each leaf, however I don't see a great way to do that.
Related Issues
Resolves #12947
Relates #15007
Check List

[x] Functionality includes testing.
~- [ ] API changes companion pull request created, if applicable.~
[ ] Public documentation issue/PR created, if applicable. (TODO: Create docs issues)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@sohami @rishabhmaurya @mch2 could you please help review?

{"run-benchmark-test": "id_3"}

{"run-benchmark-test": "id_5"}

@rishabhmaurya @mch2 I think we can resolve https://github.com/opensearch-project/OpenSearch/issues/15007 with this PR as well, however I would like to increase the test coverage some more first. Are there any more derived fields integration tests that we can run with concurrent search enabled and disabled?
It looks like DerivedFieldFetchAndHighlightTests doesn't actually go through the query phase so I don't think there's any point parameterizing that class. Similarly the DerivedFieldAggregationTests calls IndexSearcher.search directly instead of through the query phase so there isn't a great way to trigger concurrent search there either.

The Jenkins job url is https://build.ci.opensearch.org/job/benchmark-pull-request/250/ . Final results will be published once the job is completed.

@rishabh6788 this looks to have failed with
	in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@3bc37493
	in object of type org.jenkinsci.plugins.workflow.cps.CpsThreadGroup
Caused: java.io.IOException: Stale file handle
	at java.base/java.io.FileInputStream.available0(Native Method)

mind taking a look?
@jed326 My main thought would be any added overhead here of not reusing the lookup but getLeafSearchLookup is already called per segment so I think we should be ok...waiting on the perf run to approve

@rishabhmaurya @mch2 I think we can resolve #15007 with this PR as well, however I would like to increase the test coverage some more first. Are there any more derived fields integration tests that we can run with concurrent search enabled and disabled?
It looks like DerivedFieldFetchAndHighlightTests doesn't actually go through the query phase so I don't think there's any point parameterizing that class. Similarly the DerivedFieldAggregationTests calls IndexSearcher.search directly instead of through the query phase so there isn't a great way to trigger concurrent search there either.

I don't know that we have any existing internalClusterTests that we could add to here, for aggs I heavily leveraged yaml rest tests.  I think we need to add internalClusterTests in the painless plugin.

The Jenkins job url is https://build.ci.opensearch.org/job/benchmark-pull-request/250/ . Final results will be published once the job is completed.

@rishabh6788 this looks to have failed with
	in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@3bc37493
	in object of type org.jenkinsci.plugins.workflow.cps.CpsThreadGroup
Caused: java.io.IOException: Stale file handle
	at java.base/java.io.FileInputStream.available0(Native Method)

mind taking a look?
@jed326 My main thought would be any added overhead here of not reusing the lookup but getLeafSearchLookup is already called per segment so I think we should be ok...waiting on the perf run to approve

Seems there was a deployment in jenkins, please feel free to comment again to restart the job.

{"run-benchmark-test": "id_5"}

I think we need to add internalClusterTests in the painless plugin.

@mch2 @rishabhmaurya  I added 2 basic ITs using derived fields in the latest revision. Unfortunately both fail with concurrent search enabled, it looks like it's the same issue as with the agg scripts, a SourceLookup is being shared across threads somewhere.

@jed326 could you rebase please (main build for fixed), thank you

Not sure if there is a better way to compare the perf numbers, but I checked against the big5 run on https://opensearch.org/benchmarks/ and the numbers look basically the same.

@reta @sohami @mch2 mind taking another look now that the perf numbers are in and look fine?

Not sure if there is a better way to compare the perf numbers, but I checked against the runs on some other PRs (#14464 (comment), #14993 (comment)) and the numbers look very similar.
Ah found the baseline dashboard: https://github.com/opensearch-project/OpenSearch/blob/main/PERFORMANCE_BENCHMARKS.md#how-to-compare-my-results-against-baseline

https://github.com/opensearch-project/OpenSearch/issues/14826 is in progress and would add the baseline comparison results on the PR as well once done.

Not sure if there is a better way to compare the perf numbers, but I checked against the runs on some other PRs (#14464 (comment), #14993 (comment)) and the numbers look very similar.
Ah found the baseline dashboard: https://github.com/opensearch-project/OpenSearch/blob/main/PERFORMANCE_BENCHMARKS.md#how-to-compare-my-results-against-baseline

#14826 is in progress and would add the baseline comparison results on the PR as well once done. Till then you can use the dashboard to compare the numbers.

@reta @sohami @mch2 mind taking another look now that the perf numbers are in and look fine?

Changes looks fine to me, I think we should create an issue in opensearch benchmark workload to add support for painless script as well.

@jed326 mind please backport manually? thank you


[question] is it possible to render with SkiaSharp clientside via Brazor ?

I am just wondering if it is possible to render with SkiaSharp (or alike) client side via Brazor engine?

Thanks for contacting us, @YordanYanakiev.
This is not something we have plans for.

To clarify, this is something you can certainly do on your own, in the sense that you have access to JS interop and can therefore call any third-party API that you want.
We are just not planning any built-in features that are specific to SkiaSharp.

Oww. Thank You for the clarification. For a second I was thinking that You are abandoning the client side of this project, which well.. would make the whole super-awesome project just meaningless.
Waiting impassionately for the client side implementation ! :)


Add the event handler in tests using dom-events

IE (perhaps before 11) doesn't have Element.prototype.addEventListener. It has Element.prototype.attachEvent, instead.
This module handles this difference.

This prevents the tests from running in node.js. I'm not sure how easy it would be to work around it.

You're right, Brian.
Please see #16, instead.


VSCode Version: 1.33.1
OS Version: Windows 10

Related to #35148, #36122
Steps to Reproduce:

Laptop connected to external monitor
Start vscode
Use normally
Leave VS code running and undock/disconnect from monitor
VScode is stuck offscreen
Attempted all these steps to fix: https://www.technipages.com/bring-off-screen-window-back-onto-screen. None of them worked
Restarted laptop
Started vscode - it is still stuck off-screen

How did I fix it:

I had to connect back to my external monitor to start using vscode again

This makes vscode unusable.

Does this issue occur when all extensions are disabled?: Yes/No

I cannot reproduce this.  My VS Code instance always switches to the laptop screen and resizes itself to fit in the screen.   There is probably some missing detail, a particular video driver or video feature that is tripping this.
I also don't see why VS Code wouldn't resond to the normal method for retrieving an off screen window (click on taskbar tile, ALT-SPACE, M, arrow-key, then move mouse.)
However, I was able to get Explorer to lock up when I moved VS Code's window offscreen.  However, when I restarted Explorer, the VS Code window that had been off screen popped back to its original position.

I also don't see why VS Code wouldn't resond to the normal method for retrieving an off screen window (click on taskbar tile, ALT-SPACE, M, arrow-key, then move mouse.)

Possibly due to the custom menu, @sbatten ?
@smultani does it reproduce when you configure the title to be native?

@sbatten custom menu/title does not prevent the alt+space m trick. In fact I use this sometimes myself.

@sbatten is there something you want/can change?


Add Crashlytics JS client

Not sure there is a JS client, but if there is, add JS support

Crashlytics is only available for Android, iOS and Unity.

👍


⚠ Samsung has degraded performance

In 88c7232, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2752 ms

Resolved: Samsung performance has improved in 460b65d.


What about manual installation?

Not everyone use composer, so please add a manual installation guide, or at least a folder that we can drag&drop in our projects, thanks 😃

Hi @portapipe of course you can use the library without composer. However I wouldn't advise to do so because manual version management and autoloading are much more error prone. That's one of the reasons why it is not described in the readme.
Still, if you want to do this manually all you have to do is

get the project files from GitHub (either clone or download as ZIP)
make sure your autoloader looks in the right directories

Custom autoloading in PHP is a bit beyond the scope of this library but you can find many sources via Google regarding this topic.
Good luck!


Hide the button

Is there a way to hide the button itself? I want to open programmatically with my own flow

This feature was added in #72
It's available on master which you can use by pointing directly to this repo.  It will be part of the next npm release.

Version 0.0.28 is now released and contains this feature.


Hosted Docs (doxygen)

It may be useful to set up online hosted docs to build using doxygen.
This may require tweaking cmake, such that it is possible to build the docs without cuda being required.

It's been suggested that breathe can be used with sphinx etc

Build docs without CUDA has been added to master

203482c4d055de6154b64af7a24d974a885fa215


[util] Fix compile_commands.json for git worktrees

I tried running util/generate_compilation_db.py in a git worktree and
discovered that the "bazel-opentitan" symlink did not exist! It turns
out that "bazel-opentitan" was just a special case and the general rule
is "bazel-${workspace_name}" [0].

This PR removes the hardcoded "bazel-opentitan" and instead builds the
path relative to the source file. It moves some related path
construction into a class and adds tests.

[0]: https://bazel.build/remote/output-directories

Thanks @dmcardle! I'm now curious why this worked fine for me from the beginning. I use worktrees but my bazel setup uses a custom output_base for every worktree. Not sure if that's relevant.

Clangd's behavior is a little opaque. Are your git worktrees nested under a parent checkout? Is it possible clangd rejected the deeper compile_commands.json and grabbed one in a higher directory?


implement propositional truncation

these basic facts about propositional truncations don't seem to be anywhere else yet

Unfortunately the implementation of recursive HIT (such as truncation
but not suspension) has not been fixed yet.
Thierry

On 01 Jun 2016, at 17:15, Auke Booij<EMAIL_ADDRESS>wrote:
these basic facts about propositional truncations don't seem to be anywhere else yet
You can view, comment on, or merge this pull request online at:
https://github.com/mortberg/cubicaltt/pull/39 https://github.com/mortberg/cubicaltt/pull/39
Commit Summary
implement propositional truncation
File Changes
A examples/propTrunc.ctt https://github.com/mortberg/cubicaltt/pull/39/files#diff-0 (26)
Patch Links:
https://github.com/mortberg/cubicaltt/pull/39.patch https://github.com/mortberg/cubicaltt/pull/39.patch
https://github.com/mortberg/cubicaltt/pull/39.diff https://github.com/mortberg/cubicaltt/pull/39.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/mortberg/cubicaltt/pull/39, or mute the thread https://github.com/notifications/unsubscribe/ACrXlGWKxbdhTKKsDByMRVCMuxkwWq07ks5qHaIIgaJpZM4IrqNg.

Fixed in theory or in the implementation? What about the circle, in particular? What goes wrong with the current implementation?

Fixed in the implementation.
The problem occurs only for HIT with parameters. So the circle should be correctly
implemented.
Hopefully the theory is all right. The paper on cubical type theory describes the semantics/type
theoretic rules for propositional truncation, which is surprisingly subtle.
The circle is also described
there and for the circle everything behaves in the theory as one expects. It is essential for this
to have a diagonal operation on names (and this was the motivation for introducing this operation).

On 02 Jun 2016, at 20:50, Martin Escardo<EMAIL_ADDRESS>wrote:
Fixed in theory or in the implementation? What about the circle, in particular? What goes wrong with the current implementation?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/mortberg/cubicaltt/pull/39#issuecomment-223386452, or mute the thread https://github.com/notifications/unsubscribe/ACrXlHXgE8kD0P4oDspfF3luW3U_x0SLks5qHyX-gaJpZM4IrqNg.

Thanks, and good. What is wrong with the implementation of propositional truncation?
Where and how will I see a symptom of its incorrect implementation?

Thanks, Auke, for the patch!  Maybe we could add it with a disclaimer.  I propose to rename pTrunc into inh though.
Martin, if I recall correctly, there was an example of this wrong behaviour in:
https://github.com/mortberg/cubicaltt/issues/35
The problem is when defining the "squeeze" and "transp" functions (which are needed to define compositions for propositional truncation); the problematic case is for elements of the form VPCon (path constructors).

@simhu, I implemented your proposals.

Thanks and sorry that it took so long to merge...


Define signon-api-token

This defines a new secret for Whitehall to access the newly minted user details endpoint in Signon.
The secrets are set up in all three AWS environments

Closing this as these are automatically generated by the sync_token_secrets task in Whitehall.


#14: Vector graphics mode and infinite canvas

#14

@ShubertMunthali I believe there are build errors at the moment in this branch (as below).
Please resolve them.

It would be nice if you can provide demo video of the change

[ ] Also, we should make title property optional for graphic notes.

[ ] Also, we should make title property optional for graphic notes.

So what should we use to show the note on the list of notes?

@ShubertMunthali ok, probably purely optional is not good idea. I think, we should generate a placeholder when we create a note. If the user clicks on the title field and starts typing, the placeholder is replaced by what the user writes. If the user didn't provide title, the placeholder becomes real title.
The placeholder should be generated in unique way, e.g. contain date and time.

@ShubertMunthali ok, probably purely optional is not good idea. I think, we should generate a placeholder when we create a note. If the user clicks on the title field and starts typing, the placeholder is replaced by what the user writes. If the user didn't provide title, the placeholder becomes real title.
The placeholder should be generated in unique way, e.g. contain date and time.

Or I thought for graphic notes we can just use a thumbnail for preview, and it will make graphic notes easy to identify among the notes list. But no problem with title, we can combine. We already show date, we can choose a default title like "Note ${number}". Maybe number maybe date

https://github.com/ARK-Builders/ARK-Memo/assets/87703131/7356c28a-9ce0-488f-bd10-5e8b367268be

@ShubertMunthali There's one thing which relates to git commits that I'd like to add: It seems you're using fast-forward merge when syncing latest changes from main.
Using fast-forward merge often results in messy git history because it creates extra merge commit unnecessarily.
It's recommended to use rebase strategy in feature/bug fix branch instead to keep history clean/straight-forward.

@ShubertMunthali There's one thing which relates to git commits that I'd like to add: It seems you're using fast-forward merge when syncing latest changes from main.
Using fast-forward merge often results in messy git history because it creates extra merge commit unnecessarily.
It's recommended to use rebase strategy in feature/bug fix branch instead, to keep history clean/straight-forward.
This could be done with future PRs since it will be pretty cumbersome to fix the current branch's history :)

Thanks buddy, I had no idea. I will make sure

Thanks for mentioning that @tuancoltech Merge commits are removed during the final merge (because we use only rebase or squash), but they still make reviewing slightly more difficult.

@ShubertMunthali Relating to UI/UX, I find the current 2 FABs quite boring when they stack on top of each other, as that occupies more screen space.
What do you think about this grouping UI instead?
https://www.geeksforgeeks.org/extended-floating-action-button-in-android-with-example/

When I edited a graphic note's title, it says Note already exists.
https://recordit.co/M4LQXvZT1S

When I edited a graphic note's title, it says Note already exists.
https://recordit.co/M4LQXvZT1S
It seems to be related to the special characters in the title, since I can reproduce that even with text note: https://recordit.co/qTpKM0jh1h

@tuancoltech I understand the problem here. I will create new issue to describe how we should solve that bug

@ShubertMunthali Relating to UI/UX, I find the current 2 FABs quite boring when they stack on top of each other, as that occupies more screen space.
What do you think about this grouping UI instead?
https://www.geeksforgeeks.org/extended-floating-action-button-in-android-with-example/

Looks pretty awesome. If we have some better options we should consider too. I like this at the moment. Cleans UI indeed :)

@ShubertMunthali Sorry, i missed this.
The color scheme of the FAB doesn't seem to match our current theme color.
Can you update it please?

@tuancoltech good observation about colors, but yes we'll work on it separately.


Toolkit Reports broken

Defective ToolKit Feature Name: Toolkit Reports
Expected Behavior
The toolkit reports should show my income properly, but doesn't anymore.
Actual Behavior
Suddenly the entire graph is messed up. It suddenly thinks I racked up 3k in debt (which I don't have at all, nor does it show in the default reports). It doesn't recognize January 2017 but instead says "Undefined". It also doesn't correctly show my spending. There's a lot of info not taken into account.
Clearing my cache or cookies didn't help.
Steps to Reproduce the Behavior

It happened after the last update to 0.7.11, no problems before
Open up the toolkit reports any time.

Export of Toolkit Settings (Bottom Left of Settings Page)
[{"key":"accountTransactionSearch","value":true},{"key":"accountsClearSelection","value":true},{"key":"accountsDisplayDensity","value":"0"},{"key":"accountsEmphasizedOutflows","value":false},{"key":"accountsRowHeight","value":"0"},{"key":"accountsSelectedTotal","value":true},{"key":"accountsStripedRows","value":true},{"key":"activityTransactionLink","value":true},{"key":"betterScrollbars","value":"0"},{"key":"budgetBalanceToZero","value":true},{"key":"budgetProgressBars","value":"0"},{"key":"budgetQuickSwitch","value":false},{"key":"budgetRowsHeight","value":"0"},{"key":"calendarFirstDay","value":"0"},{"key":"categoryActivityPopupWidth","value":"0"},{"key":"changeEnterBehavior","value":false},{"key":"checkCreditBalances","value":true},{"key":"checkNumbers","value":false},{"key":"closeReconcileWindow","value":false},{"key":"collapseExpandBudgetGroups","value":true},{"key":"collapseSideMenu","value":false},{"key":"colourBlindMode","value":false},{"key":"compactIncomeVsExpense","value":false},{"key":"currentMonthIndicator","value":true},{"key":"daysOfBuffering","value":true},{"key":"daysOfBufferingHistoryLookup","value":"0"},{"key":"editButtonPosition","value":"0"},{"key":"enableRetroCalculator","value":true},{"key":"enlargeCategoriesDropdown","value":true},{"key":"enterInRegisterNow","value":true},{"key":"goalIndicator","value":true},{"key":"goalIndicatorWarningColor","value":false},{"key":"googleFontsSelector","value":"0"},{"key":"hideAOM","value":false},{"key":"hideHelp","value":false},{"key":"highlightNegativesNegative","value":false},{"key":"importNotification","value":"0"},{"key":"incomeFromLastMonth","value":"0"},{"key":"l10n","value":"0"},{"key":"largerClickableIcons","value":true},{"key":"monthlyNotesPopupWidth","value":"0"},{"key":"navDisplayDensity","value":"0"},{"key":"pacing","value":"0"},{"key":"popupCalculator","value":false},{"key":"printingImprovements","value":false},{"key":"privacyMode","value":"0"},{"key":"reconciledTextColor","value":"0"},{"key":"removePositiveHighlight","value":false},{"key":"removeZeroCategories","value":true},{"key":"reports","value":true},{"key":"resizeInspector","value":false},{"key":"rightClickToEdit","value":true},{"key":"runningBalance","value":false},{"key":"seamlessBudgetHeader","value":true},{"key":"showIntercom","value":true},{"key":"splitKeyboardShortcut","value":true},{"key":"splitTransactionAutoAdjust","value":true},{"key":"squareNegativeMode","value":false},{"key":"stealingFromNextMonth","value":true},{"key":"swapClearedFlagged","value":false},{"key":"toBeBudgetedWarning","value":true},{"key":"toggleSplits","value":true},{"key":"toggleTransactionFilters","value":"0"},{"key":"warnOnQuickBudget","value":false}]

System Settings
ToolKit Version: 0.7.11
Operating System (Mac/Windows/Etc): Windows 10
Browser (Chrome/Firefox/Safari): Chrome
Browser Version:  56.0.2924.87 64 bit
Other Extensions: OneNote, Grammarly, Click&Clean, Pushbullet, Lastpass

This has been reported by multiple users. @blargity is working on it.

@Zendio, I suspect your date format is dd/mm/yyyy like mine is. If that's the case, what will hopefully turn out to be the fix is available on the fix-702 branch and if you know how to build from source some additional testers would be tops.
If not, this will be in our next release.
Thanks for reporting this!


Python - local imports cause invalid serialization code

kiota generate -d https://raw.githubusercontent.com/APIs-guru/openapi-directory/gh-pages/v2/specs/github.com/api.github.com/1.1.4/openapi.json -l Python

Results in
integration_test/client/models/workspace_user.py:115: error: Argument 2 to "write_object_value" of "SerializationWriter" has incompatible type "ErrorDetails | None"; expected "None"  [arg-type]

The imports were added in this commit 139c9589cc8a4a995700cdbab7330ea736c4e2bf

Maybe @andreaTP can help provide more context here? (the question being, why are we adding local imports as opposed to to level ones?)

This is the change I'm referring to for context. https://github.com/microsoft/kiota/pull/5632/files#diff-e772039541f9ede8f8ee87a4c43f45c040c4a1bbe831615688f289f211fdf35aR649


Float Script Editor other Workspace slow

Godot version
4.1 RC1
System information
Fedora Linux Gnome or Hyprland - Wayland
Issue description
Editor becomes unresponsive when I drag float script editor to another workspace, same behavior happens both in Gnome and Hyprland both in Wayland
error that appears in the log I don't know if it helps: platform/linuxbsd/x11/display_server_x11.cpp:1727 - Condition "prev_parent == p_parent" is true.
Steps to reproduce
Drag float script editor to another workspace
Minimal reproduction project
N/A

Related to https://github.com/godotengine/godot/issues/75140.

Related to [4.0] weird focus behavior on window with child windows #75140.

This sounds like the FPS limiter going haywire. You may be able to work around this issue by setting Unfocused Low Processor Mode Sleep Usec to 6900 (same value as Low Processor Mode Sleep Usec).

Tested and still with the same behavior

Just found a workaround that fixed for me.
Launch godot with MESA_VK_WSI_PRESENT_MODE=immediate


disqus comments are not loading

Every post file contains comments: true to load disqus
So disqus should load in every post.
But disqus loads in code highlighting post. It doesn't load in other posts.

They're working fine on the demo site just fine, something must be screwy on your fork. You have the Disqus shortname set in _config.yml as well right?

@mmistakes Yes. I've set it to rootpy But not showing it.
I forked it 2 days ago and I don't think you've updated your repo in last 2 days.
disqus was loading only for  code highlighting post when I checked on live and decided to report here.
You can check it by visiting my first post Hello World
It says We were unable to load Disqus. If you are a moderator please see our troubleshooting guide.

Sounds like a Disqus issue. I've seen this happen before and usually is a
cookie problem. You can verify if that's the case by opening the page in
Chrome incognito mode or by clearing your browser/cookie cache.
On Saturday, April 4, 2015, Prasad Muley<EMAIL_ADDRESS>wrote:

@mmistakes https://github.com/mmistakes Yes. I've set it to rootpy
https://disqus.com/by/rootpy/ But not showing it.
I forked it 2 days ago and I don't think you've updated your repo in last
2 days.
disqus was loading only for code highlighting post
https://mmistakes.github.io/minimal-mistakes/code-highlighting-post/
when I checked on live and decided to report here.
You can check it by visiting my first post Hello World
http://rootpy.com/Hello-World/
It says We were unable to load Disqus. If you are a moderator please see
our troubleshooting guide.
—
Reply to this email directly or view it on GitHub
https://github.com/mmistakes/minimal-mistakes/issues/139#issuecomment-89645599
.

@mmistakes.
I had deleted cookie cache and Disqus isn't loading in chrome incognito mode or priavate browing in firefox.
Checked disqus documentation for jekyll and I've entered correct user_name there.
I am not able to figure out why is it not loading in Hello World!

Well the problem isn't Jekyll or the theme. Looks to me you haven't registered a shortname for your site on Disqus yet. If you had rootpy.disqus.com would work and it doesn't. rootpy is your username and a shortname.
Check out this page on Disqus on more info on how to setup one for your site. After you create a shortname for your site then replace rootpy in your _config.yml Disqus should work properly.

@mmistakes Yes. I forgot to add my website's  shortname
I thought username and shortname are same.
It is working in Hello World!
Thanks for pointing my mistake.

I have been successful with the problem, by simply replacing a new account using the account facebook or tweeter. Previously, it was the same account to the account of my website

disqus comment not displaying not my website..
http://androidtoall.com/best-wordpress-free-responsive-theme/

@navneet1991

Your site isn't using the Minimal Mistakes theme.
Your site is using Wordpress and not Jekyll.
Disqus is indeed working on your site.


SixLabors.ImageSharp dependency

Hi! Any chance of a nuget release that depends on the latest SharpLabors.ImageSharp version (1.0.1) rather than 1.0.0-beta0007 ?

Hey, the dependency has been updated in the newest version (version 1.2.3)

Excellent; thanks!


SkyDemon Support

Stratux version:

Stratux config:
SDR

[* ] single
[ ] dual

GPS

[* ] yes
[ ] no
type:

AHRS

[ ] yes
[ *] no

power source: LiPo via regulator and soldered directly to voltage rails
usb cable: N/A

EFB app and version: SkyDemon 3.7.1
EFB platform: Android 7.1.1
EFB hardware: Nexus 9

Description of your issue:

There doesn't seem to be any information on whether SkyDemon should/will be able to connect to Stratux. SkyDemon supports data in via FLARM, Dynon SkyView, X-Plane and GDL90 - does Stratux support any of these protocols? There is a brief mention that by changing the IP of Stratux to <IP_ADDRESS> you can get SkyDemon to work with Flarm but this was never fully described: http://forums.skydemon.aero/Topic19950.aspx

Note: for use in Europe, RunwayHD and SkyDemon are the only applications really.

@crablab - there is some commentary here on using SkyDemon with Stratux:
https://github.com/cyoung/stratux/issues/489
Stratux outputs data in GDL90 format to port 4000 to all Wi-Fi clients.


[Request/Question] Is it okay to borrow some of the C generation code for own project?

I love DNNE this far for my Clarion based projects - now I want to take it a step further and also generate the necassary files for my Clarion projects (mostly wrappers and stuff for interactions with the DLL) - is it okay if I borrow/look into the generation part of this project?
Or maybe, we could extract the generation part in a generic way, maybe make it hookable, so one can generate additional files based on the collected methods and signatures (as this is the part I really need to generate the consuming side).

To be more precise - hooking into here:
https://github.com/AaronRobinsonMSFT/DNNE/blob/570d39ef439d54eaa6c68ca423f17a3753eeb6b7/src/dnne-gen/Generator.cs#L311
That's all I need to do and then export my own files 🤣

is it okay if I borrow/look into the generation part of this project?

@ThaDaVos DNNE is MIT, but honestly it could be public domain as far as I am concerned. Feel free to copy code if that is helpful or suggest extension points for dnne-gen. I wrote dnne-gen specifically to be used independently if desired so if there is an extension point that would help you suggest it and we can discuss the best way to move forward.

Thanks! I'll first try to add it straight into the source code of dnne-gen so I can check what kind of information I need to generate the files I need - based on that I can give some recommendations on how we can move forward and open up dnne-gen to also generate the files for consumption

I am still working on this - but to better support a scenario I made changes inside the MSBUILD part by adding a new PROP - but it seems it's not picked up - do I need to seperatly build something? (A contribution guide could help a lot)

Got the above working - was looking in the wrong place - needed to pass it to the generator, not the compiler so had to add some conditions etc to DNNE.targets file was enough

@AaronRobinsonMSFT - I am finally done with all the changes and extra
The things I did:

Extracted all classes from the Generator.cs to separate files to get a clear overview of the files used
Re-Written/Moved logic from the Generator.cs class regarding reading of the Assembly to a seperate class called AssemblyReader and introduced a few new types to keep the hierarchy in sync
Implemented generators for the specific files I needed for Clarion - I followed a per file generator logic as it's easier to implement and hook in this way if you would go with a plugin like system to register extra generators (as each generator can be used independently of each other)
Added an extra version of the C99Generator which prefixes certain stuff with the resolving classname to make it easier to create class based wrappers in other languages and prevent conflicts if multiple classes export methods with the same name (this way the library resolves this, instead of the user/developer needing to do this, thinking back to the ease-of-use discussion we participated in a few weeks ago)

All can be found here:
https://github.com/ThaDaVos/DNNE
If you want a PR to ease the comparison, please ask - but it shouldn't be merged as it contains code specifically for Clarion

@ThaDaVos Thanks for the example. It looks reasonable to me. The idea of a plugin system is nice and it seems you've got a reasonable architecture. I think all plugins would need to exist in DNNE itself however. If not, a loading mechanism and SDK would need to be designed. If you want I would move all language specific code into respective sub folders (that is, C99/ and Clarion/) and we can figure out the right UX for triggering those generators via MSBuild.
I'm also happy to accept the work even if the Clarion support isn't included.

Something like in my latest commit?
I cleaned up the code, moved everything which seemed correct to their own namespaces and added support for selecting the generators one wants to use.
As you said, it's probably best to have this included in the base of DNNE or maybe eventually through some plugin like system like with nuget packages which register their own generators, like DNNE.Clarion for example.
Also, if this is included in the base code, other people can benefit from it also

Just thinking - currently I've got two C99 generators in there - as when you gonna generate Class wrappers (for example Clarion) you'll end up with conflicting Export names, so DNNE takes care of this for you - but maybe the functionality in which the Class generator version differs, would be a great default instead of having the possibility to generate conflicts and have compile errors due that? Helps with the ease of use of the package and feels like a more logical default - if you want I can generate a .g.c file for each scenario so you can compare

Here are the diffs:

Without Classes
//
// Auto-generated by dnne-gen
//
// .NET Assembly: ExperimentalCalculator
//

//
// Declare exported functions
//
#ifndef __DNNE_GENERATED_HEADER_EXPERIMENTALCALCULATOR__
#define __DNNE_GENERATED_HEADER_EXPERIMENTALCALCULATOR__

#include <stddef.h>
#include <stdint.h>
#ifdef DNNE_COMPILE_AS_SOURCE
    #include <dnne.h>
#else
    // When used as a header file, the assumption is
    // dnne.h will be next to this file.
    #include "dnne.h"
#endif // !DNNE_COMPILE_AS_SOURCE

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Init
DNNE_EXTERN_C DNNE_API intptr_t DNNE_CALLTYPE_CDECL Init(double data);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Result
DNNE_EXTERN_C DNNE_API double DNNE_CALLTYPE_CDECL Result(intptr_t calculator);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Add
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Add(intptr_t calculator, double value);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Subtract
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Subtract(intptr_t calculator, double value);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Multiply
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Multiply(intptr_t calculator, double value);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Divide
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Divide(intptr_t calculator, double value);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Power
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Power(intptr_t calculator, int32_t value);
#endif // (DNNE_WINDOWS)

#endif // __DNNE_GENERATED_HEADER_EXPERIMENTALCALCULATOR__

//
// Define exported functions
//
#ifdef DNNE_COMPILE_AS_SOURCE

#ifdef DNNE_WINDOWS
    #ifdef _WCHAR_T_DEFINED
        typedef wchar_t char_t;
    #else
        typedef unsigned short char_t;
    #endif
#else
    typedef char char_t;
#endif

//
// Forward declarations
//

extern void* get_callable_managed_function(
    const char_t* dotnet_type,
    const char_t* dotnet_type_method,
    const char_t* dotnet_delegate_type);

extern void* get_fast_callable_managed_function(
    const char_t* dotnet_type,
    const char_t* dotnet_type_method);

//
// String constants
//

#ifdef DNNE_TARGET_NET_FRAMEWORK
    static const char_t* t1_name = DNNE_STR("Experimental_Calculator.Export");
#else
    static const char_t* t1_name = DNNE_STR("Experimental_Calculator.Export, ExperimentalCalculator");
#endif // !DNNE_TARGET_NET_FRAMEWORK

//
// Exports
//

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Init
static intptr_t (DNNE_CALLTYPE_CDECL* Init_ptr)(double data);
DNNE_EXTERN_C DNNE_API intptr_t DNNE_CALLTYPE_CDECL Init(double data)
{
    if (Init_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Init");
        Init_ptr = (intptr_t(DNNE_CALLTYPE_CDECL*)(double data))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Init_ptr(data);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Result
static double (DNNE_CALLTYPE_CDECL* Result_ptr)(intptr_t calculator);
DNNE_EXTERN_C DNNE_API double DNNE_CALLTYPE_CDECL Result(intptr_t calculator)
{
    if (Result_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Result");
        Result_ptr = (double(DNNE_CALLTYPE_CDECL*)(intptr_t calculator))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Result_ptr(calculator);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Add
static int32_t (DNNE_CALLTYPE_CDECL* Add_ptr)(intptr_t calculator, double value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Add(intptr_t calculator, double value)
{
    if (Add_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Add");
        Add_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, double value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Add_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Subtract
static int32_t (DNNE_CALLTYPE_CDECL* Subtract_ptr)(intptr_t calculator, double value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Subtract(intptr_t calculator, double value)
{
    if (Subtract_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Subtract");
        Subtract_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, double value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Subtract_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Multiply
static int32_t (DNNE_CALLTYPE_CDECL* Multiply_ptr)(intptr_t calculator, double value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Multiply(intptr_t calculator, double value)
{
    if (Multiply_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Multiply");
        Multiply_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, double value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Multiply_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Divide
static int32_t (DNNE_CALLTYPE_CDECL* Divide_ptr)(intptr_t calculator, double value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Divide(intptr_t calculator, double value)
{
    if (Divide_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Divide");
        Divide_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, double value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Divide_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Power
static int32_t (DNNE_CALLTYPE_CDECL* Power_ptr)(intptr_t calculator, int32_t value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Power(intptr_t calculator, int32_t value)
{
    if (Power_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Power");
        Power_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, int32_t value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Power_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#endif // DNNE_COMPILE_AS_SOURCE

With Classes
//
// Auto-generated by dnne-gen
//
// .NET Assembly: ExperimentalCalculator
//

//
// Declare exported functions
//
#ifndef __DNNE_GENERATED_HEADER_EXPERIMENTALCALCULATOR__
#define __DNNE_GENERATED_HEADER_EXPERIMENTALCALCULATOR__

#include <stddef.h>
#include <stdint.h>
#ifdef DNNE_COMPILE_AS_SOURCE
    #include <dnne.h>
#else
    // When used as a header file, the assumption is
    // dnne.h will be next to this file.
    #include "dnne.h"
#endif // !DNNE_COMPILE_AS_SOURCE

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Init
DNNE_EXTERN_C DNNE_API intptr_t DNNE_CALLTYPE_CDECL Export_Init(double data);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Result
DNNE_EXTERN_C DNNE_API double DNNE_CALLTYPE_CDECL Export_Result(intptr_t calculator);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Add
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Add(intptr_t calculator, double value);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Subtract
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Subtract(intptr_t calculator, double value);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Multiply
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Multiply(intptr_t calculator, double value);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Divide
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Divide(intptr_t calculator, double value);
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Power
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Power(intptr_t calculator, int32_t value);
#endif // (DNNE_WINDOWS)

#endif // __DNNE_GENERATED_HEADER_EXPERIMENTALCALCULATOR__

//
// Define exported functions
//
#ifdef DNNE_COMPILE_AS_SOURCE

#ifdef DNNE_WINDOWS
    #ifdef _WCHAR_T_DEFINED
        typedef wchar_t char_t;
    #else
        typedef unsigned short char_t;
    #endif
#else
    typedef char char_t;
#endif

//
// Forward declarations
//

extern void* get_callable_managed_function(
    const char_t* dotnet_type,
    const char_t* dotnet_type_method,
    const char_t* dotnet_delegate_type);

extern void* get_fast_callable_managed_function(
    const char_t* dotnet_type,
    const char_t* dotnet_type_method);

//
// String constants
//

#ifdef DNNE_TARGET_NET_FRAMEWORK
    static const char_t* t1_name = DNNE_STR("Experimental_Calculator.Export");
#else
    static const char_t* t1_name = DNNE_STR("Experimental_Calculator.Export, ExperimentalCalculator");
#endif // !DNNE_TARGET_NET_FRAMEWORK

//
// Exports for Export
//

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Init
static intptr_t (DNNE_CALLTYPE_CDECL* Export_Init_ptr)(double data);
DNNE_EXTERN_C DNNE_API intptr_t DNNE_CALLTYPE_CDECL Export_Init(double data)
{
    if (Export_Init_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Init");
        Export_Init_ptr = (intptr_t(DNNE_CALLTYPE_CDECL*)(double data))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Export_Init_ptr(data);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Result
static double (DNNE_CALLTYPE_CDECL* Export_Result_ptr)(intptr_t calculator);
DNNE_EXTERN_C DNNE_API double DNNE_CALLTYPE_CDECL Export_Result(intptr_t calculator)
{
    if (Export_Result_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Result");
        Export_Result_ptr = (double(DNNE_CALLTYPE_CDECL*)(intptr_t calculator))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Export_Result_ptr(calculator);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Add
static int32_t (DNNE_CALLTYPE_CDECL* Export_Add_ptr)(intptr_t calculator, double value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Add(intptr_t calculator, double value)
{
    if (Export_Add_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Add");
        Export_Add_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, double value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Export_Add_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Subtract
static int32_t (DNNE_CALLTYPE_CDECL* Export_Subtract_ptr)(intptr_t calculator, double value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Subtract(intptr_t calculator, double value)
{
    if (Export_Subtract_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Subtract");
        Export_Subtract_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, double value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Export_Subtract_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Multiply
static int32_t (DNNE_CALLTYPE_CDECL* Export_Multiply_ptr)(intptr_t calculator, double value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Multiply(intptr_t calculator, double value)
{
    if (Export_Multiply_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Multiply");
        Export_Multiply_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, double value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Export_Multiply_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Divide
static int32_t (DNNE_CALLTYPE_CDECL* Export_Divide_ptr)(intptr_t calculator, double value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Divide(intptr_t calculator, double value)
{
    if (Export_Divide_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Divide");
        Export_Divide_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, double value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Export_Divide_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#if (defined(DNNE_WINDOWS))
// Computed from Experimental_Calculator.Export.Power
static int32_t (DNNE_CALLTYPE_CDECL* Export_Power_ptr)(intptr_t calculator, int32_t value);
DNNE_EXTERN_C DNNE_API int32_t DNNE_CALLTYPE_CDECL Export_Power(intptr_t calculator, int32_t value)
{
    if (Export_Power_ptr == NULL)
    {
        const char_t* methodName = DNNE_STR("Power");
        Export_Power_ptr = (int32_t(DNNE_CALLTYPE_CDECL*)(intptr_t calculator, int32_t value))get_fast_callable_managed_function(t1_name, methodName);
    }
    return Export_Power_ptr(calculator, value);
}
#endif // (DNNE_WINDOWS)

#endif // DNNE_COMPILE_AS_SOURCE

Latest commits contain a new Generator which is help full for creating new generators as it writes the collected data to an XML file - sadly, a few properties had to be ignored as they gave errors regarding Hexadecimal values.
Maybe you can shed some light on something, but how can I get the following attribute value?

I can get the UnmanagedCallersOnly but I am interrested in the return: MarshalAs for my clarion code (this is just a test example) - as I want to change certain the used return type - or wrap it, based on that attribute (or I'll create a custom attribute for it) but I can't find any way to access it - the current GetCustomAttributes does not return it, it seems

@ThaDaVos I am about to be on vacation for a few weeks. I'll try to check this out at some point. Overall I like the direction and I think your suggestion about adding the class to the export names is a good start, but I think it needs to include the namespace too. I agree it should be an option and not the default, you already have that. I would rename the new property, DnneWrapWithClasses, to DnneFullyQualifyExport. Please add an example in the comments. Something simple like MethodName if false, Namespace_Class_MethodName if true.

First of, have a nice vacation @AaronRobinsonMSFT - there are more changes coming - I'll commit them as soon as possible - but in short:

The changes I did for the generators, I also did regarding the custom Attributes (so you can have attributes per language) - I called them Attributors
The DnneAdditionalGenerators supports * now to select all generators
Combining the seperate ArgumentTypes and ArgumentNames into a single struct and list

Maybe we should convert this to a discussion?

@AaronRobinsonMSFT
Something cool I just figured out - you can remove the:
<Platforms>x86</Platforms>
<PlatformTarget>x86</PlatformTarget>
<RuntimeIdentifier>win-x86</RuntimeIdentifier>

And use only
<DnneRuntimeIdentifier>win-x86</DnneRuntimeIdentifier>

This will make your Library DLL be AnyCPU but the DNNE native library be targeted to win-x86 - it works for me in Clarion - also, it seems to be easier to deal with

I've got XSD generation working also - needed two changes, one is the above change so the DNNE.dll and project.dll is the same and second I had to increase the target of DNNE to dotnet7 - because I was getting System.Runtime not found errors

second I had to increase the target of DNNE to dotnet7

Which part of DNNE? An important goal here is to ensure support for all currently supported .NET runtimes.

there are more changes coming - I'll commit them as soon as possible - but in short:

If you are thinking of adding this logic to DNNE officially it is probably a good idea to start a PR so we can review the changes. Even the first past of changes from some time ago contained a substantial number of changes that will be difficult to review if the PR gets too big.

Which part of DNNE? An important goal here is to ensure support for all currently supported .NET runtimes.

I increased the dotnet version for dnne-pkg and dnne-gen - this was needed else the Assembly of the compiled managed binary cannot be loaded for XSD (as you need to extract the types).
Maybe, if you want to support all currently supported .NET Runtimes - the package needs to be made for each version if you want to support stuff like loading the Assembly in a generator

If you are thinking of adding this logic to DNNE officially

Some of the logic would be great to have added to DNNE officially to ease the addition of generators and logic for other programming languages - my current version gave me a great insight in how it works and how we could restructure the project to allow this - but it may be better if I did it from a clean fork and not include language specific implementations - but maybe we first have to discuss which pattern we want to allow for extending DNNE.
We could take a look at how Fody is setup (the configuration file way maybe easier for people too)

@ThaDaVos I'm going to close this for now. If you decide to integrate your changes we can discuss strategy in a PR.

@AaronRobinsonMSFT  - sorry for commenting on a closed ticket, still playing around with a fork - but got a question, I see you used the low-level MetaData reader, the thing is, documentation on it isn't that highly available - would it be a good option to convert to something higher level, to ease extending?
https://learn.microsoft.com/en-us/dotnet/standard/assembly/inspect-contents-using-metadataloadcontext
Or would I be better of creating my own new package for that?

the low-level MetaData reader, the thing is, documentation on it isn't that highly available

The Metadata is an officially documented specification and the API follows that specification precisely. I would avoid trying to abstract away the metadata details as they aren't always expressed precisely in the higher level APIs you're point out.
Is there some aspect of Metadata that is causing you grief? I'm happy to help elucidate any areas of confusion.

The thing which is causing me grief - is getting the values of const fields - the only thing I can find is something about a blobreader

Got the retrieval of values working, currently trying to figure out what the TGenericContext is used for and what it should be - can't find any documentation about it and only see it used in the System.Reflection.Metadata.ISignatureTypeProvider and second generic type

The thing which is causing me grief - is getting the values of const fields - the only thing I can find is something about a blobreader

Yes, Blobs fully defined in ECMA-335. They are typically strings, byte arrays, primitives, or Guids. This requires using a Blob index into the Blob table and decoding the bytes appropriately.

currently trying to figure out what the TGenericContext is used for and what it should be

The ISignatureTypeProvider is used for participating in a callback mechanism for parsing. For DNNE and most other interop related scenarios this is likely unnecessary. You can look at the use of ISignatureTypeProvider in DNNE for converting .NET types into C99 types.
https://github.com/AaronRobinsonMSFT/DNNE/blob/3e39b7ca22dea40c4346be7e94810248ab402b38/src/dnne-gen/Generator.cs#L981

@AaronRobinsonMSFT - I checked your C99TypeProvider, just trying to understand how it works and what everything means, that's why i asked what the TGenericContext generic parameter is for, as you're using an empty class in there - and I was curious as to why?
Also, one thing I think is a bit stupid - is that the ISignatureTypeProvider doesn't allow one to specify the type to be returned separatly from the type provided - like you can only say string - so you get a string and have to return a string - you can't say, "Give me a string and I'll return a type or an enum value` or something like that

you're using an empty class in there - and I was curious as to why?

Because I need to provide something to handle the generic context that may be passed around. Since I am not using it, it can be an empty class. I recommend debugging through it with some signatures to get a sense of the semantics.

I've got everything working except the Generic part - would really love to find a way to resolve the generic parameter - but the only parameters one seems to get  are the Generic Context and the index of the parameter - but no single reference to the method and such - so I am guessing the Generic Context needs methods or something to make this work - but I cannot find anything about needing to implement an interface - you've got any idea @AaronRobinsonMSFT ?

@ThaDaVos basically you need to define your own type that contains the generic context for your scenario (ie, generic arguments for the containing class and the method). You'd instantiate your type using the typeDef and methodDef objects available around here:
https://github.com/AaronRobinsonMSFT/DNNE/blob/3e39b7ca22dea40c4346be7e94810248ab402b38/src/dnne-gen/Generator.cs#L214
Then when parsing the signature, you pass your object in here:
https://github.com/AaronRobinsonMSFT/DNNE/blob/3e39b7ca22dea40c4346be7e94810248ab402b38/src/dnne-gen/Generator.cs#L224
Then, your object will be passed back to the methods as the signature is decoded, enabling you to look up the generic types.
System.Reflection.Metadata allows you to provide your own representation for the generic context as it has no requirements internally.

Thanks for the info @jkoritzinsky  - but I don't see how to helps with resolving the Generic parameters, as the TypeResolver itself, in it's callbacks, doesn't supply the needed info - only the TGenericContext and an Index:

Which results in public string GetGenericInstantiation(string genericType, ImmutableArray<string> typeArguments) having as typeArguments parameter containing a value of NOT_IMPLEMENTED - which means, to resolve the typeArguments i need to somehow figure them out in above mentioned methods.
For example, in my fork, I have the following debugs logs for the following type (Using VS Code Logpoints):
GetGenericMethodParameter | Requested a generic method parameter on index: 0
2
GetTypeFromReference | KnownType: "CLASS" | Type: 18
GetGenericMethodParameter | Requested a generic method parameter on index: 0
GetGenericInstantiation | KnownType: "CLASS" | Type: "CLASS" | KnownArgs: "UNKNOWN" | args: "NOT_IMPLEMENTED"
GetGenericInstantiation | KnownType: "CLASS" | Type: "CLASS" | KnownArgs: "CLASS" | args: "CLASS"
GetTypeFromReference | KnownType: "VALUETYPE" | Type: 17
	Method[NOT_IMPLEMENTED]: Run

The type inspected:
public TResult Run<TResult>(Func<Task<TResult>> taskFunc, CancellationToken token = default(CancellationToken))
{
}

See my fork: https://github.com/ThaDaVos/DNNE/blob/feature/refactor-of-assembly-reader/src/dnne-gen/assembly/AbstractSignatureTypeProvider.cs

You can update the AbstractTypeResolver type in your fork to require that TGenericContext implements some interface and then you can have those methods call the methods on the generic context object to determine how to build the string signature.
With System.Reflection.Metadata (which DNNE is built on) you need to do most of the work yourself.
DNNE here doesn't need the generic context as it doesn't support generics, so it doesn't need to add constraints or do anything here.
If you're adding generics support, then you need to wire up these pieces to work.

@jkoritzinsky  - thanks for the insight, but I think my question/goal is not clear to you - I am trying to implement the Generic support, but I cannot find any guidance on how to do it - no explanations or spec regarding it - the only thing I have, is the order of the methods being called in, per method it's trying to decode - which doesn't help at all, as the order isn't logical.
But I'll add a test project to the repo, which can be used for testing the dnne-gen part on, this way I won't have many types to look through and it may be easier to see what's happening - as currently I was testing it against a library I already have written.


Upgrade to z2jh 4

Fixes https://github.com/2i2c-org/infrastructure/issues/5209

I tested this on staging and it worked. Then I tried testing it with profile options and GitHubOAuthenticator and ran into some issues.


Update examples, templates, docs with config key

User-facing materials should reflect the config key declared in project config. The Pulumi YAML reference should point to the project file for information on declaring config.

[x] pulumi/examples
[ ] pulumi/templates
[ ] pulumi/docs

Going to close this as duplicate as we have individual tickets

https://github.com/pulumi/pulumi-yaml/issues/404
https://github.com/pulumi/examples/issues/1304
https://github.com/pulumi/templates/issues/453


Media new mutation

Description of changes
Use new mutations for creating a media contents and linking records.
Currently, that does not remove deselected records. The backend needs to update the mutation or provide a new one for this.

See #108
How has this been tested?
Please describe the test strategy you followed.

[ ] automated unit test
[ ] automated integration test
[ ] automated acceptance test
[ ] manual, exploratory test

In case of manual test, please document the test well including a set of user instructions and prerequisites. Each including an action, it's result, and where appropriate a screenshot.
Checklist before requesting a review

[x] My code follows the coding guidelines of this project
[x] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] My code fulfilles all acceptance criteria
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] I have made corresponding changes to the documentation
[ ] I have added explanation of architectural decision and rationales to wiki/adr
[ ] I have updated the changes in the ticket description

Checklist for reviewer

[ ] The code works and does not throw errors
[ ] The code is easy to understand and there are no confusing parts
[ ] The code follows the coding guidelines of this project
[ ] The code change accomplishes what it is supposed to do
[ ] I cannot think of any use case in which the code does not behave as intended
[ ] The added and existing tests reasonably cover the code change
[ ] I cannot think of any test cases, input or edge cases that should be tested in addition
[ ] Description of the change is included in the documentation

In what form is this different to the PR of @v-morlock connected to issue #85 ?


Just Dance 2024

How can I make a pairing in Just Dance 2024

See #31. JD2023 and JD2024 use the same app.


getting error while creating the acs cluster

Describe the bug
error: unknown option `--orchestrator-type
To Reproduce
azure acs create --name Clustername --orchestrator-type=DockerCE --resource-group resourcename --generate-ssh-keys
Expected behavior
will create a cluster
Do correct me if i am wrong

If this is a bug, please reopen, providing all of the information requested in the issue template.


New term requests for microtubule nucleation

After minor rearrangements
https://github.com/geneontology/go-ontology/issues/12679
mictotubule nucleation for eMTOC PMID:19001497

I think the new term above would be 'microtubule nucleation from eMTOC'.
We can also use 'cytoplasmic microtubule nucleation'.

yes "from' (typo)

GO:0051415	interphase microtubule nucleation by interphase microtubule organizing center
would become a descendant of
GO:0051415	interphase microtubule nucleation by interphase microtubule organizing center
(in case this is confusing rename this term to
GO:0051415 cytoplasmic microtubule nucleation by interphase microtubule organizing center)
But we do require the new parent term because cytoplasmic microtubules can be nucleated  i) in other ways, and ii) during M-phase)

Hi @dosumis  this one looks straight forward. could you do, or let us know  if you require any more info from us?
Val


Allow pulling / pushing to repositories with existing content

Is your feature request related to a problem? Please describe.
Unable to publish or pull from repositories with existing content on initial setup of Git Sync in Insomnia Designer. When attempting to connect to a repository that already contains content such as a readme, a user will run into the error dialog detailing that they cannot pull or push depending on the action taken.
Describe the solution you'd like
Configure and publish to a repository with existing content.
Describe alternatives you've considered
Currently, the only way to connect git sync is to use a completely clean repository, or an existing insomnia designer repository.
Additional context

Hi folks, any update on this?
My current workaround for syncing API spec with an existing repo (using github):

Import spec via file
Create a temp repo
Setup git sync in Insomnia Designer with new repo
Commit and push to temp repo
Clone temp repo
Grab .insomnia dir from temp repo and move to your existing repo
Commit and push in existing repo
Open Insomnia designer and create new spec by importing from existing repo

^ I'm sure there's a simpler/quicker way, but this worked first try 🤷

Hi, any news regarding this?
Also I encountered something related which might be related:
I started with an existing repo but with a brand new branch (didn't know of the issue above), commit, push. Everything went fine at the time.
Then I recently (1 month later) switched to my regular /develop branch where I have the rest of the project (Java code, and I use Eclipse IDE) so my repo now includes some hidden files/folders like .project or .settings or "target".
Now switching back to my "spec-only" branch these files & folders are considered new & unstaged because there isn't any .gitignore file.
I managed to commit new changes I made to my OpenAPI spec file but I don't see the commit, and I can't even push because I get this error:


Refactor server-side blazor startup to allow Azure SignalR. Fixes #1227

Fixes #1227
The main challenge with this is:
(1) We don't want to take a strict dependency on the Microsoft.Azure.SignalR package (at least not for everyone)
(2) The Microsoft.Azure.SignalR package uses its own custom equivalent to MapHub that looks the same as the standard Microsoft.AspNetCore.SignalR one, but actually uses different types. So if we comply with (1), then we can't possibly call MapHub on the developer's behalf.
Possible resolutions, then:

A: Use reflection tricks and make it more magic (no thanks)
B: Have a separate Microsoft.AspNetCore.Blazor.AzureSignalR package that depends on Microsoft.Azure.SignalR
C: Let the developer call MapHub themselves, meaning that (a) we have to remove all the other magic from UseServerSideBlazor and (b) we have to make BlazorHub public.

Between B and C, I prefer option C because it makes us less tightly coupled to SignalR APIs, so if SignalR adds more overloads to MapHub or other config styles in the future, we should automatically be composable with it.
This PR implements options C.
Result
The result is that app.UseServerSideBlazor<TClient>(); is now exactly equivalent to:
app.UseSignalR(route => route.MapHub<BlazorHub>(BlazorHub.DefaultPath));
app.UseBlazor<TClient>();

So, if a developer wants, they can choose not to call UseServerSideBlazor, but rather use those two lines instead, and from there they can replace UseSignalR with UseAzureSignalR if they want.
Therefore the way to use Azure SignalR with a server-side Blazor app becomes (in the server app's Startup.cs):
    public class Startup
    {
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddSignalR().AddAzureSignalR(/* optionsOrConnectionString */);
            services.AddServerSideBlazor<Client.Startup>();
            // ...
        }

        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            app.UseAzureSignalR(route => route.MapHub<BlazorHub>(BlazorHub.DefaultPath));
            app.UseBlazor<Client.Startup>();
            // ...
        }
    }

Note: This means that AddSignalR is being called twice - once by the developer manually, and once inside AddServerSideBlazor. @anurse, do you know if there's any problem or disadvantage with calling that twice? We can stop it if we absolutely must, but it will be easier for developers to understand this way (because it works both with and without Azure SignalR with a minimum of user code changes).
More notes, mainly for @rynowak
I moved all the "register startup actions" logic out of UseServerSideBlazor and into AddServerSideBlazor (so that UseServerSideBlazor can be equivalent to the two lines above). I believe the net result is the same, and it still can be extended to support registering multiple server-side Blazor endpoints in the future (with different TClient values), but for now I still left it hard-coded to use /_blazor as the path without implementing support for configuring an alternate path.

@anurse, do you know if there's any problem or disadvantage with calling that twice?

I do not believe this will cause a problem. We use TryAdd to register things in DI so the second time should just no-op (other than giving you an ISignalRBuilder to register additional stuff).

I spoke a little too soon. There are two things that will be duplicated in the DI container if both you and the user call .AddSignalR. But I think we're OK:

SignalRMarkerService - This is just used to detect that you've called AddSignalR when you call UseSignalR (like MVC), duplicates here should be fine.
HubOptionsSetup - This establishes the defaults for IOptions<HubOptions>. Two instances of this should be fine, it'll just mean we set the options twice.

I'll file a bug to make those TryAdd, there's not really a reason they can't be. But I think this change can go forward without changing them since there isn't really a negative impact to the doubling up.

@anurse - if you want to nuke this kind of problem from orbit, consider doing something like https://github.com/aspnet/Mvc/blob/release/2.2/test/Microsoft.AspNetCore.Mvc.Test/MvcServiceCollectionExtensionsTest.cs#L54
We unit test this in MVC, and it's a little annoying to maintain, but we know for sure that everything we do only happens once 😬

MVC doesn't have the ability to 'discover' public things for much the same reason signalR doesn't. I think there's a strong case to be made for 'discovery' to be based on public visibility, but manual wireups to support any visibility.


Slow zoom animation in Chrome when using more than one tile layer

Hi,
I experience a weird behavior with a WMS layer (on top of the OSM layer) .  Using Chrome, if I zoom 2 times in row or pan then zoom quickly, the processor goes to 100% for many seconds (over 10 sometimes) until it come back.   Chrome event ask me to kill the page after a while.  But on both firefox and safari, the speed is fine.
It's tested it with the latest leaflet.js and a version from 2 weeks ago and it's the same.
Any idea why ?
Jeff
If it makes a difference, i'm running on OS X 10.6.

Hi Guys,
After searching a lot about slow tile on zoom, i came to this link.
I am using the latest version of the leaflet, but facing issues with the zoom effect.  I have multiple layers on the map, and getting the slow zoom in leaflet.

The map tiles and wms layer tiles are taking more time to load when we zoom on it.

Please  let me know i am beginner in leaflet.

If I understand correctly, it is not a Leaflet issue.
Most probably it's because of parallel http connections limit in browser.
Please note that this issue tracker is used for bugs. For support or questions please use Stack Overflow or gis.stackexchange.com.
If you really sure it's Leaflet issue, please fill up issue template and provide link to minimal example reproducing the issue.


[QUESTION] How can I use response body path in JSON request body

What is your question?
I'm building a POST request with a JSON request body.
Trying to use { {Response}} + control + space doesn't do anything. I can use ENV variables fine, but what's the point of only parsing and using the response values in query params only?!?!
What is the context?

Copying from a query into JSON also doesn't work

Hi @ortonomy! It still needs to be valid JSON so you need to wrap the template tag in double-quotes:
{
  "image": "{% response 'raw', ... %}"
}

Your current implementation is:
{
  "image": {% response 'raw', ... %}
}

@develohpanda -- doesn't help, thanks. the response from the server is JSON itself. I need it to be an object. Putting inside double quotes makes it a string.
MOREOVER why do I have to create this template string in the address bar of the URL and then copy it over?!?!

MOREOVER why do I have to create this template string in the address bar of the URL and then copy it over?!?! Seems like the template tag builder should work in JSON body?

Why so hostile? It's against the repos Code of Conduct, please stop.
Also, you can, it's done through a hotkey - CTRL+Space.
Not sure that this is supported, you can do it with a plugin though - but it's a bit more. Perhaps we will see the support of before and after scripts like other tools! That said, I haven't had much of a need for this though.

MOREOVER why do I have to create this template string in the address bar of the URL and then copy it over?!?! Seems like the template tag builder should work in JSON body?

Why so hostile? It's against the repos Code of Conduct, please stop.
Also, you can, it's done through a hotkey - CTRL+Space.
Not sure that this is supported, you can do it with a plugin though - but it's a bit more. Perhaps we will see the support of before and after scripts like other tools! That said, I haven't had much of a need for this though.

@apisaint -- this isn't hostile. Replies should be reserved for being helpful and bringing answers.

Also, you can, it's done through a hotkey - CTRL+Space.

This is not helpful. I wouldn't have opened an issue if this was working. I actually think I've discovered a bug --
In POST request, the ctrl + space works fine:

but in a PUT request, it fails completly:

@apisaint -- this isn't hostile. Replies should be reserved for being helpful and bringing answers.

Also, you can, it's done through a hotkey - CTRL+Space.

This is not helpful. I wouldn't have opened an issue if this was working. I actually think I've discovered a bug --
In POST request, the ctrl + space works fine:

but in a PUT request, it fails completly:

The way you communicate is against the repositories code of conduct. I would advise to stop.
Nice, then it looks like it was helpful :) You also learned a new hotkey! Now you don't have to do it in the url bar anymore - except for PUT - haha.

The way you communicate is against the repositories code of conduct. I would advise to stop.
Nice, then it looks like it was helpful :) You also learned a new hotkey! Now you don't have to do it in the url bar anymore - except for PUT - haha.

The way you communicate is against the repositories code of conduct. I would advise to stop.

I would advise you to stop with this. It's just noise and hot air.

You also learned a new hotkey!

I didn't learn it, I already knew it. It didn't work before in PUT requests. I hadn't tried it in POST. I advise you stop being patronising. This should probably be against the code of conduct, too.

The way you communicate is against the repositories code of conduct. I would advise to stop.

I would advise you to stop with this. It's just noise and hot air.

You also learned a new hotkey!

I didn't learn it, I already knew it. It didn't work before in PUT requests. I hadn't tried it in POST. I advise you stop being patronising. This should probably be against the code of conduct, too.

I would advise you to stop with this. It's just noise and hot air.

Please read the code of conduct.

I would advise you to stop with this. It's just noise and hot air.

Please read the code of conduct.

Please read the code of conduct.

Please leave.

Please read the code of conduct.

Please leave.

Following up on the items not working for the maintainers of this repository - I could not replicate. Tried on a fresh request, on an existing request, and on a request where I converted from POST to PUT. Since Ortonomy didn't provide replication steps I tried all that I could think of.
I tried with Insomnia Core and Insomnia Designer (versions - develop, 2020.5 and 2020.4)

Following up on the items not working for the maintainers of this repository - I could not replicate. Tried on a fresh request, on an existing request, and on a request where I converted from POST to PUT. Since Ortonomy didn't provide replication steps I tried all that I could think of.
I tried with Insomnia Core and Insomnia Designer (versions - develop, 2020.5 and 2020.4)

Update. I have found a way to replicate the issue, not sure if intentional, seems to be though. Haven't looked through the code yet.

Update. I have found a way to replicate the issue, not sure if intentional, seems to be though. Haven't looked through the code yet.

Second update, for ortonomy. This is an upstream issue from Codemirror, it's intentional, while inside a string or comment the hint menu hotkey will not trigger. To get it to trigger, the maintainers must write their own hint menu helper for each language they wish to support. Unfortunate.
See more details in #2981

Second update, for ortonomy. This is an upstream issue from Codemirror, it's intentional, while inside a string or comment the hint menu hotkey will not trigger. To get it to trigger, the maintainers must write their own hint menu helper for each language they wish to support. Unfortunate.
See more details in #2981

Ummm, no. Here's me, smashing ctrl + space outside of a string, in a JSON body

Ummm, no. Here's me, smashing ctrl + space outside of a string, in a JSON body

Ummm, no. Here's me, smashing ctrl + space outside of a string, in a JSON body

Interesting. Does the console log show anything? View -> Toggle DevTools (CMD+Option+I)
For reference, same thing (wanted to make sure that I did nested as well). Maybe I could try the same JSON body that you have too.

Ummm, no. Here's me, smashing ctrl + space outside of a string, in a JSON body

Interesting. Does the console log show anything? View -> Toggle DevTools (CMD+Option+I)
For reference, same thing (wanted to make sure that I did nested as well). Maybe I could try the same JSON body that you have too.

I was actually wrong, there is an autocomplete hint helper for Insomnia - looking through it now. Hopefully there is something in your console @ortonomy.

I was actually wrong, there is an autocomplete hint helper for Insomnia - looking through it now. Hopefully there is something in your console @ortonomy.

Nothing in dev console. It's also first place I looked.

Nothing in dev console. It's also first place I looked.

Alright, found out that autocomplete.js is controlling the dropdown, however, there are a lot of cases where it won't show up (and I am not aware of all of them - for example sometimes whenever you try when your cursor is positioned after an open square bracket [)

Alright, found out that autocomplete.js is controlling the dropdown, however, there are a lot of cases where it won't show up (and I am not aware of all of them - for example sometimes whenever you try when your cursor is positioned after an open square bracket [)

Figured out the reason @ortonomy looks like it depends on whether or not the matcher thinks that the character you're touching is worth looking up or not, and I think it'll be best to leave it up to the maintainers to decide whether or not to remove support for []{} as matching characters.
As for why it doesn't show up in strings, its trying a partial match against the word value of that string:
"World|" where your cursor is on the d does a match against World within the hint dropdown. No matches, no dropdown.

Figured out the reason @ortonomy looks like it depends on whether or not the matcher thinks that the character you're touching is worth looking up or not, and I think it'll be best to leave it up to the maintainers to decide whether or not to remove support for []{} as matching characters.
As for why it doesn't show up in strings, its trying a partial match against the word value of that string:
"World|" where your cursor is on the d does a match against World within the hint dropdown. No matches, no dropdown.

Thanks for going to such lengths @apisaint - but it's not just at the boundaries of the chars {[]} - I can 'ctrl'+'space' click anywhere in that sample JSON and it just won't show the prompt to insert response info. Sounds like this is definitely a partial reason. I may go back to postman - it's relatively much easier to save response info and pass it back in to a request.

Thanks for going to such lengths @apisaint - but it's not just at the boundaries of the chars {[]} - I can 'ctrl'+'space' click anywhere in that sample JSON and it just won't show the prompt to insert response info. Sounds like this is definitely a partial reason. I may go back to postman - it's relatively much easier to save response info and pass it back in to a request.


Greet system

this is a voluntary job, please assign yourself if you want to work with it
TODO :

[x] A Greet Disable/Enable which is unique for each server (yes, this is not a global command and more guild-specific)

[x] add a way to automatically detect (and disable the greet system) on Discord Developers' Servers (Discord API, Discord Bots and Discord Developers)

you can whitelist a bot collection server too if you wish.

:thinking: do we really want this

Added in 60329f1


Implement file sharing

We need a way to share files/directories with the crc VM. The first use case would be for the podman preset, to implement podman-remote -v localdir:remotedir.
There are several options:

virtiofs
sshfs

The way podman does it is by mounting (on a mac) /Users/teuf to /Users/teuf inside its VM using 9p. Then podman-remote -v /Users/teuf/localdir:remotedir can be used successfully.
I've added virtiofs support to vfkit in https://github.com/cfergeau/vfkit/tree/virtiofs , though this depends on an unmerged PR in Code-Hex/vz
This is then used by https://github.com/cfergeau/crc/commits/macos-vf (various paths/tags are hardcoded at the moment).
With these changes, I can start a VM using the podman bundle, and /Users/teuf is mounted to the same path in the VM. However /Users/teuf in the VM is owned by root.root.  Because of this, podman -v /Users/teuf/localdir:remotedir fails as rootless podman can't access /Users/teuf.
I did not have time to dig around mount options/... to understand if we can tweak this behaviour.

podman-machine has a trick to access /Users/teuf. Through 9p,  /Users/teuf is owned by a user with the same uid as I have on my mac. When podman-machine creates its VM, its ignition file creates a core user with the same uid as the one teuf has locally on the mac. This eases some of the permissions/ownership issues that might occur.

closing this more updates in the epic


mark: 5.4 -> 5.5

Motivation for this change
https://github.com/kovetskiy/mark/releases/tag/5.5
❯ ./result/bin/mark --version
5.5

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Closing in favor of #118074


Resizimg plugin bug. After page refreshes, the resizing image doesn't show

Image resizing works perfectly before. I find the img tag in it. After I save html code generated from the image resizing & refresh the page to show the resize image. The resizing image doesn't show. I can't find the img tag, instead I find a blank canvas tag.
OS: Mac
Browser: Chrome
Link: http://rmgvirgo.sangnilaindonesia.com/Faq/edit/1
I'm using Bootstrap & Jquey UI

Hi @riskadewi,
I think this issue is described as well here : Issue #1158
May be this issue can be marked as duplicated.

ok. will check it later. thank you

Should be fix in v2.22 (will be released soon)


EI 6.5.0 docker images error - Unable to find JDBC when updating the docker image from <IP_ADDRESS> to <IP_ADDRESS> all docker alpine images

Description
Unable to find JDBC when updating the docker image from <IP_ADDRESS> to <IP_ADDRESS> all docker alpine images.
activator cannot start user manager core bundle
Caused by: javax.naming.NameNotFoundException: Name [jdbc/WSO2UMDB] is not bound in this Context. Unable to find [jdbc].
at org.apache.naming.NamingContext.lookup(NamingContext.java:824)
at org.apache.naming.NamingContext.lookup(NamingContext.java:173)
at org.wso2.carbon.context.internal.CarbonContextDataHolder$CarbonInitialJNDIContext.lookup(CarbonContextDataHolder.java:709)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at javax.naming.InitialContext.doLookup(InitialContext.java:290)
at org.wso2.carbon.user.core.util.DatabaseUtil.lookupDataSource(DatabaseUtil.java:89)
Steps to Reproduce
It seems that something went wrong with the EI 6.5.0 docker images and all level are failing while starting with or without customised configuration with the above error message.
Affected Component
Analytics
Version
6.5.0
Environment Details (with versions)
No response
Relevant Log Output
No response
Related Issues
No response
Suggested Labels
No response

This issue is only reproducing in U2 updated packs. Hence closing the issue.


[AbortError] And and check AbortError while abort

Fix: https://github.com/grpc/grpc/issues/33890
In case of abort, currently we don't log anything, this change added an AbortError as the default error for abort, if any other error happened, we'll log the error so user will be aware of the other error.
Testing

Tested locally, after this change, any non-AbortError will be logged, sample log:

ERROR:grpc._server:Exception happened while abort: Other exception happened!
Traceback (most recent call last):
  File "/usr/local/google/home/xuanwn/.cache/bazel/_bazel_xuanwn/da3828576aa39e99a5c826cc2e2e22fb/sandbox/linux-sandbox/9/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/src/python/grpcio_tests/tests/unit/_abort_test.native.runfiles/com_github_grpc_grpc/src/python/grpcio/grpc/_server.py", line 553, in _call_behavior
    response_or_iterator = behavior(argument, context)
  File "/usr/local/google/home/xuanwn/.cache/bazel/_bazel_xuanwn/da3828576aa39e99a5c826cc2e2e22fb/sandbox/linux-sandbox/9/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/src/python/grpcio_tests/tests/unit/_abort_test.native.runfiles/com_github_grpc_grpc/src/python/grpcio_tests/tests/unit/_abort_test.py", line 95, in abort_with_status_unary_unary_raise_additional_exception
    raise Exception("Other exception happened!")
Exception: Other exception happened!

TODO

Create gRFC.

gRFC for adding new errors: https://github.com/grpc/proposal/pull/388

Discussed offline. It probably makes sense to share these between the sync stack and the asyncio stack so that regardless of why stack you're doing you can just do:
try:
  ...
except grpc.BaseError as e:
  ...

@gnossen PTALA


Space under dynamic table

I am using one dynamic table, populated with some webservices. The issue is that there is a huge space between my table and footer.
Do you guys experience this?

Hi @xredpanda ,
That happens on dynamic tables only or in other pages to, i'm saying this because on my side it seems to behave just fine.

How do you solve this problem? I'm with the same problem! Thank's

Hi, I remember having this problem on one of my page, wasn't using DataTables as OP though, and seems to be a problem on th speed of the rendering so as a workaorund I just called again init_sidebar() again after I knew some elements where, the most important ones, loaded.
So this is just a workaround this seems to happen because init_sidebar() is called first than the other elements, in the case of DataTables the data is loaded first and the hided to do the pagination so maybe that's the reason.


parallel-r.md: No need to set options(future.availableCores.methods = "Slurm")

Hello.  Author of the future and parallelly packages here.  In https://github.com/CSCfi/csc-user-guide/blob/master/docs/support/tutorials/parallel-r.md you suggest:
library(future)
options(future.availableCores.methods = "Slurm")
availableCores()

It should not be necessary to set that option; availableCores() defaults to availableCores(which = "min"), which means it will return the smallest number of cores specified by any of the known settings.  This means that if detectCores() reports 40 but the job request 8 cores, then 8 will be returned.
FWIW, I just use availableCores() as-is myself on both a Slurm cluster and on an SGE cluster.

I tested this now with my geo examples: https://github.com/csc-training/geocomputing/tree/master/R/puhti/05_parallel_future
The multicore version detected available cores correctly also without the options row, BUT the cluster option got it wrong, it thought it had only one core, although I had reserved 4.
Knowing all the tiny internal problems with R installation at CSC, I would may-be rather leave the options row there. I guess it does not do any harm?

Knowing all the tiny internal problems with R installation at CSC, I would may-be rather leave the options row there.

This should have nothing to do with your R installation.  parallelly::availableCores() queries the system settings, including Slurm environment variables and the system's Linux CGroups setting (nproc).  If it's a bug in parallelly, then I'd like to fix it.
It could also be that you're actually only get a single CPU core do to CGroups settings and that you need to request the Slurm CPU resources in a different way (I've seen that "mistake" happen elsewhere).

I guess it does not do any harm?

If CGroups, which is configured by Slurm and the OS, limits you to a single CPU core, but you still try to run as you have 4 cores, your job will run much slower.  It gets worse the more you overuse the system.
Can you please show the sbatch call and the script you use when submitting to Slurm?  If you can include
echo "CGroups:"
nproc

echo "SLURM environment variables:"
env | grep SLURM

in your job script and
print(parallelly::availableCores(which = "all")

in your R script, I can try to see what's going.

I hope this is what you requested?
CGroups:
4
SLURM environment variables:
SLURM_MPI_TYPE=pmix_v3
SLURM_MEM_PER_CPU=1000
SLURM_NODEID=0
SLURM_TASK_PID=699502
SLURM_PRIO_PROCESS=0
SLURM_SUBMIT_DIR=/scratch/project_2002044/ekkylli/geocomputing/R/puhti/05_parallel_future
SLURM_PROCID=0
SLURM_JOB_GID=10004642
SLURMD_NODENAME=r07c01
SLURM_TASKS_PER_NODE=4
SLURM_NNODES=1
SLURM_JOB_NODELIST=r07c01
SLURM_CLUSTER_NAME=puhti
SLURM_NODELIST=r07c01
SLURM_NTASKS=4
SLURM_JOB_CPUS_PER_NODE=4
SLURM_TOPOLOGY_ADDR=r07c01
SLURM_THREADS_PER_CORE=1
SLURM_WORKING_CLUSTER=puhti:slurm.master.bullx:6817:9472:109
SLURM_JOB_NAME=r_multi_proc
SLURM_JOBID=15767472
SLURM_CONF=/etc/slurm/slurm.conf
SLURM_NODE_ALIASES=(null)
SLURM_JOB_QOS=normal
SLURM_TOPOLOGY_ADDR_PATTERN=node
SLURM_CPUS_ON_NODE=4
SLURM_JOB_NUM_NODES=1
SLURM_JOB_UID=10007378
SLURM_JOB_PARTITION=test
SLURM_HINT=nomultithread
SLURM_JOB_USER=ekkylli
SLURM_NPROCS=4
SLURM_SUBMIT_HOST=puhti-login12.bullx
SLURM_JOB_ACCOUNT=project_2002044
SLURM_GTIDS=0
SLURM_JOB_ID=15767472
SLURM_LOCALID=0
system  nproc  Slurm
    40      1      4
$mc.cores
character(0)

$BiocParallel
character(0)

$`_R_CHECK_LIMIT_CORES_`
character(0)

$LSF
character(0)

$PJM
character(0)

$PBS
character(0)

$SGE
character(0)

$Slurm
[1] "r07c01" "r07c01" "r07c01" "r07c01"

$custom
character(0)

$system
 [1] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
 [7] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
[13] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
[19] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
[25] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
[31] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
[37] "localhost" "localhost" "localhost" "localhost"

$fallback
character(0)

   user  system elapsed
 11.318   0.007  11.615

Thanks. And does your job script look like, and what is the full sbatch call?
FYI,
the added nproc reports:
CGroups:
4

but according to availableCores(which = "all") it's 1;
system  nproc  Slurm
    40      1      4

I hope there's a simple explanation for this.

Files are here: https://github.com/csc-training/geocomputing/tree/master/R/puhti/05_parallel_future

Okay, then I assume you submit it as:
sbatch parallel_batch_job_future_cluster.sh

Do you have info on what:

apptainer_wrapper
RMPISNOW

are?
Also, I don't availableCores() is used anywhere in your code, because you do:
cl<-getMPIcluster()
plan(cluster, workers = cl)

where snow::getMPIcluster() does its own thing, i.e. it does not depend on the value of availableCores().  The future framework is then just taking whatever you give it, i.e. plan(cluster, workers = cl).   If so, there's also no need to set the option either.

appatainer_wrapper is just a wrapper script to start R from container and to map some folders to be visible from container. It should not be important here.
RMPISNOW should come from snow package, http://homepage.divms.uiowa.edu/~luke/R/cluster/cluster.html, I personally do not really know the details behind, it has been just the way to run multinode R-code at CSC for 7+ years.
But if snow::getMPIcluster() ignores options(future.availableCores.methods = "Slurm") setting, then I guess it could be removed. @jessepharrison @helijuottonen opinions?

Sorry for the extremely delayed response, but we were finally able to discuss this issue properly within our team. Thank you @HenrikBengtsson for bringing the issue to our attention and for taking the time discuss it! We really appreciate it and will modify this tutorial and our documentation to remove the unnecessary line and use availableCores() only.
We have also double-checked that everything works as expected with detection and use of multiple cores in our system. The case where nproc=1 for multiple cores seems to have been because it was run within srun and there were multiple processes that each had one core. The whole job had the correct number of cores.


Add switch between access_token and id_token

Fixes #1303 in a backwards compatible manner.

Thanks for the PR!


permission denied for nyc.js

I get premiss denied (i'm on OSX)
coverage running the following command /Users/XXX/.vscode/extensions/maty.vscode-mocha-sidebar-0.20.22/node_modules/nyc/bin/nyc.js --all  --reporter=html --reporter=text --reporter=lcov  /Users/XXX/.vscode/extensions/maty.vscode-mocha-sidebar-0.20.22/node_modules/mocha/bin/mocha --require ts-node/register  **/.ts --exit
coverage running the following command /Users/XXX/.vscode/extensions/maty.vscode-mocha-sidebar-0.20.22/node_modules/nyc/bin/nyc.js --all  --reporter=html --reporter=text --reporter=lcov  /Users/XXX/.vscode/extensions/maty.vscode-mocha-sidebar-0.20.22/node_modules/mocha/bin/mocha --require ts-node/register  **/.ts --exit
sh: /Users/XXX/.vscode/extensions/maty.vscode-mocha-sidebar-0.20.22/node_modules/nyc/bin/nyc.js: Permission denied

I, too, am running into this issue.

I was able to get around this by manually changing permissions of the istanbul and mocha binaries.
cd ~/.vscode-insiders/extensions/maty.vscode-mocha-sidebar-0.20.22/
chmod +x node_modules/nyc/bin/nyc.js node_modules/nyc/bin/wrap.js node_modules/mocha/bin/mocha

added your workaround to the README
I couldn't reproduce it on my pc  maybe it related to macOS
so if you have more information it could be really helpful


[BUG] Creating a Utility Files Block With An Empty Input Causes  LoliCodeParsingExpection

Describe the bug
Creating a Utility Files Block With An Empty Input Causes  LoliCodeParsingExpection
To Reproduce
Steps to reproduce the behavior:

Create a Utility Files block
Click on lolicode
Try switching back to stacker
See error

Expected behavior
An empty input no error
Screenshots
https://i.imgur.com/x4hLQxi.png
https://i.imgur.com/FDIqEuJ.png
Desktop (please complete the following information):

OS: Windows 10
Browser Chrome
Version 89.0.4389.90 (Official Build) (64-bit)

I decided that this behaviour is correct, the LoliCode parser should in fact throw an error if this happens because the config cannot be compiled to C# if the variable name is left empty. It's easy to fix as the parser informs the user about which setting is giving problems and the user can simply add some letters after the @ to make it work.


[Bug Report] Slider in collapse component other than the active panels can't smooth drag

Element UI version
1.4.2
OS/Browsers version
Mac os sierra(10.12.6) Chrome(60)
Vue version
2.4.2
Reproduction Link
https://jsfiddle.net/hvzeote0/3/
Steps to reproduce

click collapse panel #2 or #3
drag the button of slider

What is Expected?
slider works properly
What is actually happening?
slider does not work

Fixed in #6593