Feature: Add AMD support

Would love an AMD version of dust that works with RequireJS that kicked out AMD compiled templates as well.

+1

+1


Scaffolding tool needs update: “pipeline” field should be “tasks” in turbo.json

Verify canary release

[X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue
https://github.com/PixelHabits/turborepo-2.0-issue
What package manager are you using / does the bug impact?
pnpm
What operating system are you using?
Mac
Which canary version will you have in your reproduction?
2.0.0-canary.4
Describe the Bug
After scaffolding a new template application using pnpm dlx create-turbo@latest, running the turbo dev command results in the following error:
× found `pipeline` field instead of `tasks`
    ╭─[turbo.json:3:1]
  3 │       "globalDependencies": ["**/.env.*local"],
  4 │ ╭─▶   "pipeline": {
  5 │ │       "build": {
  6 │ │         "dependsOn": ["^build"],
  7 │ │         "outputs": [".next/**", "!.next/cache/**"]
  8 │ │       },
  9 │ │       "lint": {
 10 │ │         "dependsOn": ["^lint"]
 11 │ │       },
 12 │ │       "dev": {
 13 │ │         "cache": false,
 14 │ │         "persistent": true
 15 │ │       }
 16 │ ├─▶   }
    · ╰──── rename `pipeline` field to `tasks`
 17 │     }
    ╰────
  help: changed in 2.0: `pipeline` has been renamed to `tasks`

Expected Behavior
The turbo dev command should run without errors after scaffolding a new template application.
To Reproduce
Scaffold a new template application using:
pnpm dlx create-turbo@latest

Additional context
The issue can be fixed by changing the pipeline field in turbo.json to tasks as suggested by the command line output. The scaffolding tool should be updated to reflect this change.

I also verified the issue is still present on v.2.0.1

We're working on this here: https://github.com/vercel/turbo/pull/8284
Sorry for not having them ready to go right away! We had to wait for the npm publish to get the examples fixed up.

Should be good now! Sorry again for the wait.


Move FilteringEndpointsConfigHandler into openshift-sdn

It's currently osdn specific anyway, plus we need it here to avoid
an import loop once some of liggitt's suggestions are taken for
origin SDN initialization ordering.

See https://github.com/dcbw/origin/pull/1/files for the general direction here for origin.

@pravisankar @danwinship @rajatchopra

LGTM


Modules reviewed by @gaow

@hsun3163 this ticket keeps track of modules that I (re)reviewed : at this round I mostly make all the changes myself and try to finalize everything. Every time I finalize a module I'll post an update.  Please be very careful not to make heavy changes on codes that I finalized without discussing with me first.
Finalized codes for V 1.0 release:

GWAS QC
PCA
HG reference data preprocessing
RNA calling
Sample level RNA-seq QC
genotype data formatting
VCF QC
Normalization
Gene coord annotation

In progress by @gaow:
Pending 2nd round of test @hsun3163 -- both with MWE and with ROSMAP data

[x] GRM LOCO
[x] PEER -- works for Wenhao's MWE but need to use our MWE to connect to next steps
[x] Bi-CV -- add your old MWE

List of improvements still needed (I'll keep on adding to it):

[ ] In this folder https://www.synapse.org/#!Synapse:syn36422500 remove the stderr files


Update website to add John Wu

John: "John Wu is an Assistant Astronomer at Space Telescope Science Institute (STScI) who primarily studies galaxies with wide-area imaging surveys and machine learning methods like convolutional neural networks and graph neural networks. At STScI, he has advanced strategic initiatives in artificial intelligence and supported the upcoming Roman Space Telescope mission."

closed with #56


Split process stop from disposal and use disposal helpers

This change splits out the behavior of stopping the process runners from disposing the process runners. This allows consistent access to process information after the process has stopped but before it is disposed (which the process information, such as exit code, are no longer available). As it was before, owners of the process runner are NOT required to call StopAsync before calling DisposeAsync.
The AppRunner.ExecuteAsync extension methods have been updated to call StopAsync instead of DisposeAsync in order to maintain the symmetry with the method starting the runner rather than creating and owning the runner. Callers are now responsible for disposing the runners.
Additional changes were made to unify the disposal patterns to use the common disposal helpers.

/backport to release/7.x


Add Support for Unity 2021.3.9f1

been looking for il2cppinspector that support 2021.3.9f1 but seem the repo is unmaintaned...and no guide how to update it...looking for cpp2il that support 2021.3.9f1 thank you

which version? the latest released or build from sources?

it said
===Cpp2IL by Samboy063===
A Tool to Reverse Unity's "il2cpp" Build Process.
Version 2022.1.0-pre-release.12
[Info] [Program] Running on Win32NT
[Info] [Plugins] Loading plugins from D:\ModToramPython\Cpp2IL-2022.1.0-pre-release.12-Windows-Netframework472\Plugins...
[Info] [Plugins] Using Plugin: Cpp2IL Built-In
[Fail] [Program] Execution Failed: Could not find a valid unity game at C:\Program

put quotes around the game path

now its appear
[Fail] [Program]
Cpp2IL.Core.Exceptions.LibCpp2ILInitializationException: Fatal Exception initializing LibCpp2IL! ---> System.Exception: Failed to find pCodegenModules
at LibCpp2IL.BinarySearcher.FindCodeRegistrationPost2019() in C:\Users\Sam\RiderProjects\Cpp2IL\LibCpp2IL\BinarySearcher.cs:line 196
at LibCpp2IL.Il2CppBinary.FindCodeAndMetadataReg(Int32 methodCount, Int32 typeDefinitionsCount) in C:\Users\Sam\RiderProjects\Cpp2IL\LibCpp2IL\Il2CppBinary.cs:line 450
at LibCpp2IL.LibCpp2IlBinaryRegistry.CreateAndInit(Byte[] buffer, Il2CppMetadata metadata) in C:\Users\Sam\RiderProjects\Cpp2IL\LibCpp2IL\LibCpp2IlBinaryRegistry.cs:line 68
at LibCpp2IL.LibCpp2IlMain.Initialize(Byte[] binaryBytes, Byte[] metadataBytes, UnityVersion unityVersion) in C:\Users\Sam\RiderProjects\Cpp2IL\LibCpp2IL\LibCpp2IlMain.cs:line 159
at LibCpp2IL.LibCpp2IlMain.LoadFromFile(String pePath, String metadataPath, UnityVersion unityVersion) in C:\Users\Sam\RiderProjects\Cpp2IL\LibCpp2IL\LibCpp2IlMain.cs:line 202
at Cpp2IL.Core.Cpp2IlApi.InitializeLibCpp2Il(String assemblyPath, String metadataPath, UnityVersion unityVersion, Boolean allowUserToInputAddresses) in C:\Users\Sam\RiderProjects\Cpp2IL\Cpp2IL.Core\Cpp2IlApi.cs:line 55
--- End of inner exception stack trace ---
at Cpp2IL.Core.Cpp2IlApi.InitializeLibCpp2Il(String assemblyPath, String metadataPath, UnityVersion unityVersion, Boolean allowUserToInputAddresses) in C:\Users\Sam\RiderProjects\Cpp2IL\Cpp2IL.Core\Cpp2IlApi.cs:line 67
at Cpp2IL.Program.MainWithArgs(Cpp2IlRuntimeArgs runtimeArgs) in C:\Users\Sam\RiderProjects\Cpp2IL\Cpp2IL\Program.cs:line 487
at Cpp2IL.Program.Main(String[] args) in C:\Users\Sam\RiderProjects\Cpp2IL\Cpp2IL\Program.cs:line 429
Waiting for you to press enter - feel free to copy the error...

the other error is appeared

Try a release build not a pre-release

That usually implies your game is protected somehow.

I can say for certain that 2021.3.12f1, 2021.3.16f1, and 2021.3.26f1 all work

do you have a solution? because il2cpp inspector that usually im used is not supported 2021.3.9f1 yet

Or maybe there are other similar applications?


Apply Pedantic recommendations

Applied most health suggestions from https://pub.dev/packages/socket_io#-analysis-tab- using on Pedantic code analysis (except 3 recommended fixes, to avoid logic changes)
Code has passed by tests
All code change commits include the rule in which they were based

@guilhermecaldas Thanks for the contribution.


29 add documentation with doxygen breathe and sphinx

Closes #29
I setup several of the classes incorrectly. They should have been templated but were not. Adding the templates means that most of the files are now inherently header only so most of their source files were removed. This may be my misunderstanding of what is meant to be templated though.

Codecov Report
Base: 100.00% // Head: 100.00% // No change to project coverage :thumbsup:

Coverage data is based on head (1428e72) compared to base (8a38b5a).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@                      Coverage Diff                       @@
##           18-hard-coded-chapman-solver       #30   +/-   ##
==============================================================
  Coverage                        100.00%   100.00%
==============================================================
  Files                                 1         6    +5
  Lines                                 2        22   +20
==============================================================
+ Hits                                  2        22   +20

Impacted Files
Coverage Δ

include/micm/process/arrhenius_rate_constant.hpp
100.00% <100.00%> (ø)

include/micm/process/external_rate_constant.hpp
100.00% <100.00%> (ø)

...nclude/micm/process/intraphase_process_builder.hpp
100.00% <100.00%> (ø)

include/micm/system/phase.hpp
100.00% <100.00%> (ø)

include/micm/system/species.hpp
100.00% <100.00%> (ø)

include/micm/system/system.hpp
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[Kubevirt] Added testcase for backup and restore with different kubevirt VM states

What this PR does / why we need it:

[x] Added a new test KubevirtVMBackupRestoreWithDifferentStates which does multiple backup and restore operations while the Kubevirt VMs are in different states
[x] Added a new parameter to the VM operation helper functions called - waitForCompletion which will wait till the desired state is reached
[x] Added functions StartAllVMsInNamespace, StopAllVMsInNamespace and RestartAllVMsInNamespace

Which issue(s) this PR fixes (optional)
Closes #PA-1733
Special notes for your reviewer:
Jenkins Vanilla S3 Run
Jenkins Vanilla NFS Run

Latest successful run


STOR-1019: Bump to k8s 1.26 libs for OCP 4.13

This patches bumps library-go, which in turn updates to k8s 1.26 libs:
go get github.com/openshift/library-go && go mod tidy && go mod vendor
This also pulls in https://github.com/openshift/library-go/pull/1460, which bumps golang.org/x/net v0.5.0 and addresses a CVE.
CC @openshift/storage

# k8s.io/component-base/metrics
vendor/k8s.io/component-base/metrics/registry.go:35:29: undefined: atomic.Bool
note: module requires Go 1.19

I guess go version in the CI job needs to be updated. Or moved to .ci-operator.yaml.

# k8s.io/component-base/metrics
vendor/k8s.io/component-base/metrics/registry.go:35:29: undefined: atomic.Bool
note: module requires Go 1.19

I guess go version in the CI job needs to be updated. Or moved to .ci-operator.yaml.

PTAL:

Add .ci-operator.yaml file: https://github.com/openshift/csi-driver-shared-resource-operator/pull/67
Configure CI: https://github.com/openshift/release/pull/35872

/test verify

/test test

/assign @gabemontero

/approve
/lgtm

/label docs-approved
/label px-approved
/label qe-approved
This is not a user-facing change.
Also, discussed with the team offline and we agreed that CI should be enough for testing this change.


Added temperature measurement to DMM6500

Added support for the temperature (in Celcius) measurement to the Keithley DMM6500.

Codecov Report

Merging #128 (de7422c) into master (78839ee) will decrease coverage by 0.15%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #128      +/-   ##
==========================================
- Coverage   19.49%   19.33%   -0.16%
==========================================
  Files         121      121
  Lines       14448    14411      -37
==========================================
- Hits         2816     2787      -29
+ Misses      11632    11624       -8

Impacted Files
Coverage Δ

...contrib_drivers/drivers/Tektronix/Keithley_6500.py
0.00% <0.00%> (ø)

...b_drivers/tests/QDevil/test_real_qdac2_ieee_std.py
33.33% <0.00%> (-9.53%)
:arrow_down:

...ontrib_drivers/tests/QDevil/real_qdac2_fixtures.py
30.90% <0.00%> (-3.58%)
:arrow_down:

...contrib_drivers/tests/QDevil/sim_qdac2_fixtures.py
78.78% <0.00%> (-1.22%)
:arrow_down:

...trib_drivers/drivers/ZurichInstruments/ZIHDAWG8.py
55.33% <0.00%> (-0.88%)
:arrow_down:

...odes_contrib_drivers/tests/test_Keysight_M3201A.py
15.22% <0.00%> (-0.76%)
:arrow_down:

...ivers/drivers/Keysight/SD_common/memory_manager.py
79.64% <0.00%> (-0.36%)
:arrow_down:

...contrib_drivers/drivers/ERAInstruments/erasynth.py
0.00% <0.00%> (ø)

...contrib_drivers/tests/test_rohdeschwarz_HMC804x.py
100.00% <0.00%> (ø)

...ontrib_drivers/drivers/Keysight/Keysight_E8267D.py
0.00% <0.00%> (ø)

... and 7 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Add redirect for Zero-to-Dapp pages

As reported in https://github.com/blockstack/blockstack.org/issues/1123, we could use redirects for all the previous Zero-to-Dapp paths so that any remaining inbound links don't dead-end visitors.
Perhaps we should have them all go to https://docs.blockstack.org/browser/todo-list.html for now since that's our main tutorial at the moment?

This is done! 👍


Can not move window to other special workspace

Hyprland Version
 System/Version info
Hyprland, built from branch HEAD at commit 84ab8d11e8951a6551d1e1bf87796a8589da6d47  (props: bump ver to 0.35.0).
Date: Mon Feb 5 01:59:02 2024
Tag: v0.35.0

flags: (if any)

System Information:
System name: Linux
Node name: czh-81tc
Release: 5.5.13-1-mbp
Version: #1 SMP PREEMPT Wed, 15 Apr 2020 00:54:47 +0000

GPU information:
00:02.0 VGA compatible controller [0300]: Intel Corporation CometLake-U GT2 [UHD Graphics] [8086:9b41] (rev 02) (prog-if 00 [VGA controller])

os-release: NAME="Manjaro Linux"
PRETTY_NAME="Manjaro Linux"
ID=manjaro
ID_LIKE=arch
BUILD_ID=rolling
ANSI_COLOR="32;1;24;144;200"
HOME_URL="https://manjaro.org/"
DOCUMENTATION_URL="https://wiki.manjaro.org/"
SUPPORT_URL="https://forum.manjaro.org/"
BUG_REPORT_URL="https://docs.manjaro.org/reporting-bugs/"
PRIVACY_POLICY_URL="https://manjaro.org/privacy-policy/"
LOGO=manjarolinux

plugins:

Bug or Regression?
Bug
Description
The window only can be moved to default special workspace. My keybindings:
How to reproduce
# Special workspaces (scratchpad)
bind = $mainMod SHIFT, SPACE, movetoworkspacesilent, special
bind = $mainMod, SPACE, togglespecialworkspace

bind = $mainMod SHIFT, E, movetoworkspacesilent, wechat
bind = $mainMod, E, togglespecialworkspace, wechat
bind = $mainMod CONTROL, E, exec, wechat-uos

bind = $mainMod SHIFT, W, movetoworkspacesilent, qq
bind = $mainMod, W, togglespecialworkspace, qq
bind = $mainMod CONTROL, W, exec, linuxqq

Using Mod Shift Space will move window to default special workspce. But Mod Shift W cannot.
Crash reports, logs, images, videos
No response

qq is not a special workspace, it moved it to a named workspace called qq. https://wiki.hyprland.org/Configuring/Dispatchers/#workspaces

@vaxerski Thank you for your reply. According to the documents:

You can define multiple named special workspaces, but the amount of those is limited to 97 at a time.

Does it mean that I can define multiple special workspace but cannot move to them expect for the default special workspace?
Using bind = $mainMod, W, togglespecialworkspace, qq indeed toggle a new special workspace. How can I move window to it?

For those who may have same question. To move window to user-defined special workspace, you can use
bind = $mainMod SHIFT, Q, movetoworkspacesilent, special:qq
bind = $mainMod, Q, togglespecialworkspace, qq


fix for weird type error with animations

Hello, I am having some issues with the attached SVG (code pasted at the bottom), relating to its animation.
Upon investigation, it would seem that there's this function called "toNumberArray" that expects a string, however the animation functions perform a tween that result in a number. not a string, nor an array- What happens is canvg tries to call the "replace" method on a number, which causes an error. This patch fixes that problem, but reveals a new problem that the animation that canvg then shows is incorrect: specfically the center point for the rotation of the flower is wrong.
the svg spec defines rotation as 'around a given point", so the rotation animation here includes the coordinates 24,24 as additional parameters to the rotation transform, which appear to be ignored by canvg.
ref: https://developer.mozilla.org/en/docs/Web/SVG/Attribute/transform
<svg version="1.1"
     xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:a="http://ns.adobe.com/AdobeSVGViewerExtensions/3.0/"
     x="0px" y="0px" width="50px" height="50px" viewBox="-0.9 -0.9 50 50"
     style="overflow:visible;enable-background:new -0.9 -0.9 50 50;" xml:space="preserve">
<style type="text/css">
<![CDATA[
    .st0{fill:#EF8F5A;}
    .st2{fill:#FFDA82;}
    .st4{fill:#FFDA82;stroke:white;stroke-width:4px;}
    .st3{fill:#FFEECC;}
]]>
</style>
<defs>
    <symbol  id="petal" viewBox="-8 -8 14 16">
        <g transform="translate(0,0)">
        <path class="st4" d="M5.8,0c0-3.2-2.6-5.8-5.8-5.8S-5.8-3.2-5.8,0S-3.2,5.8,0,5.8S5.8,3.2,5.8,0z"/>
        <path class="st2" d="M5.8,0c0-3.2-2.6-5.8-5.8-5.8S-5.8-3.2-5.8,0S-3.2,5.8,0,5.8S5.8,3.2,5.8,0z"/>
        <path class="st3" d="M2.5,5.2C0.6,4.2-0.8,2.3-0.8,0s1.3-4.2,3.2-5.2C4.4-4.2,5.8-2.3,5.8,0S4.4,4.2,2.5,5.2z"/>
        <animateTransform
    attributeName="transform"
    begin="-200"
    dur="2.5s"
    type="translate"
    values="0 0; 7 0; 0 0"
    calcMode="spline"
    keySplines=".5 0 .5 1 .5 0 .5 1"
    repeatCount="indefinite"
/>
        </g>
    </symbol>
</defs>
<g transform="rotate(0)">

<use xlink:href="#petal"  width="14" height="14" x="-6" y="-5.8" transform="matrix(-1.50 0 0 1.50 40 24)" style="overflow:visible;"/>
<use xlink:href="#petal"  width="14" height="14" x="-6" y="-5.8" transform="matrix(1.5 0 0 -1.5 9 24)" style="overflow:visible;"/>
<use xlink:href="#petal"  width="14" height="14" x="-6" y="-5.8" transform="matrix(0 1.5 1.5044 6.575995e-08 24.0703 9.0264)" style="overflow:visible;"/>
<use xlink:href="#petal"  width="14" height="14" x="-6" y="-5.8" transform="matrix(0 -1.5 -1.5 0 24.0 39)" style="overflow:visible;"/>
<circle class="st0" cx="24.1" cy="24.1" r="8.7"/>

<animateTransform
    attributeName="transform"
    begin="-200"
    dur="5s"
    type="rotate"
    values="0 24 24; 1080 24 24; 0 24 24"
    calcMode="spline"
    keySplines=".5 0 .5 1 .5 0 .5 1"
    repeatCount="indefinite"
/>
</g>
</svg>

animations were implemented very naively and only support a single value changing.  i'll have to extend them properly to tween multiple semicolon-delimeted values to get this to work right.  also, i don't support spline/keyframe yet either, just basic linear tweening

closing out of staleness please reopen against latest master @Breton


Documents folder forgotten after store update

I just received the 13.2 update from the store (🎉) and both my auto-saved documents and the manually saved documents in the left pane disappeared.
So I clicked the "Change Documents Location" button and put the old folder there but nothing happened. Only after I restarted RP did everything come back to normal (including the auto-saved "New" tabs).

Hmm, did you by any chance delete the directory Documents\RoslynPad? In v13 I moved the settings from .NET app settings to a JSON file stored in that location.

No I didn't touch it... I just checked and it's still there, with RoslynPad.json intact.

Was the previous version 12 or 13?

Pretty sure it was 13.

Closing, since I can't find a bug that could cause this. Please reopen if that happens in the next upgrade.


#1635 - remove WCM Inbox Cleanup functionality

remove plugin functionality and replace it with a simple message
note - this is untested

Pull Request Test Coverage Report for Build 3558

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 42.337%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

bundle/src/main/java/com/adobe/acs/commons/wcm/impl/WCMInboxWebConsolePlugin.java
0
1
0.0%

Totals

Change from base Build 3556:
0.0%

Covered Lines:
9636

Relevant Lines:
22760

💛  - Coveralls


Design and implement Dart VM FFI

The present feature tracks the implementation of Dart FFI support enabling interop with C & C++ code from Dart.
Status
The feature will be in stable as of Dart 2.12. More details here:
https://dart.dev/guides/libraries/c-interop
For any discussion of this feature, or feedback or questions regarding the feature, kindly join and post the dart-ffi group: https://groups.google.com/forum/#!forum/dart-ffi
We will still actively adding new features to dart:ffi. Please refer to https://github.com/dart-lang/sdk/labels/library-ffi for open issues and feature requests.
Background
Some inspirational references:

http://luajit.org/ext_ffi.html
https://github.com/JetBrains/kotlin-native/blob/master/INTEROP.md
https://github.com/scala-native/scala-native/blob/master/docs/user/interop.rst

A general FFI could also be useful for interaction with JavaScript, so we should consider whether it's possible to make something general wihout compromising on the usability on each platform.
The current JS-interop functionality is not a clean design, it should be possible to improve it.

@lrhn:

The current JS-interop functionality is not a clean design, it should be possible to improve it.

I would not assume that without talking to @vsmenon. The syntax might leave a little to be desired, but I have no desire to make a breaking change to the syntax unless it vastly improves end users, not just the fact it isn't a "clean design".

Providing a new syntax / library doesn't necessarily imply removing the existing support.  With that in mind, I think @lrhn 's suggestion is good.

@lrhn @vsmenon I think C/C++-oriented interop makes sense on the Web in the context of WebAssembly. I wouldn't sacrifice native FFI capabilities due to JS. It's fine to have two systems that work best for their respective constraints.

Is this referring to Dart calling C/C++ interfaces or vice versa?

@krisgiesing Initially it will be Dart calling C, later we would like to extend this to C calling Dart.
Though it is very hard to draw a line because we plan to support callbacks as part of the initial offering - so C would be able to call back into Dart.
Here is the markdown version of the vision doc  (implementation is currently being prototyped).
Let me know if you have any comments on that one.

This potentially requires using conditional imports.

Conditional compilation is a pain for IDEs. It's worth looking into how Kotlin handles a similar problem with expect/actual declarations on a language level. In a nutshell, expect/actual means that you write a single header/interface file, and a corresponding implementation for each platform. Because the interface is shared between all platforms, IDE does not need to know the "current" platform to do IDE stuff.

@matklad the expect/actual stuff is pretty close to interface libraries that were discussed in context of the conditional imports long time ago - exactly to solve IDE problem. However I don't think this went anywhere - and I think conditional imports are pretty much dead.
I think in the context of FFI we will be implementing what Structure Layouts and Portability describes.
I will amend the section that still talks about conditional imports.

Any updates for dart lang FFi ? very happy if there have usable FFI .
We can import c/cpp language lib with it.

@netroby I'm working on it. We plan to add an initial version during Q1. So far the prototype closely follows the vision doc. Let me know whether that covers your use case.

The dart:ffi prototype has progressed to the point where we are deciding on API decisions. (Some design decisions are discussed here.)
To make informed design decisions, we would like more examples on what C APIs you would like to bind. The Flutter C++ interop issue mentions SQLite, Realm, OpenCV, Superpowered, and ffmpeg. Any other APIs we should consider?

@dcharkes two APIS i've been thinking about are libvlc and telegrams tdlib https://github.com/tdlib/td  and possibly SDL2

I believe databases support is often arises as a problem. ODBC, Oracle OCI and so on would be helpful

I personally made some half baked wrapper for http://www.libxl.com/ . I would be much prettier with the new FFI, I think

@dcharke I'm personally looking to use audio libraries such as [portAudio] (http://www.portaudio.com/) and [JUCE] (https://github.com/WeAreROLI/JUCE)

Having a convenient way to interact with Kotlin/Native would be extremly usefull (even if it would of course "somehow" work with a simple C FFI).

I'm looking forward to the possibility of accelerated cryptographic ops using libsodium or openssl

It isn't popular case (server-side) but still:  SDL2, OpenGL, Vulkan, OpenAL. I wish I can remove all native extensions code and achieve better performance with FFI.
Also kernel32.dll and user32.dll on Windows and libc.so on Linux.

I am interested in using GTK+ with Dart.

Is there a plan for distributing packages that use the new FFI, i.e. on Pub?
On Tue, Mar 5, 2019 at 8:20 AM Herman Bergwerf<EMAIL_ADDRESS>wrote:

I am interested in using GTK+ with Dart.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/34452#issuecomment-469676879,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJiKPIItmOmpY0HU8T4sdiAYiiWR1MgXks5vTm86gaJpZM4Wl8MH
.

Is there a plan for distributing packages that use the new FFI, i.e. on Pub?

@thosakwe I've discussed this @jonasfj, but at this point we don't know how yet we want to approach this. Should we distribute source or binaries? (Or provide the option to do either?) It probably depends on the use case. And if the package is used within Flutter, can we reuse something that Flutter does? We're aware of these questions, but do not have an answer yet.

@thosakwe i believe providing both options would be a huge plus.

I think that distributing binaries would be easier in that Pub itself
wouldn’t really need to change at all.
I personally prefer getting a source package, and then the package manager
compiling it, but the infrastructure for that doesn’t exist in Dart, and I
guess it’s unrealistic to expect things to change so drastically for just
one feature. There’s also no post-install hook functionality in Pub, so
ultimately I think distributing binaries would be the best way.
If there’s any disadvantage to that, it’s that you’d have to build your
project separately on each different platform. But also, if you are
providing support for Linux, etc., you’ve probably already built it on that
platform, so it’s not that much of a problem.
On Tue, Mar 5, 2019 at 9:04 AM Glenford Williams<EMAIL_ADDRESS>wrote:

@thosakwe https://github.com/thosakwe i believe providing both options
would be a huge plus.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/34452#issuecomment-469691161,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJiKPPWk_ptAL32032NeYHDzluAQeABFks5vTnlzgaJpZM4Wl8MH
.

Google LevelDB bindings would be an interesting exercise for the FFI.
On Tue, 5 Mar 2019 at 15:14, Tobe Osakwe<EMAIL_ADDRESS>wrote:

I think that distributing binaries would be easier in that Pub itself
wouldn’t really need to change at all.
I personally prefer getting a source package, and then the package manager
compiling it, but the infrastructure for that doesn’t exist in Dart, and I
guess it’s unrealistic to expect things to change so drastically for just
one feature. There’s also no post-install hook functionality in Pub, so
ultimately I think distributing binaries would be the best way.
If there’s any disadvantage to that, it’s that you’d have to build your
project separately on each different platform. But also, if you are
providing support for Linux, etc., you’ve probably already built it on that
platform, so it’s not that much of a problem.
On Tue, Mar 5, 2019 at 9:04 AM Glenford Williams<EMAIL_ADDRESS>
wrote:

@thosakwe https://github.com/thosakwe i believe providing both options
would be a huge plus.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/34452#issuecomment-469691161,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AJiKPPWk_ptAL32032NeYHDzluAQeABFks5vTnlzgaJpZM4Wl8MH
.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/34452#issuecomment-469694585,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMrs3lZBtub-BRHFWYFuAgbu2sOok25ks5vTnu4gaJpZM4Wl8MH
.

@dcharkes  For me this would be to use libgit2 with Flutter.

I'm interested in embedding my own business-logic as a library in Flutter.

crypto lib write by C or Rust

It would be nice to have a rust-bindgen alternative  for the Dart

Using Dart / Flutter with Qt might be interesting for desktop apps.

Using Dart / Flutter with Qt might be interesting for desktop apps.

Why?
Skia works in all the major platforms. Sublime Text uses Skia and is pretty fast and lightweight as well :)

Using Dart / Flutter with Qt might be interesting for desktop apps.

Why?
Skia works in all the major platforms. Sublime Text uses Skia and is pretty fast and lightweight as well :)

Skia is brilliant and it can be used in Qt apps as well (either by painting an SkSurface to a QWidget with QPainter, or by painting directly to a QWindow's context) with Qt providing things like cross-platform windowing and docking, c++ plugin support, installers/bundle builders, networking APIs, parsers and more.

@vejmartin This would increase the size of the executable by a big margin in case you use native features, I think

Another great thing would be using the FFI to access serial ports.

for serial support but I'd rather see it implemented as part of the standard libraries.
c.f.: https://github.com/dart-lang/sdk/issues/25360

@dcharkes, ti would be great to have support for openBLAS, and graphBLAS.  These provide powerful building blocks for working with linear algebra and graphs.

We are now at a point where we would like to offer an early preview to get some feedback. We’re looking for feedback on how ergonomic the feature is to use, what the main gaps are, and lastly on general stability and performance.
We are looking for testers who are willing to:

Test the feature by interfacing with a concrete C based library such as SQLite or libphonenumber
Tolerate a feature that is still incomplete and unstable
Offer us feedback on their experience

There is a small SQLite sample demonstrating the FFI feature in the SDK: Readme, FFI wrapper, sample program.
If you are interested, please consider one or more potential libraries you are interested in, and send an email with a description of those to<EMAIL_ADDRESS>(we’ll try to make sure we don’t several wrapping the same lib).
Also, you should be aware of the following current main limitations:

C++ not yet supported
Callbacks from C back into Dart not supported
Finalizers not supported
Field support is limited to 64-bit desktop
No iOS support

On behalf of the Dart FFI team, Michael

I'm so sorry, but we had an issue with the google group mentioned above, so if you already sent an email to<EMAIL_ADDRESS>can you please send it again?
Thanks much!

@mit-mit tried loading a basic shared library built from go on Linux but i am unable to get it to work
shared library build command
go build -o dart/bin/test.so -buildmode=c-shared lib.go
and tried opening using

dlopenPlatformSpecific comes from the examples provided
ArgumentError (Invalid argument(s): Failed to load dynamic library(lib/home/kingwill101/go/src/gitlab.com/kingwill101/test/dart/bin/libtest.so: cannot open shared object file: No such file or directory))
loading a system library works but trying to load using a relative path to the library fails.

@glenfordwilliams I suggest extending dlopenPlatformSpecific to create a full path based on where your .dart file is. You can construct the full path in Dart.
Explanation: the paths are directly interpreted by dlopen. This means the relative paths are relative to the dart binary. (This test uses a relative path, and the .so file lives right next to the dart binary.) In general, one would want a path to the .so file relative to the .dart file using that .so file, which would be constructed in Dart.

@dcharkes i tried doing that earlier but for some reason dart adds lib in front of the created path
Exception has occurred.
ArgumentError (Invalid argument(s): Failed to load dynamic library(lib/home/kingwill101/go/src/gitlab.com/kingwill101/test/dart/bin/test.so: cannot open shared object file: No such file or directory))

@glenfordwilliams Yes, you're gonna have to modify or make your own version of dlopenPlatformSpecific. The example implementation simply prepends "lib" on Linux on line 10.

It's Linux convention to prepend lib to the names of shared libraries. However, go build won't do that automatically. If you're not planning to run your code on other platforms, you can just use DynamicLibrary.open directly.

sorry about that, vscode seems to be caching old code

import 'dart:ffi' as ffi;
import 'dart:io';
import "package:path/path.dart" show dirname;
import 'dart:io' show Platform;

typedef GoFunction = ffi.Void Function();
typedef GoFunctionOp = void Function();

ffi.DynamicLibrary dlopenPlatformSpecific(String name, {String path}) {
  String fullPath = dirname(Platform.script.path) + "/" + name;
  return ffi.DynamicLibrary.open(fullPath);
}

main(List<String> arguments) {
  Directory current = Directory.current;
  print(current.path);
  ffi.DynamicLibrary l;
  try {
    l = dlopenPlatformSpecific("libtest.so");
  } catch (e) {
    print(e.toString());
  }

  var go = l.lookupFunction<GoFunction, GoFunctionOp>("AGoFunction");
  go();
}

strange thing is i get everything to work when i run from the command line
kingwill101@kingtech:~/go/src/gitlab.com/kingwill101/test/dart$ ~/flutter/bin/cache/dart-sdk/bin/dart bin/main.dart
/home/kingwill101/go/src/gitlab.com/kingwill101/test/dart
AGOFUNCTION()
kingwill101@kingtech:~/go/src/gitlab.com/kingwill101/test/dart$

but when i run from vscode i get
bin/main.dart:25:14: Error: Expected type 'NativeFunction<Void Function()>' to be a valid and instantiated subtype of 'NativeType'.
 - 'NativeFunction' is from 'dart:ffi'.
 - 'Void' is from 'dart:ffi'.
  var go = l.lookupFunction<GoFunction, GoFunctionOp>("AGoFunction");
             ^

What version of the Dart SDK is your VSCode install using?

totally forgot another version installed system wide, so vscode may be using that one
kingwill101@kingtech:~/go/src/gitlab.com/kingwill101/test/dart$ dart --version
Dart VM version: 2.2.0 (Unknown timestamp) on "linux_x64"

The FFI Preview is not available on that version of Dart; you'll need use top-of-tree or wait for Dart 2.3.

Hi,
First of all thanks to bring such functionality in Dart it’s really awesome and a long awaited feature.
We come after some testing of the ffi on the libsodium and here are what we gather.
First of all starting is not really hard with the Sqlite example and we can quickly wrap some key function of libsodium for benchmark purpose.
Here is our repos https://github.com/santetis/libsodium.dart.
We have wrap the initialization of libsodium, sha256 and sha512 to benchmark then against the crypto package (only support sha256) and pointycastle package (which support both). And here are the result :
Really quick to have something working with ffi (something like ~2/3h to have the 3 functions)
ffi is really slow (up to 10 time slower) (in fact it’s not really ffi but the conversion between type)
Here are the result of the benchmark we run (you can see the code in the benchmark folder).
For the crypto package :
master $ dart crypto/main.dart
CryptoSHA256: FooBar
CryptoSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
CryptoSHA256(RunTime): 16.493551047336304 us.

For the pointycastle package :
master $ dart pointycastle/main.dart
PointyCastleSHA256: FooBar
PointyCastleSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
PointyCastleSHA256(RunTime): 15.91678736848807 us.
PointyCastleSHA512: FooBar
PointyCastleSHA512: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
PointyCastleSHA512(RunTime): 70.59812919166961 us.

For the libsodium package :
master $ dart libsodium/main.dart
LibSodiumSHA256: FooBar
LibSodiumSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
LibSodiumSHA256(RunTime):<PHONE_NUMBER>2581453 us.
LibSodiumSHA512: FooBar
LibSodiumSHA512: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
LibSodiumSHA512(RunTime):<PHONE_NUMBER>4018267 us.
LibSodiumSHA512FFIOnly: FooBar
LibSodiumSHA512FFIOnly: Pointer<Uint8>: address=0x55b2ebc8fe80
LibSodiumSHA512FFIOnly(RunTime): 7.8313598346020115 us.

How to read this result :
LibSodiumSHA512 is the benchmark for using a function that can be used in every dart program no ffi type is in the function signature
LibSodiumSHA512FFIOnly is the benchmark with only the ffi part. Here you will need to pass a Pointer<T> to the function and the result you get is also a Pointer<T>
What can we extract from this result ?
ffi seems super quick but we need to transform the data to send it and receive it since the Pointer<T> object is not really useful in dart.
Here we are converting String to Pointer<Uint8> that it seems it’s the part that is taking all the time.
So some question now :

Is there a way to improve that part already ?
If not is there some plan to automatically marshall built-in Dart type in a way that is faster than what we can do now.

@mit-mit

@Kleak Thanks for trying FFI out, Kevin!
This was not explicitly highlighted in @mit-mit's invitation - but we are well aware that performance of FFI is lacking - especially when it comes to workloads which are reading and writing a lot of data through Pointer<T> on the Dart side. This is however will be fixes in the MVP and Release versions of FFI.
What we are looking for is feedback on API design - rather than performance.
To put this into perspective a line of code like this:
      ptr.elementAt(i).store(codeUnits[i]);

currently translates into two calls into runtime system and an allocation - which means CPU has to chew through several hundred CPU instructions (if not more). Once all planned optimizations are implemented this like of code should translate into just a few CPU instructions.

If not is there some plan to automatically marshall built-in Dart type in a way that is faster than what we can do now.

We do plan to provide some builtin ways to marshal common data types like strings and arrays.
I think one of the obvious conclusions we can make here is that CString-like class should be provided by FFI code (either dart:ffi or maybe package:ffi), rather than implemented by all FFI clients again and again.

Thanks for all the explanation.
For the API design in found it pretty intuitive. And except that yes provide basic type in FFI code i found it's fine. I haven't test all the capabilities so this is based on what i have seen.
Happy to help on the CString-like class.
I will continue to explore more things in the coming week.

@mraleph one more question
How to distribute the .so/.dylib.dll ?
Should we let the user deliver this part or should we distribute it ?
I think this part should be covered in the example because it's really not obvious.

@mraleph haven't had a chance to experiment with FFI yet, but are there plans to have an easy way to bridge external allocations to typed data? Like maybe a way to create External TypedData from a pointer+length pair?
I have a Rust library that takes large images and streams in fixed-sized chunks of raw image data. I need to turn these into typed data (hopefully without unnecessary copies) so I can pass it into dart:ui's decodeImageFromPixels function, which takes a Uint8List.
It would also be helpful to have zero-copy String and non-external TypedData handling, dunno if that's possible since Dart moves heap objects around during GC.

@Kleak We are aware of the distribution question, but we do not have an answer yet.

Is there a plan for distributing packages that use the new FFI, i.e. on Pub?

@thosakwe I've discussed this @jonasfj, but at this point we don't know how yet we want to approach this. Should we distribute source or binaries? (Or provide the option to do either?) It probably depends on the use case. And if the package is used within Flutter, can we reuse something that Flutter does? We're aware of these questions, but do not have an answer yet.

@ds84182 yes, supporting marshalling of arrays and strings as external types is one the list of items we would totally want to support in FFI. I am not sure if we have a bug filed for this (/cc @dcharkes @sjindel-google) but it was definitely part of the original vision.

It would also be helpful to have zero-copy String and non-external TypedData handling, dunno if that's possible since Dart moves heap objects around during GC.

This is possible if you can guarantee that GC can't occur, e.g. if native method is declared as a leaf (@sjindel-google just filed a bug to support those https://github.com/dart-lang/sdk/issues/36707).

The support for strings and arrays can be built as a library on top of the API available now, but these should probably come included with dart:ffi. I've filed https://github.com/dart-lang/sdk/issues/36711

https://github.com/dart-lang/sdk/issues/36247 is tracking FFI performance. Other benchmarks are welcome.

Is there an example to create a flutter application with ffi ?
Especially the part where to put the .so file.

@Kleak Please check out samples/ffi/sqlite/doc/android.md in the repo.

@mraleph @sjindel-google
One little problem i'm facing right now is that in libsodium some part of the lib are available only on some platform. So i ended up writing some generic code to add the capabilities to check is the function is available before running it.
Here is the generic code :
abstract class Bindings<DartSignature extends Function, NativeSignature> {
  DartSignature f;

  Bindings(DynamicLibrary sodium, String functionName) {
    try {
      f = sodium
          .lookup<NativeFunction<NativeSodiumInitSignature>>(functionName)
          .asFunction<DartSignature>();
    } catch (_) {}
  }

  bool get isAvailable => f != null;
}

and here an example of how to use it :
class SodiumInitBindings
    extends Bindings<SodiumInitSignature, NativeSodiumInitSignature> {
  @override
  SodiumInitSignature f;

  SodiumInitBindings(DynamicLibrary sodium) : super(sodium, 'sodium_init');

  int call() {
    if (!isAvailable) {
      throw BindingIsNotAvailableException(
          'sodium_init bindings is not available on ${Platform.operatingSystem}');
    }
    return f();
  }
}

the problem is when i run this code i got
lib/src/bindings/bindings.dart:12:12: Error: Expected type 'DartSignature' to be 'int Function()', which is the Dart type corresponding to 'NativeFunction<Int32 Function()>'.
 - 'NativeFunction' is from 'dart:ffi'.
 - 'Int32' is from 'dart:ffi'.
          .asFunction<DartSignature>();
           ^

So my question is : Is that normal behavior ? because I found it really annoying :/

Could you please add ability to create Pointer from TypedData or ByteBuffer or List?
It should neither allocate memory nor copy data operation.
It is "must have" feature to use Dart FFI with audio, video and grapchic libraries.

@rootext Note that in general creating and using interior pointers into Dart objects is not safe, because GC is managing these objects (deleting them and moving them around) - so if you need a pointer that is valid for a longer period of time then you can't just pass around pointer into a typed data object. If GC happens the pointer might become invalid.

@mraleph, I am going to use Dart FFI for OpenGL API.
There are few functions calls where I need to send pointer to raw byte buffer:
glShaderSource, glBufferData, glTexImage2D.
What is the recomended way for this?

@rootext you will have to allocate raw native memory, then fill it in on the Dart side then send it. There will be a way to make typed data object from raw native memory to make actual filling more ergonomic.

@mraleph, Great. Is it for 2.3 or future version?

@rootext future. FFI is not part of 2.3 anyway - it's in (super early stage) preview.

@Kleak sorry for the unhelpful error message - but the code you are writing is not expected to work. We currently require that type arguments arguments to lookup and asFunction are compile time constants. The reason for that is AOT: in AOT we need to know what kind of trampolines to generate for going between Dart and native code. These trampolines would be the most efficient when their Dart and native signatures is statically known.

But this definition is known at compile time

@Kleak it would be know in compile time if Dart generics worked like C++ templates, which are instantiated by expansion (essentially cloning them with type parameters replaced with actual types), but they don't. When you write
abstract class Bindings<DartSignature extends Function, NativeSignature> {
  final String functionName;

  Bindings(DynamicLibrary sodium, this.functionName) {
    try {
      f = sodium
          .lookup<NativeFunction<NativeSodiumInitSignature>>(functionName)
          .asFunction<DartSignature>();
    } catch (_) {}
  }
}

that more or less means (this is not a valid syntax - but that's how it works)
abstract class Bindings {
  final List<Type> :typeArguments; // secret field that contains type arguments

  Bindings(DynamicLibrary sodium, this.functionName) {
    try {
      f = sodium
          .lookup<NativeFunction<this.$typeArguments[0]>>(functionName)
          .asFunction<this.$typeArguments[1]>();
    } catch (_) {}
  }
}

is there a way to mimic that or do i have to write that every time? is this a limitation of the preview or this will stay like this (because it's something valid in dart)?

I don't think there is a way to mimic what you want - your best bet is just to generate boilerplate code using some helper script.
This limitation is somewhat intrinsic to how things are designed and how Dart generics work. So I am not too optimistic that it would change - however if we discover that this comes up often, then we need to go back to the drawing board.
[As I mentioned above - this limitation originates from the desire to make things efficient in AOT. Maybe we can make a special annotation to force some generics expand at compile time in AOT.]

@mraleph is there an example that shows how to map something like  const char * SomeFunction() to a dart function ?

@glenfordwilliams, you can use this class from sdk tests https://github.com/dart-lang/sdk/blob/master/tests/ffi/cstring.dart
typedef SomeFunction = CString Function();

@rootext tried that earlier but had issues with it. I'm trying to integrate a lib built using cgo which generates the following for strings
#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef struct { const char *p; ptrdiff_t n; } _GoString_;
#endif

tried using
@ffi.Struct()
class GoString {
  CString p;
}

but that doesn't work

@glenfordwilliams do I understand correctly that you have a function that returns _GoString_ rather than const char*, e.g. _GoString_ SomeFunction()?
Currently binding such functions is not supported. (See https://github.com/dart-lang/sdk/issues/36730)

@mraleph  yes that's pretty much it
#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef struct { const char *p; ptrdiff_t n; } _GoString_;
#endif

#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef _GoString_ GoString;
#endif

extern GString GetString();

Hi, thanks for shipping this preview, I wrote a simple prototype using Realm https://github.com/nhachicha/realm-dart-ffi
some feedback:

This has been mentioned a couple of time but there's a lack of wrappers around basics types, I think CString is a good example (maybe extend to others like enum which can map to integral types in C/C++).

Proxy objects will be created from Dart to abstract the low-level implementation in C++(Row/Table/Queries etc), since these allocations happened in the heap from the C++ side there's a need to use a finalizer object to detect instances going out of scope so we can free the native resources.

I won't mention the lack of callbacks from C/C++ since this is already in your roadmap.

Sometimes the C/C++ needs to return an array of primitives (as opposed to a pointer), currently accessing each element via an FFI call is suboptimal for large arrays... is there any similar
approach to JNI where we can access a Dart array backed by the same memory or allow some kind of move semantics internally which transfer the array element to Dart without the need for copying/marshalling.

Once we have callbacks from C/C++, it is unclear how the pthreads will interact with Dart?, will the callback invoke in an Isolates or in the Event Loop? for Realm there are still some internal resources which are thread confined, using some resources allocated in a callback in one thread then accessing them from another could be problematic.

Thanks for the feedback. The first three of suggestions we are very aware of and are on our roadmap. I'm curious about the last two though:

Sometimes the C/C++ needs to return an array of primitives (as opposed to a pointer), currently accessing each element via an FFI call is suboptimal for large arrays... is there any similar
approach to JNI where we can access a Dart array backed by the same memory or allow some kind of move semantics internally which transfer the array element to Dart without the need for copying/marshalling.

I'm not sure what you mean by "return an array of primitives (as opposed to a pointer)". What's problematic about returning a pointer to a heap-allocated array and access it via Pointer.load and Pointer.store?

Once we have callbacks from C/C++, it is unclear how the pthreads will interact with Dart?, will the callback invoke in an Isolates or in the Event Loop? for Realm there are still some internal resources which are thread confined, using some resources allocated in a callback in one thread then accessing them from another could be problematic.

In the Dart VM, each thread can be inside of at most one isolate at a time (newly created threads are not inside any isolate by default). Within each isolate, only one thread in that isolate can execute Dart code (called the "mutator" thread).
We have two plans for callbacks: synchronous callbacks and asynchronous callbacks.
A synchronous callback must be invoked on the mutator thread of the isolate it was taken from (for example, it can be invoked by C code which in turn was called by Dart via an FFI call). The VM will abort if it's called on the wrong thread.
An asynchronous callback will add a task to the microtask queue which will eventually invoke the callback on the mutator

Thanks for the feedback. The first three of suggestions we are very aware of and are on our roadmap. I'm curious about the last two though:

Sometimes the C/C++ needs to return an array of primitives (as opposed to a pointer), currently accessing each element via an FFI call is suboptimal for large arrays... is there any similar
approach to JNI where we can access a Dart array backed by the same memory or allow some kind of move semantics internally which transfer the array element to Dart without the need for copying/marshalling.

I'm not sure what you mean by "return an array of primitives (as opposed to a pointer)". What's problematic about returning a pointer to a heap-allocated array and access it via Pointer.load and Pointer.store?

Once we have callbacks from C/C++, it is unclear how the pthreads will interact with Dart?, will the callback invoke in an Isolates or in the Event Loop? for Realm there are still some internal resources which are thread confined, using some resources allocated in a callback in one thread then accessing them from another could be problematic.

In the Dart VM, each thread can be inside of at most one isolate at a time (newly created threads are not inside any isolate by default). Within each isolate, only one thread in that isolate can execute Dart code (called the "mutator" thread).
We have two plans for callbacks: synchronous callbacks and asynchronous callbacks.
A synchronous callback must be invoked on the mutator thread of the isolate it was taken from (for example, it can be invoked by C code which in turn was called by Dart via an FFI call). The VM will abort if it's called on the wrong thread.
An asynchronous callback will add a task to the microtask queue which will eventually invoke the callback on the mutator thread. An thread inside the appropriate isolate can invoke an asynchronous callback, but will block until the microtask is processed by the mutator thread.
How does this work for Realm?

@sjindel-google
Asynchronous callbacks are definitely a must-have feature.
Could you please show how it will work on an example?
Suppose we have a C++ function
using Input = std::string;
using Result = std::string;
void doHeavyTextProcessing(Input input, std::function<void(Result)> onReady);

which makes computations on an internal thread and invokes the callback on it.
What code should be written to make it available from Dart as
Future<String> doHeavyTextProcessing(String input);

?

Here's some thoughts on the current FFI api:
I somewhat dislike the current way of specifying native functions... It's great for unstructured testing, but I feel that it somewhat falls apart when you're trying to specify a large mass of functions.
Something like this would be nicer:
abstract class Kernel32 extends Library {
  bool QueryPerformanceCounter(Pointer<Uint64> out);
  bool QueryPerformanceFrequency(Pointer<Uint64> out);
}

abstract class FooBar extends Library {
  @Int32
  int Truncate64(@Int64 int x);
}

// Part of dart:ffi
class DynamicLibrary {
  // ...
  T asLibrary<T extends Library>();
}

It would make specifying and organization a bit easier.
Another thing is that bools seem to be unsupported?
Also analyzer lints would be awesome! A lot of issues I discover with my FFI code is something that could've been caught by the analyzer instead of the frontend.

@ds84182 These are some great suggestions!
We have to specify both the Dart signature and C signature, or the FFI does not know how to marshal between the Dart and C representation of values passed to functions. This encodes both those signatures:
abstract class FooBar extends Library {
  @Int32
  int Truncate64(@Int64 int x);
}

This we could maybe get to work, tracking in https://github.com/dart-lang/sdk/issues/36854.

Another thing is that bools seem to be unsupported?

Yes, we use Uint8 for those. Though I see that that requires one to convert between a bool and number on the Dart side. We could add a Bool C type that is treated as uint8_t and corresponds to the bool Dart type. Discussing in https://github.com/dart-lang/sdk/issues/36855.

Also analyzer lints would be awesome! A lot of issues I discover with my FFI code is something that could've been caught by the analyzer instead of the frontend.

Already on our todo list: https://github.com/dart-lang/sdk/issues/35777.

I have considered syntax like this
@Int32
int Truncate64(@Int64 int x);

But unfortunately it does not work with typedefs:
typedef FunctionPointer = @Int32 int TruncationFunction(@Int64 int x);
//                        ^^^^^^ can't put an annotation here

So you end up with two ways of specifying how native types map to Dart types. I do agree that the current approach is very verbose and needs to be either improved, shortened or autogenerated.

I'm agree that the current approach is too verbose and can become really messy / hard to read
I really like the annotation approach :) maybe allowing annotation on typedef might be something to consider ? and until it's done just not allowing typedef definition...

The problem with not using typedefs is that if you have a function referring to another function you have to repeat the signature of the function that is referred to.

+1 to be able to annotate typedefs because working with annotation is
always much easier to maintain.
--
Jonathan Rezende

@lrhn Would it be possible to have annotations on type defs?

@sjindel-google

I'm not sure what you mean by "return an array of primitives (as opposed to a pointer)". What's problematic about returning a pointer to a heap-allocated array and access it via Pointer.load and Pointer.store?

This works, however, it will be inefficient compared to a Dart array/list backed directly by the elements of this primitive array. Otherwise, you'll have to do N FFI call (ptr.load()) to access each element (that's my assumption).

An asynchronous callback will add a task to the microtask queue which will eventually invoke the callback on the mutator thread. An thread inside the appropriate isolate can invoke an asynchronous callback, but will block until the microtask is processed by the mutator thread.

How does this work for Realm?

Internally Realm has the possibility to hand over confined objects between threads, so as long as we resolve the result of the callback in the mutator thread this should work both for synchronous & asynchronous callbacks.
Here's the flow I imagine:
Dart (mutator thread)         C/C++ (mutator thread*)           C/C++ (worker_thread)         C/C++ (mutator thread*)                        Dart (mutator thread)
----------------------------- ---------------------------------  ----------------------------- ----------------------------------------------  --------------------------------------------------------
findAllAsync(query, callback) ------> FFI call to native method  -----> spawn a worker pthread ----> hand over the result to the mutator thread ---> trigger callback in Dart with a pointer to results

* this assumes that the FFI invocation in C/C++ runs within the mutator thread?

@sjindel-google

I'm not sure what you mean by "return an array of primitives (as opposed to a pointer)". What's problematic about returning a pointer to a heap-allocated array and access it via Pointer.load and Pointer.store?

This works, however, it will be inefficient compared to a Dart array/list backed directly by the elements of this primitive array. Otherwise, you'll have to do N FFI call (ptr.load()) to access each element (that's my assumption).

Currently you're correct that ptr.load() is slower than access through e.g. TypedData. However, we are planning to optimize it in the future and it should eventually have similar performance, and I expect we'll create wrapper classes to implement List<int> and Uint8List, etc. out of Pointers.
We are also planning to expose Dart-allocated memory to native code as well. This will always be limited to "leaf" functions: short-running native functions which don't call back into Dart. The problem with this is that GC can be triggered while Dart code is running, and GC can move the Dart-allocated memory. Exposing Dart memory to native code only works then when we pause GC across the native call.

An asynchronous callback will add a task to the microtask queue which will eventually invoke the callback on the mutator thread. An thread inside the appropriate isolate can invoke an asynchronous callback, but will block until the microtask is processed by the mutator thread.

How does this work for Realm?

Internally Realm has the possibility to hand over confined objects between threads, so as long as we resolve the result of the callback in the mutator thread this should work both for synchronous & asynchronous callbacks.
Here's the flow I imagine:
Dart (mutator thread)         C/C++ (mutator thread*)           C/C++ (worker_thread)         C/C++ (mutator thread*)                        Dart (mutator thread)
----------------------------- ---------------------------------  ----------------------------- ----------------------------------------------  --------------------------------------------------------
findAllAsync(query, callback) ------> FFI call to native method  -----> spawn a worker pthread ----> hand over the result to the mutator thread ---> trigger callback in Dart with a pointer to results

this assumes that the FFI invocation in C/C++ runs within the mutator thread?

Your assumption is correct -- this will work as you intend. In you diagram, only synchronous calls and callbacks are being used.

Wouldn't it be less messy to just have some code generation around this rather than trying to annoate typedefs? I haven't thought all the way through it so I may be missing something, but it seems like it should be possible.

@dnfield It would be great to have an FFI implementation that is concise and doesn't require code generation (which is a heavy-weight hammer) to interface with C/C++ code. For example, when I was tracking down flutter/flutter#32121 I used FFI outside of a package. It was much quicker than other languages (C++? gotta figure out a build system (CMake? GN? Visual Studio?). Rust? needs to create a project, download the winapi crate, build it...), and I'd like it to stay that way...
To be honest I tend to stay away from code generation in my own projects because of how ergonomics suffer. The analyzer still has issues on Windows when external code updates Dart files, for example, and the edit-refresh cycle for creating your own build steps is sub-par.

Annotations on typedefs? It's more likely than you think!
@eernst Erik, where are we with filling in the annotation holes in the grammar?

For the latest update on previewing this feature, please see the updated top-most comment: https://github.com/dart-lang/sdk/issues/34452#issue-359602269
For feedback, questions, and issues regarding the feature, kindly do not post them here in the issue tracker. Please post on the dart-ffi group: https://groups.google.com/forum/#!forum/dart-ffi

Wanna follow up whether we can expect iOS support by Dec 2019?

Wanna follow up whether we can expect iOS support by Dec 2019?

We hope to have that in the preview fairly soon. Please join the email group mentioned in https://github.com/dart-lang/sdk/issues/34452#issue-359602269 to get updates when that happens.

@Kleak
We've added a new feature to the Pointer api, Pointer.getExternalTypedData. This creates an external TypedData array backed by the C memory from which reads/writes are very efficient.
I modified your benchmark to use this feature, and got the following results:
Before any modifications
The results are similar to what you saw:
LibSodiumSHA256: FooBar
LibSodiumSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
LibSodiumSHA256(RunTime):<PHONE_NUMBER>9577466 us.
LibSodiumSHA512: FooBar
LibSodiumSHA512: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
LibSodiumSHA512(RunTime):<PHONE_NUMBER>025663 us.
LibSodiumSHA512FFIOnly: FooBar
LibSodiumSHA512FFIOnly: Pointer: address=0x55c12c900c80
LibSodiumSHA512FFIOnly(RunTime): 9.616287064683792 us.
After using getExternalTypedData in CArray.from and CArray.bytes
LibSodiumSHA256: FooBar
LibSodiumSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
LibSodiumSHA256(RunTime): 20.373880711047725 us.
LibSodiumSHA512: FooBar
LibSodiumSHA512: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
LibSodiumSHA512(RunTime): 22.924955869879188 us.
LibSodiumSHA512FFIOnly: FooBar
LibSodiumSHA512FFIOnly: Pointer: address=0x55d2a2cd7200
LibSodiumSHA512FFIOnly(RunTime): 9.691031461839255 us
After using getExternalTypedData and caching the allocated arrays
This optimization should not be needed after we optimize calls to Pointer.allocate and Pointer.getExternalTypedData itself.
LibSodiumSHA256: FooBar
LibSodiumSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
LibSodiumSHA256(RunTime): 8.367784881093836 us.
LibSodiumSHA512: FooBar
LibSodiumSHA512: 466f6f426172
LibSodiumSHA512(RunTime): 8.074552266524554 us.
LibSodiumSHA512FFIOnly: FooBar
LibSodiumSHA512FFIOnly: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
LibSodiumSHA512FFIOnly(RunTime): 5.994949267868651 us.
Result
The final performance (which can be achieved with the FFI today) is more than 2x faster than both PointerCastle and Crypto.

hey @sjindel-google ! Pretty nice optimization !! Do you mind make a PR of your modifications on the repo ? We would like to integrate those optimizations and as you already have done the migration that would be awesome ^^

What's the plan in releasing ffi on flutter? Should I stick to platform channel or should I investigate in exporting native extension myself. DESPERATE for calling cpp from dart.

What's the plan in releasing ffi on flutter? Should I stick to platform channel or should I investigate in exporting native extension myself or just wait for ffi. DESPERATE for calling cpp from dart, these is too much existing code in cpp to rewrite.

From what I understand, we can use FFI on Android, both debug and release, but the functionality is limited, and API subject to change. I have not found any example on iOS, nor the status of iOS in general.
Ref https://github.com/dart-lang/sdk/issues/37224#issuecomment-517663205

@truongsinh is correct, although iOS is supported equally well as Android.
See https://github.com/flutter/flutter/wiki/Binding-to-native-code-via-FFIl for details on iOS.

@truongsinh is correct, although iOS is supported equally well as Android.
See https://github.com/flutter/flutter/wiki/Binding-to-native-code-via-FFIl for details on iOS.

A small typo in your link, 1 I too much at the and of the link (FFI vs FFII)

Thanks, corrected.

@sjindel-google I was able to create an extremely simple example on iOS

(Swift code) https://github.com/truongsinh/flutter-ffi-samples/blob/master/packages/simple/ios/Classes/SwiftSimplePlugin.swift#L4-L7
(Dart code) https://github.com/truongsinh/flutter-ffi-samples/blob/master/packages/simple/lib/simple.dart#L9-L12

You can see in Swift code I use @_cdecl to declare C interface. Should this be mentioned in the wiki page?

Sure, feel free to edit the Wiki page!

Should https://github.com/dart-lang/sdk/issues/3691 be re-consider in the interest of FFI?

Hey TruongSihn,
See https://github.com/dart-lang/sdk/issues/35770
Kind regards,
On Wed, Sep 18, 2019 at 12:44 PM TruongSinh Tran-Nguyen <
<EMAIL_ADDRESS>wrote:

Should #3691 https://github.com/dart-lang/sdk/issues/3691 be
re-consider in the interest of FFI?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/34452?email_source=notifications&email_token=AA3PK33QKMHQ6MO4W6DEMEDQKIBBLA5CNFSM4FUXYMD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67UNYQ#issuecomment-532629218,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA3PK343HM77FJJHKID57U3QKIBBLANCNFSM4FUXYMDQ
.

--
Daco Harkes

Just went through a simple ffi demo and it worked well.
I have a question though, can i call Java methods from c code which is loaded via ffi on an Android device?

@WoodyGuo You would have to use JNI for that.

@WoodyGuo You would have to use JNI for that.

Thanks Daco for the prompt reply!
So I assume that I must split my single native library into 2 ones.
One is to be loaded from Java via System.loadLibrary, and the other is to be loaded from Dart via FFI.
So what would you suggest to use to call code in one library from the other one?

@WoodyGuo It looks like you can make calls to the same instance of the same native library from both Java and Dart. This cpp file worked for me (after calling the first function from Java, calling the second one returns 123 to Dart):
extern "C" {
    int32_t setByJava = 0;

    JNIEXPORT void JNICALL
    Java_com_michaeljperri_flutter_1plugin_FlutterPlugin_native_1setValue(JNIEnv *env, jobject instance) {
        setByJava = 123;
    }

    __attribute__((visibility("default"))) __attribute__((used))
    int32_t get_val_from_java(const char* filename) {
        return setByJava;
    }
}

The cpp file is linked as a shared library. I have a Java class with
System.loadLibrary("my_plugin") and a Dart file with DynamicLibrary.open("libmy_plugin.so").

@mikeperri cool, thanks for the sharing.

Is C calling Dart supported now?

Yes, it's supported, but still not entirely complete. See https://dart.dev/guides/libraries/c-interop

@mit-mit Thank you.But it seems that the examples only show how to call C functions in dart.And I want to know if invoking a dart method from the C/C++ code is supported now , just like the end of this page?

@scriptsman callbacks are supported as well:

synchronous callbacks: see samples
asynchronous callbacks: see samples

And here and here is the native code for both samples.

@dcharkes  Looking forward to supporting instance(not class) method.

Is it possible to add debugging for ide?
art(11490,0x7000070ec000) malloc: *** error for object 0x10fcb0efc: pointer being freed was not allocated
dart(11490,0x7000070ec000) malloc: *** set a breakpoint in malloc_error_break to debug

@listepo In order to get C/C++ debugging, you need to run the C/C++ debugger on the Dart executable. For more details refer to https://github.com/dart-lang/ffi/issues/47.

@dcharkes thank you so much

I am facing a problem when working with strings. For example:
https://github.com/postgres/postgres/blob/8a15e735be00f156a7227741c0ce88702e6de099/src/interfaces/libpq/fe-connect.c#L6535
char *
PQpass(const PGconn *conn)
{
	char	   *password = NULL;

	if (!conn)
		return NULL;
	if (conn->connhost != NULL)
		password = conn->connhost[conn->whichhost].password;
	if (password == NULL)
		password = conn->pgpass;
	/* Historically we've returned "" not NULL for no password specified */
	if (password == NULL)
		password = "";
	return password;
}

when I use https://github.com/dart-lang/ffigen/issues/72#issuecomment-673328056
I get an error
art(11490,0x7000070ec000) malloc: *** error for object 0x10fcb0efc: pointer being freed was not allocated
dart(11490,0x7000070ec000) malloc: *** set a breakpoint in malloc_error_break to debug

I guess the error occurs when the function returns an empty string(password = "";) and I try to free memory.(for strings without malloc)
How can I understand when I need to free up memory, and when not?(password = conn->pgpass; with malloc)

Please open a new issue instead of slow-chatting on this one. Feel free to tag me.

Please open a new issue instead of slow-chatting on this one. Feel free to tag me.

@listepo or even better use mailing list to ask questions. Issue tracker exists primarily for bug reports and feature requests, not questions.

@listepo  two ways:  1) return NULL instead of "";  2) check the length of password; if the length is = 0, just ignore it.

@dcharkes @mraleph sorry for the spam. I think the new issue is better, the answer can help not only me

We will still actively adding new features to dart:ffi. Please refer to https://github.com/dart-lang/sdk/labels/library-ffi for open issues and feature requests.

We will still actively adding new features to dart:ffi. Please refer to https://github.com/dart-lang/sdk/labels/library-ffi for open issues and feature requests.

does this work w/ emscripten on the wasm side?


Upgrade javascript dependencies and remove from being vendored in static/third/

Most of the javascript modules we have vendored under static/third could be fetched via npm with pinned versions just as well (we haven't patched most of them).  We should systematically replace as many of these as we can with entries in package.json (and upgrade them to current versions).
When doing so, we should be careful to use git log --follow on the relevant file to make sure Zulip doesn't have any patches not present in upstream (any for any where we do, we should submit them upstream so we can stop maintaining forks of javascript modules!).

We can use a similar approach to 8f73701b96c352960fd852a1492b691b36476221 for devendoring modules when we're ready to upgrade them.

@timabbott I'll work on this issue

Here's some notes on the remaining things that we can usefully make progress on:

bootstrap: Needs some significant work to upgrade.  Main pain is extracting what changes we've made that aren't to the typeahead; if we can move them to not be in bootstrap code itself, then we can upgrade further via NPM.
bootstrap-notify: Has a few small patches we've made.  Is a tiny library in general.  It's unclear to me whether we still need it -- the last time someone thought about this was like 2013.
jquery-idle: Was basically rewritten in ae08f512dd954b059450d7e6abacf66e3c08f767; given how small it is, it's barely fair to call it "jquery-idle".  I think probably we should just clean it up further and move it into static/js/idle.js, properly recording its history.

And these that will probably be in static/third indefinitely:

sockjs: Dead upstream AFAIK.
marked: A significant fork; we'd need to get extension support into upstream to not fork this.
bootstrap-typeahead: Heavily patched and also removed in newer bootstrap upstream (became twitter-typeahead).  Not a priority.
jquery-filedrop: Significantly patched.  Should be replaced by a third-party library that supports chunked uploads to fix https://github.com/zulip/zulip/issues/9391, so moving to NPM is probably not a good use of time.   I'd love to see someone work on this.

There's now just a handful of packages in static/third, most of which are there for a reason.  I think we can close this as resolved once we kill bootstrap-notify (Which I think @andersk might do at some point?).

marked: A significant fork; we'd need to get extension support into upstream to not fork this.

Is that really true? I haven’t gone through all our changes in detail, but a cursory glance suggests that we should be able to achieve them using normal JavaScript class inheritance.

OK, I think this work can all be tracked in other places:

Removing bootstrap tooltips, which we're only using for popovers now, is https://github.com/zulip/zulip/pull/17244
Upgrading bootstrap itself, which we've done a few rounds of since this issue.

And then there's things we've forked majorly:

Upgrading or replacing marked, which should be part of our broader Markdown strategy.
Migrating to a more modern typeahead library instead of our bootstrap-typeahead fork.
Our jquery-idle is a rewrite.

So I think it makes sense to close this issue as resolved.

Upgrading bootstrap itself, which we've done a few rounds of since this issue.

I note that Bootstrap 3 was released 2013-08-19, three years before this issue was opened. Although we managed a tiny upgrade from a forked 2.1.0/2.1.1 to a forked 2.3.2 (2013-05-17) seven years late (#16196), we’re still not much closer to upgrading to 3, let alone 4 or 5. This is not by any means a resolved problem.
My view is still that Bootstrap doesn’t do anything for us but get in our way. We have a bunch of CSS whose only purpose is to override Bootstrap’s CSS, and a bunch of CSS that haphazardly extends it, but we’re no longer using it as intended basically anywhere. It seems quite possible that removing Bootstrap entirely would take a similar amount of effort as upgrading it, and would be more worthwhile. But these are both large projects.

Migrating to a more modern typeahead library instead of our bootstrap-typeahead fork.

I’m not sure what you meant to imply with the position and wording of this list item, but to be clear, this has not been done—we’re still using our Bootstrap typeahead fork. Since typeahead was removed from Bootstrap 3, this may be an impediment to upgrading Bootstrap, depending on how entangled this code is with the rest of Bootstrap.
You can close issues if it makes you happy I guess, but I’m not going to be happy until our static/third directory is actually deleted. Until then, we have a bunch of critical code that isn’t being maintained either by upstream or by us, and is being excluded from our static analysis tools. An after-the-fact justification for why the particular code we have in static/third is special doesn’t change that fundamental problem.


Customize default scope

I need to customize the default_scope.

This is fixed for rails3.2. I'll be releasing a new version in the next couple of days.


FuzzyCorrectionFilter should replace FuzzyCorrFilter, maybe?

I'm working from Ubuntu 24.04 LTS, which uses gcc 13. I've also built InsightToolkit 5.4.0 locally with the same setup. I cloned the master branch of PETPVC and attempted to build it.
The compiler complained about "FuzzyCorrFilter" not existing when referenced in petpvcFuzzyCorrectionFilter. It suggested I may have intended "FuzzyCorrectionFilter" instead.
In file included from /usr/local/include/ITK-6.0/itkLightObject.h:21,
                 from /usr/local/include/ITK-6.0/itkObject.h:31,
                 from /usr/local/include/ITK-6.0/itkRegion.h:31,
                 from /usr/local/include/ITK-6.0/itkImageRegion.h:31,
                 from /usr/local/include/ITK-6.0/itkImage.h:21,
                 from /home/mike/Projects/PETPVC/src/IterativeYang.cxx:34:
/home/mike/Projects/PETPVC/lib/petpvcFuzzyCorrectionFilter.h: In member function ‘const char* petpvc::FuzzyCorrectionFilter<TImage>::GetNameOfClass() const’:
/home/mike/Projects/PETPVC/lib/petpvcFuzzyCorrectionFilter.h:56:18: error: ‘FuzzyCorrFilter’ was not declared in this scope; did you mean ‘FuzzyCorrectionFilter’?
   56 |     itkTypeMacro(FuzzyCorrFilter, ImageToImageFilter);
      |                  ^~~~~~~~~~~~~~~
make[2]: *** [src/CMakeFiles/pvc_iy.dir/build.make:76: src/CMakeFiles/pvc_iy.dir/IterativeYang.cxx.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:300: src/CMakeFiles/pvc_iy.dir/all] Error 2
make: *** [Makefile:166: all] Error 2

I'm a python coder who hasn't written C/C++ in many years, and I never got into type libraries, so I have no real idea what I'm doing here. But making that change allowed for my build to complete successfully.

You seem to be the first to try to compile with ITK 6.0. I have done some digging, and it appears that in previous versions of ITK, the first argument of itkTypeMacro could be anything at all (it was just used as a string), but now it needs to be the class name.
I'll use your suggestion as a fix. Could you please confirm that ctest works fine, and indeed PETPVC works ok with ITK 6.0?

Thanks, Kris!
Our only use of PETPVC is the following command.
petpvc \
-i /path/to/orig_01.nii.gz \
-o /path/to/pvc_01.nii.gz \
-m /path/to/mask_orig.nii.gz \
-p STC -x 5.9 -y 5.9 -z 5.9

I used it on 25 volumes with our prior environment (a ubuntu:jammy docker container with petpvc 1.2.11, compiled with ITK 5.2.1 and gcc12). I then used it on the same 25 volumes with my new local environment ( ubuntu:noble with petpvc 1.2.11, compiled with ITK 5.4.0 and gcc13). I compared the volumes from each build and the values at every voxel were identical in every image.

From ctest:
Test project /opt/build_petpvc
      Start  1: generate
 1/23 Test  #1: generate ...................................   Passed    0.09 sec
      Start  2: simulate
 2/23 Test  #2: simulate ...................................   Passed    0.07 sec
      Start  3: RunIterativeYang
 3/23 Test  #3: RunIterativeYang ...........................   Passed    0.12 sec
      Start  4: CompareIterativeYang
 4/23 Test  #4: CompareIterativeYang .......................   Passed    0.07 sec
      Start  5: RunIterativeYangUndercorrect
 5/23 Test  #5: RunIterativeYangUndercorrect ...............   Passed    0.05 sec
      Start  6: CompareIterativeYangUndercorrect
 6/23 Test  #6: CompareIterativeYangUndercorrect ...........   Passed    0.01 sec
      Start  7: RunIterativeYangOvercorrect
 7/23 Test  #7: RunIterativeYangOvercorrect ................   Passed    0.06 sec
      Start  8: CompareIterativeYangOvercorrect
 8/23 Test  #8: CompareIterativeYangOvercorrect ............   Passed    0.01 sec
      Start  9: RunRBV
 9/23 Test  #9: RunRBV .....................................   Passed    0.09 sec
      Start 10: CompareRBV
10/23 Test #10: CompareRBV .................................   Passed    0.01 sec
      Start 11: RunRBVUndercorrect
11/23 Test #11: RunRBVUndercorrect .........................   Passed    0.03 sec
      Start 12: CompareRBVUndercorrect
12/23 Test #12: CompareRBVUndercorrect .....................   Passed    0.01 sec
      Start 13: RunRBVOvercorrect
13/23 Test #13: RunRBVOvercorrect ..........................   Passed    0.03 sec
      Start 14: CompareRBVOvercorrect
14/23 Test #14: CompareRBVOvercorrect ......................   Passed    0.01 sec
      Start 15: RunDiscreteIterativeYang
15/23 Test #15: RunDiscreteIterativeYang ...................   Passed    0.10 sec
      Start 16: CompareDiscreteIterativeYang
16/23 Test #16: CompareDiscreteIterativeYang ...............   Passed    0.01 sec
      Start 17: RunDiscreteIterativeYangUndercorrect
17/23 Test #17: RunDiscreteIterativeYangUndercorrect .......   Passed    0.03 sec
      Start 18: CompareDiscreteIterativeYangUndercorrect
18/23 Test #18: CompareDiscreteIterativeYangUndercorrect ...   Passed    0.01 sec
      Start 19: RunDiscreteIterativeYangOvercorrect
19/23 Test #19: RunDiscreteIterativeYangOvercorrect ........   Passed    0.04 sec
      Start 20: CompareDiscreteIterativeYangOvercorrect
20/23 Test #20: CompareDiscreteIterativeYangOvercorrect ....   Passed    0.01 sec
      Start 21: Compare_iy_diy
21/23 Test #21: Compare_iy_diy .............................   Passed    0.01 sec
      Start 22: Compare_iy_diy_Undercorrect
22/23 Test #22: Compare_iy_diy_Undercorrect ................   Passed    0.01 sec
      Start 23: Compare_iy_diy_Overcorrect
23/23 Test #23: Compare_iy_diy_Overcorrect .................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 23

Total Test time (real) =   0.94 sec

Excellent. Thanks @mfschmidt!
I'm marginally confused by you claiming to use ITK 5.4.0, while the error messages report ITK-6.0, but fine.
master has (a somewhat more elaborate version of) the fix. I will release PETPVC 1.12.2 soon, after sorting out some more CI issues.

That's interesting. I intended to use 5.4.0 rather than 6.0, and I explicitly specified 5.4.0 to correct your 6.0 comment, but I now see your point and see the directory in my original error. Sorry for the confusion. I cloned their git repo and checked out the v5.4.0 branch, but I may have made a mistake or specified the wrong directory when I built it. I appreciate the correction/comment and I'll do some digging on my end. It's probably good either way here, so thanks for the rapid fix! :)


Update documentation

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (ca9f10b) 100.00% compared to head (1580f06) 100.00%.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #449   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           58        58
  Lines         1577      1577
  Branches       192       192
=========================================
  Hits          1577      1577

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


tour: windows: cannot find package "golang.org/x/tour/reader"

I am using Go 1.6 in Windows - downloaded today.  I was working on the 'Exercise:Readers' part of the tour ( http://<IP_ADDRESS>:3999/methods/22 ), and got the following error when trying to Run:
compile291.go:3:8: cannot find package "golang.org/x/tour/reader" in any of:
C:\Go\src\golang.org\x\tour\reader (from $GOROOT)
C:\Go\misc\tour\src\golang.org\x\tour\reader (from $GOPATH)
This looks like the same thing as described in https://github.com/golang/go/issues/14184 , but I am reporting for Windows.  Steps to fix as described by lernisto worked for me as well.

CC @adg @campoy

This will be fixed when Go 1.6.1 is released (soon).


🏗 maintenance: tooling

What do these changes do?

Related issue/s

How to test

Checklist

[ ] test/test_workflows.py runs against fake server
[x] Is api/openapi.json up-to-date
[x] Auto-doc generated
[x] Links to notebooks modified

Rather use #30


Experiment: Try out TypeScript Program in compiler

Issue :
Hi guys! Is there a way to get access to the TS Program/TypeChecker instance from inside of a custom AST transformer?
There was a similar question #303 At that time it wasn't possible but now ts-jest uses TS language service I guess it is possible? Would passing the languageService to ConfigSet introduce some issues?

Hey!
I looked a bit into ts-jest recently to look for the TS Program (and the TypeChecker), but it seems to me that ts-jest does avoid to use it. I found this comment:

[...] If we don't return undefined it results in undefined === "undefined" and run createProgram again (which is very slow).

ts-jest exclusively uses transpileModule to transpile all tests independently, see https://github.com/kulshekhar/ts-jest/blob/master/src/compiler.ts#L97-L102.
Here ts is the actual resolved typescript module, but createProgram is never called.
Maybe we could learn e. g. from Angular/webpack, how it compiles incrementally using the TS language service with TS Program and TypeChecker (this is the place where Angular's webpack plugin creates the program etc). Obvoiusly tansformers without TypeChecker functionality are way less powerful and restricted. The language service itself gets created in createCompiler (compiler.ts) when isolatedModules is not set to true.

I'm interested in your findings as well, cheers!

 So if such PR would be accepted I can prepare modifications providing the language service for transformers.

I am not a ts-jest-member, just stumbled upon the same issue around the same time.
But I personally would be interested in performance results of an implementation with a single Program instance for all compilations done in ts-jest. Maybe calling transpileModule is actually slower when called for each file individually, e. g. if a big dependency tree has to be loaded every single time than doing it just once using the TS Program, and the comment in the code is a wrong assumption for big projects.
I am not in the position to suggest a PR, but I think it is valuable for ts-jest to learn about the performance implications of this implementation.

According to @huafu , he said once about transformer should be only initialized once for all tests. Does it mean about Program instance ?

No, it is not really related to the transformer lifecycle.
Basically, there are two ways to compile TS code to JS calling the tsc compiler programmatically:

transpileModule, which is a simpler, more naive approach of transpiling a simple module (usually a single file), it cannot verify import statements in depth
createProgram and program.emit creates a compiler host that gathers lots of context and diagnostics during the compilation, if program is used and supplied to transformers, they can access the TypeChecker, which knows about references of other modules, e. g. which class is referenced by a class reference. This additional context information can be super useful.

huafu once stated (see above) that createProgram is slow, and I totally agree it is slow when using it in unit tests for tiny applications. But in a big actual application, where the compiler host and program is created only once for all tests, it can be very helpful.
See https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API for further information.

Not dive into createProgram yet but creating instance once for all tests already sounds logical. I’d go for it.
How do we approach that solution ?

This might be too costly, as huafu stated before, but we should probably give it a shot I think.
I also have not worked much with the createProgram of tsc.
Probably it is best to create the program instance on the top level of createCompiler inside compiler.ts, or maybe even a language server instance, although I am also not too sure about the implications of it.
There is not much literature about the TS compiler API. It is probably best to have a look at how other compilers (e. g. Angular-cli, Vue-cli, webpack-ts-loader, webpack-awesome-typescript-loader) do this and learn from them. I looked a bit into Angular Compilation, but it is quite complicated, it might be better to have a look at the others (and check if they use createProgram).

Thanks for the info 🙏, I’m curious how the examples do with that pattern

@mtebenev , @wtho Program is available now for transformers. You can test against master

Yeah! Thanks so much!


Fix regular expression for optional params

The router converts patterns with optional parameters to regular expressions that are not strict enough.
Example: /profile/:id?/:tab? matches /profile-random-bug, /profile// and /profile/// paths.
This PR fixes it.
11 bytes added.

Thanks a lot!

Released in 0.14.2


Function and class reference with sphinx

Make use of the automodule option.

Since we're Python 3.6+ anyway, is it an idea to use type hinting?


added abat AG as consulting and SAP partner who uses abapGit

Thank you for this great piece of software and the very well cooperation @larshp and all others of this wonderful community.

thanks 👍


Issue Controlling Non-Addressable LEDs

I've just been trying to control some 5050 LEDs using the 'NonAddressable' example for an Arduino Leonardo with the device set to a Lighting Node Pro.
There seems to be an issue with the color of the LEDs - when I set the color to R:255 G:0 B:0 in iCUE, the LEDs show green and blue at full brightness with a dim red color, and when I set it to R:0 G:0 B:255, the LEDs show red and green at full brightness but blue is very dim.
I haven't modified the example, but if you had any ideas as to why this is happening your help is much appreciated.
Thank you.

The wiring of your LEDs is inverted. What LEDs do you use, depending on that you have to options: change the wiring or invert the arduino output signal.

I’m wanting to set this up to control the LEDs on a waterblock, and I’m using a strip of 5050 LEDs I bought from China to test it (they are the same LEDs and the same issue occurs on both the waterblock and test strip).
The +12V on the strip is wired into a molex plug, with the R G B wires from the strip connected to the 3, 5, 6 pins on the Arduino.
On 29 Dec 2020, 11:31 PM +1300, Leon Kiefer<EMAIL_ADDRESS>wrote:

The wiring of your LEDs is inverted. What LEDs do you use, depending on that you have to options: change the wiring or invert the arduino output signal.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

You should not connect the R G B wires directly to the arduino, because the arduino works with 5V (thats why your leds are not off but only dim). You need to use transistors which can handle the 12V and can be controlled by an arduino

I’m assuming wiring it like it shows in this article would work? https://www.makeuseof.com/tag/connect-led-light-strips-arduino/
I’m very new at this and have practically no idea what I’m doing so thank you for helping me.

Yes the article looks good, but I think it's much easier to use WS2811/12/12B LED strips, which are also explained in the article. They work with 5V and can directly be controlled by the arduino. When the arduino is connected with USB you should not connect the 5V of a power supply to the arduino pins, because the arduino is powered by USB, only the strip should get the power from the Power supply.


How to Change/Update Events

I understand setting the default events to fire
angular.extend($scope, {
    events: {
        map: {
            enable: ['drag', 'click', 'etc'],
            logic: 'emit'
        }
    }
});

When a specific event is triggered, I'd like to update the events that can fire, or even clear them all. For example, this is not working for me.
<!-- My Directive -->
<leaflet id="myMap" paths='paths' tiles="tiles" markers="markers" center="center" events="events" event-broadcast="events"></leaflet>

// My Controller
// Expected Behavior: when the map is clicked, remove all events from firing
$scope.$on('leafletDirectiveMarker.click', function (e, args) {
    leafletData.getMap('myMap').then(function(map){
        $scope.events = {
		    map : {
		       enable : [],
		       logic : 'emit'
		   }
		};
    });
});

I click the map, see my click event run (which should remove future click events), but then when I click the map again the click event continues to fire.
How can I accomplish behavior I'm looking for?
Thanks

For the time being I'm just going to flag my own state. For example, I want to show the user the lat/lon as they move their mouse over the map. When they click the map a popup will open, and I no longer want the user to see the lat/lon update when the mouse is moved over the map:
$scope.isPopupOpen = false;

// set isPopupOpen to true when map is clicked
$scope.$on('leafletDirectiveMap.click', function(event, args){
// open the popup here (code not shown), and set isPopupOpen to true
$scope.isPopupOpen = true;
});
$scope.$on('leafletDirectiveMap.mousemove', function(event, args){
//$log.log('map:mousemove', args.leafletEvent.latlng.lat, args.leafletEvent.latlng.lng);
// if popup is not open go ahead and show the user the lat/lon the mouse is hovering/moving over
if(!$scope.isPopupOpen){
leafletData.getMap('mainMap').then(function(map){
$scope.maptoolsLat = args.leafletEvent.latlng.lat.toFixed(3);
$scope.maptoolsLon = args.leafletEvent.latlng.lng.toFixed(3);
});
}
});

I handle event deletion like this:
//create an array which will hold your events
var eventArr =[];
//declare click as an event
var Click = $scope.$on('leafletDirectiveMap.click', function(ev, featureSelected, leafletEvent) {
//etc..
});
eventArr.push(click)
//To delete your events:
angular.forEach(eventArr, function(data, index) {
data.call();
});
How does this work?  In Angular if you call a listener function, it destroys that listener. See here for more: http://stackoverflow.com/questions/14898296/how-to-unsubscribe-to-a-broadcast-event-in-angularjs-how-to-remove-function-reg

This issue was moved to angular-ui/ui-leaflet#88

I'm going to rework&redesign angular-leaflet-directive to be compatible with Leaflet v1.0. It will mantain almost all its functionality, and will be compatible with the current features of the directive, but I must start from a fresh point, so I'm going to close this issue. If you think it must be worked with the new version, please reopen it.


multipart request: always skip boundary, fix #1220

#1220

I'd like to eventually add a validation that the readUntil actually reads 0 bytes until it hits the boundary. But that requires either adding a byte counter to readUntil, or using something else to consume the marker. I'll pull this in the meantime to see if it fixes the test mentioned above.

I think it's a good idea.
Something like stream.consumeExpected(cast(ubyte[])boundary);?

I think I'd go with something like enforceHTTP(stream.trySkip(cast(ubyte)[])boundary), HTTPStatus.badRequest);, so that a proper HTTP status is generated. I'll add that and create an alpha tag for 0.7.25.

I think I'd go with something like enforceHTTP(stream.trySkip(cast(ubyte)[])boundary), HTTPStatus.badRequest);, so that a proper HTTP status is generated.

I thought about validation as in readUntil. But, yes, your version is much better. And more reusable, which is very important.

I'll add that and create an alpha tag for 0.7.25.

Why doesn't you use SemVer normally? Just increase patch number and release new version. Because I can't really get this bug fix. I'm stuck with 3 options: use old 0.7.23 or use alpha tag or edit local source of 0.7.24. I like neither of this options.

Hm, I do intend to tag a new release shortly, but right now it doesn't make sense yet to switch to full SemVer (>=1.0.0), because more or less every release still contains some breaking changes or new features. But the plan is to split up the library into multiple pieces and tag some of them with a stable version shortly afterwards.
But you can safely use the alpha tag for now, as there are no critical changes and I will not introduce any of those until 0.7.25.

I'm receiving a EOF before end stream marker on this using the MultiPartPart script with FileMultiPart:
https://github.com/rejectedsoftware/vibe.d/blob/master/source/vibe/inet/webform.d#L262
I don't add CRLF to the end boundary, so I'm wondering if the readLine is erroneous or if I should append CRLF.

According to RFC 1341 there is no CRLF required after the "--", so it looks like the correct thing to do would be to always read two bytes and require them to be either "--" or "\r\n" in case of "--", it should then nullSink.write(bodyReader); to drop the epilogue.

That's what I was assuming, I'll submit a pull request sometime today or tomorrow for this


Use AttributeInterface to mark attributes that should be serialized by the VM for runtime reflection.

iree.reflection is an unstructured grab-bag for everything and the merging/combining code for SIP stuff deeper down in the system always trips me up. We could use an AttributeInterface as a trait marker for allowing attributes to be serialized to the VM. This way, other sources that want to encode information can do so like how the tflite binding generator attaches names/types/etc while keeping the exact encoding format for VM compatibility separate.
This would let us remove the special-casing here:
https://github.com/google/iree/blob/01e8cb54096933fd3d5abd7f2eb269dfdc0f1054/iree/compiler/Dialect/VM/Conversion/StandardToVM/ConvertStandardToVM.cpp#L66-L70
Cleanup how (currently) we aren't storing the reflection attrs on exports but internal funcs and encoding them directly:
https://github.com/google/iree/blob/3c8880e9aa75c0d7363295702401b3705fd191f9/iree/compiler/Dialect/VM/Target/Bytecode/BytecodeModuleTarget.cpp#L240-L259
And generally make the PublicAbiGeneration.cpp file go away (it currently has to parse and manipulate stuff it shouldn't):
https://github.com/google/iree/blob/b7381623652bdcfd67ad2cf836f503c9d634c567/iree/compiler/Dialect/HAL/Transforms/PublicAbiGeneration.cpp#L283-L299

Closing for now until we have more structured information we want to serialize.