Atlas based segmentation with AMU7T template

Description
AMU7T is a 3D template created in 2 contrasts acquired at 7T:

qT1-> from 26 subjects
t2s -> from 72 subjects
an atlas of 50 labels is associated (WM parcels from Levi et al., NeuroImage (2015)., and GM parcels derivated from Hausman, L. Atlases of the Spinal Cord and Brainstem and the Forebrain. (Thomas, 1962).
AMU7T was created in the space of PAM50 (related in spinalcordtoolbox/PAM50#14 ), and in a resolution of 0.17x0.17x0.5mm.

@jcohenadad  this issue is to discuss how to automatically segment based on this new high resolution template/atlas through SCT:
here is a t1q 7T image that was automatically segmented using Sct registration (a) and a method with Optimized parameters (b).

From ISMRM2023
Sct registration (a)
# Register 7T MP2RAGE image  (t1q) to PAM50
sct_register_to_template -i t1q.nii.gz -s t1q_sc.nii.gz -l landmarks_t1q.nii.gz -c t2s -v 1
# Register of  AMU7T_50_labels.nii.gz to to subject space
sct_apply_transfo  -i PAM50/AMU7T/atlas/AMU7T_50_labels.nii.gz -d t1q.nii.gz -w warp_template2anat.nii.gz -o tract_from_t1q_wm.nii.gz -x nn

Optimized parameters (b)
# Register 7T MP2RAGE image  (t1q) to PAM50
sct_register_to_template -i t1q.nii.gz -s t1q_sc.nii.gz -l landmarks_t1q.nii.gz -c t2s -v 1

# Register of AMU26_qt1.nii.gz and AMU7T_wm.nii.gz to subject space using warps
antsApplyTransforms -d 3 -i PAM50/AMU7T/template/AMU26_qt1.nii.gz -o qT1_AMU26_ants.nii.gz -t warp_template2anat.nii.gz  -r t1q.nii.gz
antsApplyTransforms -d 3 -i PAM50/AMU7T/template/AMU7T_wm.nii.gz -o wm_AMU26_ants.nii.gz -t warp_template2anat.nii.gz  -r t1q.nii.gz

# Adjustment of the registration in the subject space with a SyN step
antsRegistration --verbose 1 --dimensionality 3 --float 0 --collapse-output-transforms 1 --output [wm_t1q_s,wm_t1q_sWarped.nii.gz] --interpolation Linear --transform SyN[0.2,3,0] --metric CC[t1q.nii.gz,qT1_AMU26_ants.nii.gz,0.2,2] --metric CC[t1q_wm.nii.gz, wm_AMU26_ants.nii.gz,0.8,2] --convergence [100x75x50,1e-6,50] --shrink-factors 4x2x1 --smoothing-sigmas 2x1x0vox -g 1x1x0 -x [t1q_sc.nii.gz, t1q_sc.nii.gz]

# Register of  AMU7T_50_labels.nii.gz to to subject space using 2 warps
antsApplyTransforms -d 3 -i PAM50/AMU7T/atlas/AMU7T_50_labels.nii.gz  -o tract_from_t1q_wm_ants.nii.gz   -t wm_t1q_s0Warp.nii.gz -t warp_template2anat.nii.gz -r t1q.nii.gz -n MultiLabel

Exp_1. Registration using PAM50:

you should not be using -c t1 because this is a t2w like contrast

you should never do straightening with 5mm thick images

Thank you @jcohenadad
following your suggestion, I worked on the registration of anisotropic images in the subject space:
Registration  using AMU7T in subject space:
sct_register_to_template -i image.nii.gz -s image_wm.nii.gz -l landmarks.nii.gz -c t1 -v 1   -t ../../../template_AMU7T/ -s-template-id 4
sct_apply_transfo -i ../../../template_AMU7T/template/AMU7T_wm_qt1.nii.gz  -d image.nii.gz -w  warp_template2anat.nii.gz -o wm_amu7Ttoanat.nii.gz -x nn
sct_register_multimodal -i template2anat.nii.gz -d image.nii.gz -iseg wm_amu7Ttoanat.nii.gz  -dseg image_wm.nii.gz -o amu7Ttoanat_t1q.nii.gz -m sc_dil.nii.gz -param step=1,type=seg,algo=slicereg,metric=MeanSquares:step=2,type=im,algo=syn,metric=CC:step=3,type=seg,algo=syn,metric=CC,gradStep=0.2

Time = 788 sec.
Atlas AMU7T to subject space using 2 warps
sct_apply_transfo -i ../../../template_AMU7T/atlas/AMU7T_50_labels.nii.gz  -d image.nii.gz -w warp_template2anat.nii.gz warp_template2anat2image.nii.gz  -o atlasAMU7T_to_subject.nii.gz -x nn

Register AMU7T atlas using sct_warp_template (in 2 times)
sct_warp_template -d image.nii.gz -w warp_template2anat.nii.gz -t ../../../template_AMU7T/  -ofolder warp_template_1/
sct_warp_template -d image.nii.gz -w warp_template2anat2t1q_aniso_or_cor_den.nii.gz -t warp_template_1/label/ -ofolder warp_template_2

Visualization of external regions (CSF contour, Central-zone, Anterior-fissure and Septum)
In template_AMU7T/atlas/.. the labels are binary, however, after this registration, non-binary masks were obtained, possibly due to linear interpolations when applying the 2 warps.

Metric extraction using sct_extract_metric
sct_extract_metric -i image.nii.gz -f label/atlas/AMU7T_label_353_bin.nii.gz -method wa -o WA_353_GM_R_fasciculus_dorsolateralis.csv

Weighted average metric (WA ± SD) in the non-binary GM Right fasciculus-dorsolateralis mask  per slice,in a qT1 image (peak in a MS lesion).

Note:
I worked on a second version of the AMU7T atlas and template  (XY zero padding = 256x256) or (45x45mm) to preserve the spinal canal and vertebral bodies on the registered images.

sct_register_to_template -i image.nii.gz -s image_wm.nii.gz -l landmarks.nii.gz -c t1 -v 1   -t ../../../template_AMU7T/ -s-template-id 4

To help the team review your code, I would add a comment that template-id 4 refers to the WM segmentation of the template: https://github.com/spinalcordtoolbox/template_AMU7T/blob/8d68e44d9da9511772d83ffee57b0b094664496f/template/info_label.txt#L8
sct_apply_transfo -i ../../../template_AMU7T/template/AMU7T_wm_qt1.nii.gz  -d image.nii.gz -w  warp_template2anat.nii.gz -o wm_amu7Ttoanat.nii.gz -x nn

Why are you using '-x nn'? And for what purpose are you bringing AMU7T_wm_qt1.nii.gz into the subject's space? (Comments would be welcome)
sct_register_multimodal -i template2anat.nii.gz -d image.nii.gz -iseg wm_amu7Ttoanat.nii.gz  -dseg image_wm.nii.gz -o amu7Ttoanat_t1q.nii.gz -m sc_dil.nii.gz -param step=1,type=seg,algo=slicereg,metric=MeanSquares:step=2,type=im,algo=syn,metric=CC:step=3,type=seg,algo=syn,metric=CC,gradStep=0.2

Why not running these registration parameters while doing sct_register_to_template and specifying a -param flag in the first place?
Sub-sections "Atlas AMU7T to subject space using 2 warps" and "Register AMU7T atlas using sct_warp_template (in 2 times)": what are these? Are they different steps for doing the template to image registration, or are they different strategies that you are comparing? Please describe what is the purpose of each sub-section.
Section "Metric extraction using sct_extract_metric": very cool result!

Thank you ++ @Nilser3 @jcohenadad and the team for all the efforts for integrating the AMU7T into the SCT!
A few comments and questions :
1) - the native image resolution you are working on is 0.30.34 mm3 (ie. anisotropic, with thickness of 4mm, not 5mm)(but that's no big deal ..)
2) -

following your suggestion, I worked on the registration of anisotropic images in the subject space:

I agree anisotropic images are not adequate for the straightening. In the ISMRM abstract, Arnaud indeed registered the template to the subject image. But then, the warping to the AMU7T template was not adequate (-> could you double check that everything is going well when you go back in the template space if you adopt this "template->subject" strategy?)
Also, how many landmarks do you use when running sct_register_to_template -l landmarks? On the native anisotropic T1q images, the vertebrae can not be seen (not as clearly as with other modalities). Consequently, 2 labels may not be sufficient/adequate.
3) - when you use the option "-c t1", registration is made from (or to) the AMU26_qt1.nii.gz, right?
4) -

I worked on a second version of the AMU7T atlas and template (XY zero padding = 256x256) or (45x45mm) to preserve the spinal canal and vertebral bodies on the registered images.

could you please provide more details? Did you start from scratch with a new crop for the native image? Or something else ?
5) -

Weighted average metric (WA ± SD) in the non-binary GM Right fasciculus-dorsolateralis mask per slice,in a qT1 image (peak in a MS lesion).

Thanks for this preliminary try!
When ready with the AMU7T template registration, we will have to investigate more carefully the robustness of the registration when a lesion (or compression, etc..) is present. In this try, the red ROI (supposedly GM fasciculus-dorsolateralis) is more likely a mix of GM and lateral CST. But for a 1st try, this is very cool. Thanks again!

the native image resolution you are working on is 0.30.34 mm3 (ie. anisotropic, with thickness of 4mm, not 5mm)(but that's no big deal ..)

Always good to be precise, thank you. For more transparency and make sure the dev team and @Nilser3 are on the same page with debugging, @Nilser3 can you please upload a 'test' data that will be used as functional tests for the registration commands we are discussing here?

I agree anisotropic images are not adequate for the straightening. In the ISMRM abstract, Arnaud indeed registered the template to the subject image. But then, the warping to the AMU7T template was not adequate (-> could you double check that everything is going well when you go back in the template space if you adopt this "template->subject" strategy?)
Also, how many landmarks do you use when running sct_register_to_template -l landmarks? On the native anisotropic T1q images, the vertebrae can not be seen (not as clearly as with other modalities). Consequently, 2 labels may not be sufficient/adequate.

If using more than 2 labels, the algorithm also performs a through-plane non-linear registration, which is to be avoided with thick slices (to minimize interpolation errors). So I would stick with 2 labels and -ref subject instead of -ref template to bypass straightening.

when you use the option "-c t1", registration is made from (or to) the AMU26_qt1.nii.gz, right?

It's a bit convoluted, but "-c t1" uses label_id=0, as per this code, which indeed refers to as per this code:
https://github.com/spinalcordtoolbox/template_AMU7T/blob/8d68e44d9da9511772d83ffee57b0b094664496f/template/info_label.txt#L4

could you please provide more details? Did you start from scratch with a new crop for the native image? Or something else ?

I agree more details would be important for the dev team to be able to reproduce the results. This is why a 'test' image is useful, as well as a full script to run the registration. @joshuacwnewton @mguaypaq any suggestions where to upload the data/script?

This is why a 'test' image is useful, as well as a full script to run the registration. @joshuacwnewton @mguaypaq any suggestions where to upload the data/script?

While working with the Phillips level files in https://github.com/spinalcordtoolbox/PAM50/pull/3, we used a code/ folder in the root of the repository to store processing scripts. Perhaps we could use the same idea here?
e.g. @Nilser3 creates a separate branch on the AMU7T repo, opens a PR for that branch, then makes sure to commit each update to the workflow as a change to the processing script.
That way, all someone has to do is clone the repo and check out the branch. (And, any further discussion can be direct PR reviews highlighting specific lines of the script itself.)

Thank you @jcohenadad
I have shared a subject HC for test and AMU7T v2 (zero padding in XY plane, dim =256x256) in this PR #6

sct_apply_transfo -i ../../../template_AMU7T/template/AMU7T_wm_qt1.nii.gz  -d image.nii.gz -w  warp_template2anat.nii.gz -o wm_amu7Ttoanat.nii.gz -x nn

Why are you using '-x nn'? And for what purpose are you bringing AMU7T_wm_qt1.nii.gz into the subject's space? (Comments would be welcome)

I consider that the first registration is not enough to have the Template and Subject correctly aligned,
so I use the warp of this first coregistration (warp_template2anat) to bring the WM of AMU7T into a "unadjusted subject space", then I add a second coregistration, based on the images and the WM masks between AMU7T in the "unadjusted  subject space" and the Subject.
Why -x nn?
Linear interpolation tends to introduce blurred edges on the contours, so as our original mask is soft,to preserve this degree of softness I chose to use -x nn

sct_register_multimodal -i template2anat.nii.gz -d image.nii.gz -iseg wm_amu7Ttoanat.nii.gz  -dseg image_wm.nii.gz -o amu7Ttoanat_t1q.nii.gz -m sc_dil.nii.gz -param step=1,type=seg,algo=slicereg,metric=MeanSquares:step=2,type=im,algo=syn,metric=CC:step=3,type=seg,algo=syn,metric=CC,gradStep=0.2

Why not running these registration parameters while doing sct_register_to_template and specifying a -param flag in the first place?

Certainly, I made an experience with 5 steps in -param:
sct_register_to_template -i image.nii.gz -s image_wm.nii.gz -l landmarks.nii.gz -c t1 -v 1 -s-template-id 4  -t template_AMU7T  -s-template-id 4 -param step=1,type=imseg,algo=centermassrot,rot_method=pcahog:step=2,type=seg,algo=bsplinesyn,slicewise=0,metric=MeanSquares,samplStrategy=None,samplPercent=0.2,iter=3,smooth=1:step=3,type=seg,algo=slicereg,metric=MeanSquares:step=4,type=im,algo=syn,metric=CC:step=5,type=seg,algo=syn,metric=CC,gradStep=0.2

the first 2 (the default steps from sct_register_to_template) + 3 steps used with sct_register_multimodal, The results were not the same because in our sct_register_multimodal stage, we used a dilated SC mask (-m sc_dil) to improve the accuracy in the region of interest, so I will continue to optimize the params so that in a single sct_register_to_template we have a more accurate registration.

Sub-sections "Atlas AMU7T to subject space using 2 warps" and "Register AMU7T atlas using sct_warp_template (in 2 times)": what are these? Are they different steps for doing the template to image registration, or are they different strategies that you are comparing? Please describe what is the purpose of each sub-section.

There are two steps I use to register the AMU7T atlas into the subject space, the first step register the AMU7T atlas into the "unadjusted subject space" and the second step takes this unadjusted  atlas and register it into the Subject space.
I know it is a redundant process and creates a unnecessary atlases, it will be simplified if we have only one Warp.

Hi @vcallot

I worked on a second version of the AMU7T atlas and template (XY zero padding = 256x256) or (45x45mm) to preserve the spinal canal and vertebral bodies on the registered images.

could you please provide more details? Did you start from scratch with a new crop for the native image? Or something else ?

I made a zero paddingaround the XY plane without changing neither the PAM50 reference, nor the resolution, nor any interpolation, here is this version,
however I have an issue about it:
In our original template AMU72_t2s: the intensities starts with negative values,
here the original histogram:

and in the zero padding affects this histogram, then maybe we should make a histogram shift, so that it starts from ZERO in the original image? before making the padding, so that ZERO is the background.

This problem is not seen in the qT1 template (fortunately).
Original histogram:

Zero padding histogram:

5) -

Weighted average metric (WA ± SD) in the non-binary GM Right fasciculus-dorsolateralis mask per slice,in a qT1 image (peak in a MS lesion).

Thanks for this preliminary try! When ready with the AMU7T template registration, we will have to investigate more carefully the robustness of the registration when a lesion (or compression, etc..) is present. In this try, the red ROI (supposedly GM fasciculus-dorsolateralis) is more likely a mix of GM and lateral CST. But for a 1st try, this is very cool. Thanks again!

of course 👍

From https://github.com/spinalcordtoolbox/template_AMU7T/issues/4#issuecomment-1641526025:

step=3,type=seg,algo=slicereg,metric=MeanSquares

This step is irrelevant bc at step1 you already do a centermass rot

step=4,type=im,algo=syn,metric=CC:step=5,type=seg,algo=syn,metric=CC,gradStep=0.2

It is unusual to precede seg-based with image-based

Hi @jcohenadad @vcallot
Here is an improved registration based on the following command lines :
sct_register_to_template -i image.nii.gz -s image_wm.nii.gz -l landmarks.nii.gz -c t1 -v 1 -s-template-id 4  -t ../template_AMU7T  -param step=1,type=imseg,algo=centermassrot,rot_method=pcahog:step=2,type=seg,algo=bsplinesyn,slicewise=0,metric=MeanSquares,samplStrategy=None,samplPercent=0.2,iter=2,smooth=1,rot_method=pcahog:step=3,type=seg,algo=syn,metric=MeanSquares,shrink=2,dof=Tz_Rz_Sz,slicewise=1,iter=20 -ref subject

sct_apply_transfo -i ../template_AMU7T/atlas/AMU7T_50_labels.nii.gz -d image.nii.gz -w warp_template2anat.nii.gz -o atlas_2_subject.nii.gz -x nn

sct_warp_template -d image.nii.gz -w warp_template2anat.nii.gz  -t ../template_AMU7T

previously I had to make a manual correction to the WM mask (red outline in the video), and if we compare it with the registered AMU7T probabilistic WM mask via sct_warp_template (light blue mask in the video) a better correspondence of masks is observed.
Video
amu7t.webm
The registered atlas presents some aberrant deformations (43 - 50 sec in the video), which were excluded for the quantification presented in Fig. 01 (notebook here)
Fig. 01

That looks pretty good @Nilser3, good job! 👏

Cool! Great job @Nilser3 ! Looks nice and hope it behave the same with different subjects!


Unit Testing + Package Upgrading

Adds

@testing-library/react and friends

removes

@testing-library/react-hooks (react package now contains these functions)

Updates

React and co to version 18
Jest and co to Version 28

Codecov Report

Merging #180 (979984f) into master (2998d4e) will increase coverage by 14.38%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master     #180       +/-   ##
===========================================
+ Coverage   36.70%   51.08%   +14.38%
===========================================
  Files          43       43
  Lines        1196     1196
  Branches      170      170
===========================================
+ Hits          439      611      +172
+ Misses        599      444      -155
+ Partials      158      141       -17

Impacted Files
Coverage Δ

...ages/use-contractkit/src/contract-kit-provider.tsx
80.00% <ø> (+42.50%)
:arrow_up:

...kages/use-contractkit/src/connectors/connectors.ts
25.95% <0.00%> (+2.12%)
:arrow_up:

...ckages/use-contractkit/src/contract-kit-reducer.ts
42.85% <0.00%> (+17.14%)
:arrow_up:

packages/use-contractkit/src/constants.tsx
77.92% <0.00%> (+24.67%)
:arrow_up:

...es/use-contractkit/src/use-contract-kit-methods.ts
38.09% <0.00%> (+32.14%)
:arrow_up:

packages/use-contractkit/src/utils/sort.ts
57.14% <0.00%> (+35.71%)
:arrow_up:

packages/use-contractkit/src/modals/connect.tsx
61.03% <0.00%> (+46.75%)
:arrow_up:

packages/use-contractkit/src/utils/helpers.ts
67.50% <0.00%> (+50.00%)
:arrow_up:

packages/use-contractkit/src/modals/action.tsx
100.00% <0.00%> (+50.00%)
:arrow_up:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2998d4e...979984f. Read the comment docs.


prefetch: widen evaluations from 32 to 64 bits

useful when someone wants to use grossly huge numbers for segment indices.

I think that making it 64-bit is good, but we should update the documentation:
https://github.com/apache/trafficserver/blob/7a3aaa4b350a6fe3ee6bb17727176757151772a6/doc/admin-guide/plugins/prefetch.en.rst?plain=1#L163
I think this change also re-introduces overflow behavior on addition.


Adjust ui text and add back buttons

Adds back buttons with a stack of menu panels. Make sure if you add a new menu panel in the script you push it onto the stack. Tested all the menu paths and seems to work perfectly. Also changed the buttons to look slightly less generic.

Ah damn i think i increased the text size right before pushing forgetting
some of it would not appear on lower resolutions.
On Thu, 27 Apr 2017 at 19:03, Kjell Iwarson<EMAIL_ADDRESS>wrote:

@lightsoda requested changes on this pull request.
Unfortunately a lot of the text is not visible in the setup menu.
http://pasteboard.co/WpldBGts.png
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Double-Fine-Game-Club/pongball/pull/143#pullrequestreview-35180597,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHxweuloCGQKH3n4GkNdcWsfYnRTrzHTks5r0NhugaJpZM4NKjSC
.

I generally use "best fit" with min max values and that solves a lot of issues with font size.  If the min value is too high this issue could still appear on lower resolutions.  Another option is to change canvas scaling but that affects everything.

The reason I turned off best fit was because all the text was inconsistent.
With a font size of 30 it was legible at the smallest and largest
resolution but then I upped it to 40 without testing properly.
On Fri, 28 Apr 2017 at 09:13, Stewart Martin<EMAIL_ADDRESS>wrote:

I generally use "best fit" with min max values and that solves a lot of
issues with font size. If the min value is too high this issue could still
appear on lower resolutions. Another option is to change canvas scaling but
that affects everything.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Double-Fine-Game-Club/pongball/pull/143#issuecomment-297936837,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHxweghyWzoyWRchnqb2MS8HWD_XJKNNks5r0Z-pgaJpZM4NKjSC
.

Oh right.  That makes sense.  I've also use content size fitters to force objects to fit my text but that doesn't apply here.

@lightsoda I have reverted the font size back down to its previous size and it is visible at the lowest resolution I can run the game at now. This only affects the setup menu and does not touch the main menu.

I seem to have the same problem on this branch. Can't see play instructions before match.

I think this still needs changing to work with intervening updates to BaseTable.unity that were needed for the Humble Release. @jctwood are you comfortable reworking things? If not, just sing out :)


Request: add option to enable check type of protected blocks on chunk load.

There are situations when block has changed or no longer exists while protection is still there. For example: new generation of old chunks, rollback with log block or /replace command. I thought about this and it would be nice to have an option for some sort of cheking protected blocks on chunk load to let LWC know that initially protected block is still there or gone. If initially protected block is not there that protection of that block vanishes.

@Gerolfing Check the wiki; you probably want to use lwc admin cleanup. It's only available as a command because it is an expensive operation. You do not want to run it on chunk load. It would almost certainly cause an unacceptable amount of lag.

@pop4959  Thanks for your answer. With single admin command it's even better.


Build failure on iOS - CHIBasePageControl

Upgraded to the newest version and now I can't build on iOS. I get the error below.
/Users/.../platforms/ios/Pods/CHIPageControl/CHIPageControl/Core/CHIBasePageControl.swift:119:55: error: 'common' has been renamed to 'RunLoopMode.commonModes'
        self.displayLink?.add(to: .current, forMode: .common)
                                                      ^~~~~~
                                                      RunLoopMode.commonModes
Foundation.RunLoop.Mode:9:23: note: 'common' was introduced in Swift 4.2
    public static let common: RunLoopMode
                      ^
/Users/.../platforms/ios/Pods/CHIPageControl/CHIPageControl/Core/CHIBasePageControl.swift:202:60: error: 'common' has been renamed to 'RunLoopMode.commonModes'
        self.displayLink?.remove(from: .current, forMode: .common)
                                                           ^~~~~~
                                                           RunLoopMode.commonModes
Foundation.RunLoop.Mode:9:23: note: 'common' was introduced in Swift 4.2
    public static let common: RunLoopMode
                      ^
/Users/.../platforms/ios/Pods/CHIPageControl/CHIPageControl/CHIPageControlChimayo.swift:97:29: error: type 'CAShapeLayerFillRule' (aka 'NSString') has no member 'evenOdd'
            mask.fillRule = CAShapeLayerFillRule.evenOdd
                            ^~~~~~~~~~~~~~~~~~~~ ~~~~~~~
/Users/.../platforms/ios/Pods/CHIPageControl/CHIPageControl/Core/CHIBasePageControl.swift:119:55: error: 'common' has been renamed to 'RunLoopMode.commonModes'
        self.displayLink?.add(to: .current, forMode: .common)
                                                      ^~~~~~
                                                      RunLoopMode.commonModes
Foundation.RunLoop.Mode:9:23: note: 'common' was introduced in Swift 4.2
    public static let common: RunLoopMode
                      ^
/Users/.../platforms/ios/Pods/CHIPageControl/CHIPageControl/Core/CHIBasePageControl.swift:202:60: error: 'common' has been renamed to 'RunLoopMode.commonModes'
        self.displayLink?.remove(from: .current, forMode: .common)
                                                           ^~~~~~
                                                           RunLoopMode.commonModes
Foundation.RunLoop.Mode:9:23: note: 'common' was introduced in Swift 4.2
    public static let common: RunLoopMode
                      ^
/Users/.../platforms/ios/Pods/CHIPageControl/CHIPageControl/CHIPageControlChimayo.swift:97:29: error: type 'CAShapeLayerFillRule' (aka 'NSString') has no member 'evenOdd'
            mask.fillRule = CAShapeLayerFillRule.evenOdd
                            ^~~~~~~~~~~~~~~~~~~~ ~~~~~~~
** BUILD FAILED **

what version of swift are u using ?

How do I check?

I am experiencing the same issue here, using Swift 4.2 & NativeScript 6.1.2 with tns-ios 6.1.1

Has this been resolved?
I am trying to execute with xcode and it errors on "framework not found "CHIPageControl"
any help would be appreciated.

Can ya'll try building the app with swift v5

I was able to build successfully with Swift 5 by making a Podfile in App_Resources/iOS/Podfile and adding this, not sure if it's the best way but it seems to work.
post_install do |installer|
    installer.pods_project.targets.each do |target|
        target.build_configurations.each do |config|
            config.build_settings['SWIFT_VERSION'] = '5.0'
        end
    end
end

I use "tns run ios" to run the app on a device as well as deploy.
I don't use swift.
When I open in xcode i get this error "Framework not found CHIPageControl"
Its worth noting that I can deploy with no issues. My only problem is opening the project in xcode.
I can open it in xcode successfully without the nativescript-pager plugin. I'm not sure what this error is telling me.
I've tried updating cocoapods and the cli. I am using the latest plugins.
Any help would be appreciated.


Email subject line has escape character added

Bug report

[x] I confirm this is a bug with Supabase, not with my own application.
[x] I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug
When an email template contains a ' (single quote) character in the subject, the rendered subject in the real email contains an escape character.
To Reproduce
Configure an email template to have a subject with a ' character in it.
Expected behavior
The email subject should be rendered identically to how it is entered in the Supabase UI.
Screenshots

System information

OS: [e.g. macOS, Windows] macOS
Browser (if applies) [e.g. chrome, safari] chrome
Version of supabase-js: [e.g. 6.0.2] This occurs on a deployed instance of Supabase, not locally
Version of Node.js: [e.g. 10.10.0] 18

Could replicate the issue, and seems this is a bug, If confirmed, I would like to work on it.

+1 would like to see this fixed!


Please tell us when TextInputFormatter will be fixed?

This regression was introduced in an August 19th engine roll: #38855.
The engine PR that's causing the problem is flutter/engine#9498.
It was reported in #39047, #39182, #39485, #39487, #39223, #39738
This issue was fixed late last week, see #39523, and flutter/engine#11720.
Closing this one as a duplicate.


Better default for protoc_rust::Args::out_dir

https://github.com/stepancheg/rust-protobuf/blob/a1333bf024b39a4f1898a873a0074cde9a5f3d00/protoc-rust/src/lib.rs#L20-L23

The current default value for out_dir via Args::default() is the empty string. I have not found this to be a good default. I would expect for the default to be either:
PathBuf::from(env::var_os("OUT_DIR")?)

or:
PathBuf::from(env::var_os("OUT_DIR")?).join("protos")

in the case that $OUT_DIR is set, which Cargo always sets for build scripts.

This will likely require replacing out_dir: &'a str with the semantically more correct out_dir: PathBuf. I would consider this a beneficial change (in combination with #366).

Args builder is merged.
I think implicit getenv could be dangerous, but a shortcut could be useful, e. g.
Args::new()
    .out_dir_cargo("subdir") // $OUT_DIR/subdir

(Better operation name suggestions welcome)

API is very different in rust-protobuf=3.


Configure all-contributors

@all-contributors please add @fzyzcjy for code, doc, ideas

@all-contributors please add @Hixie for ideas

@all-contributors please add @dnfield for ideas

@all-contributors please add @jonahwilliams for ideas

@all-contributors please add @gaaclarke for ideas

done


Add some very basic stack trace entries for module building

This would have helped track down rdar://problem/38434694 sooner. Cherry-pick of https://reviews.llvm.org/D44498, plus the follow-up removing one of the PrettyStackTrace entries that caused some problems for libclang. Reviewed by @bcardosolopes.

@swift-ci Please test


Build warnings, turn -Werror on CI

When building cardano-node there are various warnings, we should address them and turn on -Werror on CI.

it's probably outdated


Fix re-uploading segment when the previous upload failed

When the segment upload fails after segment ZK metadata is created but before segment is assigned to the ideal state, and the following cleanup didn't run properly (e.g. hard failure, disconnect from ZK), the next upload should be able to recover the previous failure and re-assign the segment (treat the uploaded segment as a new segment)

Codecov Report

Merging #9631 (c665140) into master (3e1aba3) will decrease coverage by 6.10%.
The diff coverage is 48.00%.

@@             Coverage Diff              @@
##             master    #9631      +/-   ##
============================================
- Coverage     69.92%   63.81%   -6.11%
+ Complexity     4930     4909      -21
============================================
  Files          1943     1890      -53
  Lines        103907   101557    -2350
  Branches      15745    15474     -271
============================================
- Hits          72653    64810    -7843
- Misses        26153    32018    +5865
+ Partials       5101     4729     -372

Flag
Coverage Δ

integration1
?

integration2
?

unittests1
67.34% <0.00%> (+0.04%)
:arrow_up:

unittests2
15.70% <48.00%> (+0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ache/pinot/common/metadata/ZKMetadataProvider.java
14.91% <0.00%> (-52.87%)
:arrow_down:

.../controller/helix/core/SegmentDeletionManager.java
73.75% <ø> (ø)

...apache/pinot/controller/api/upload/ZKOperator.java
60.84% <47.82%> (-10.97%)
:arrow_down:

...ntroller/helix/core/PinotHelixResourceManager.java
63.12% <100.00%> (-8.91%)
:arrow_down:

...va/org/apache/pinot/core/routing/RoutingTable.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...va/org/apache/pinot/common/config/NettyConfig.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...a/org/apache/pinot/common/metrics/MinionMeter.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...g/apache/pinot/common/metrics/ControllerMeter.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../apache/pinot/common/metrics/BrokerQueryPhase.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../apache/pinot/common/metrics/MinionQueryPhase.java
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 435 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Support for func/function keyword syntax

Hello,
I'm totally able to use functions in KCL and this is awesome. But it would be nice to have syntax support for this:
schema Config:
    mixin [
        AppIdMixin
    ]
    [...str]: any
    team: str
    teamName?: str
    cluster?: str
    project?: str
    env?: str
    baseId?: str
    appId?: any

protocol AppIdProtocol:
    team: str
    project?: str
    env?: str

mixin AppIdMixin for AppIdProtocol:
    baseId = "-".join([e for e in [team, project, env] if e])
    appId = lambda name: str -> str {
        "${baseId}-${name}"
    }

Functions are operational, though I'm not sure with how many positional arguments, one argument works at least...
To achieve this scenario a workaround setting type to any appId?: any is required, it would great to have ability to support this with some native syntax... WDYT @Peefy ?
Thank you very much!

Yep, I mean since the functions and mixins are out there already, definitely someone will try to use mixed-in functions. What more important rather than arguments imo, is that a user has an explicit notion that the attribute is a function. In my opinion it brings a bit more clarity, when say it declarations could support complementary syntax such as:
schema Config:
    mixin [
        AppIdMixin
    ]
    appId: function str -> str # not just any
    # or maybe just
    # appId: function

mixin AppIdMixin for AppIdProtocol:
     appId = lambda name: str -> str {
        "${baseId}-${name}"
    }

The above provides  more content on what the actual attribute is, automatically suggesting that even if this is a non-hidden attribute it won't ever appear in a structure or anywhere... So my point is solely about explicitness, apart from that it already works as charm))

Yep, this is that what I mean if favoring js lambda syntax...


Added badge

#118

GitHub says this PR has conflicts: "This branch has conflicts that must be resolved"

@deathcap I'll take a look

I can't really fix them as I dont have write access, can you just close this and add the change manually? Thanks


Maine48

Anyone know if a auto hack fkrn bank 101 accts fir iPhone need help on a project security hit me up

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Fix capitalization of GitHub

Sorry, just my OCD coming into play. :joy:

All this commit changes is the capitalization of GitHub in the notification title.

Thanks, @arjunkomath!


Stop uninstall background task if Marathon app vanishes

Partially addresses DCOS-17237.
Apologies for the delay on this, I thought I might not be able to have a test case for it but I eventually found a scenario that reproduced the issue consistently.

@larioj @takirala I updated this with some test changes that should make it easier to diagnose failures. I also deleted a redundant test.


Add fr de tranlsations

@incorbador please check if we need to escape some of the accents

@Aby-JS can you please test this (and have an eye on the impact of the apostrophes)


#156187450 Network Utility Class

What does this PR do?
Adds a network utility class for monitoring network changes
Description of Task to be completed?

create a network utility class
create a broadcaster which utilises the utility to monitor network changes
fix ui-ux bugs

How should this be manually tested?

run git clone to clone this repository
cd to the repository
run `git checkout feature/156187450/network-utility
run app

Any background context you want to provide?
N/A
What are the relevant pivotal tracker stories?

story title - Network Utility Class
story type - feature
story id - #156187450

Screenshots (if appropriate)

Questions:
N/A
[Finishes #156187450]

LGTM.
@codestaintin


[Fixes #121365805] New document replaces first doc in state

What this PR does
Fixes a bug where a new document would replace the first document in the state, regardless of whether it was supposed to or not.
Also, fix subtle bugs with how documents were being broadcasted
using SocketIO.
Relevant Pivotal Tracker Stories
#121365805

Coverage decreased (-0.2%) to 83.183% when pulling 68100db17f2b800ffe13f8010de1166f714f3cca on bug/edit-profile into c5c84e843287473535b9ec82f1057849d4f8adf7 on develop.


Use multiple processes per DataSource in Manager

Pull Request
Description
Fixes #311
How Has This Been Tested?

[ ] No
[ ] Yes, unit tests pass
[ ] Yes, prepare_ml_data.py runs

Checklist:

[ ] My code follows OCF's coding style guidelines
[ ] I have performed a self-review of my own code
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked my code and corrected any misspellings

This is now implemented, and the code works (I think).  But it's no faster!  So closing without merging (because concurrency makes the code harder to debug so, if there's no clear benefit, then it's best to not merge this into main!)


Closes #1420 add synonyms to "mammary alveolar cell"

@bvarner-ebi would you mind putting the closes #1420 on the description in the future so I can easily click on it please? :) thanks! (I know, I'm very lazy :p )

Absolutely, @shawntanzk. On my screen it shows the commit name with a clickable link below the description field, but I will also put it in the description field for future PRs.

I ran make normalise_xsd_string before committing. Are you saying I should run it again before merging?

@bvarner-ebi ah totally missed that, thanks :)

I ran make normalise_xsd_string before committing. Are you saying I should run it again before merging?

huh strange, just noticed that the synonyms you added do not have xsd:string behind them, give a go running make normalise_xsd_string again, if not doesn't matter too much, thanks

Now I see. Thanks for catching, @shawntanzk.
To fix, I manually added the 'xsd:string' type to the new synonyms.


[cov] prep for moving coverage to libra-ops

Motivation
Prep for pushing grcov html to aws from libra-ops.
Have you read the [Contributing Guidelines on pull requests]
Y
Test Plan
CI
Related PRs
Libra ops once this lands will take over coverage responsibilities.

Codecov Report

Merging #5255 into master will increase coverage by 0.1%.
The diff coverage is 24.5%.

@@            Coverage Diff            @@
##           master    #5255     +/-   ##
=========================================
+ Coverage    39.2%    39.4%   +0.1%
=========================================
  Files         878      891     +13
  Lines      107451   107789    +338
  Branches    50965    51454    +489
=========================================
+ Hits        42195    42494    +299
+ Misses      23990    23288    -702
- Partials    41266    42007    +741

Impacted Files
Coverage Δ

common/inject-error/src/lib.rs
7.6% <0.0%> (+7.6%)
:arrow_up:

...roptest-helpers/src/unit_tests/repeat_vec_tests.rs
54.0% <0.0%> (ø)

config/config-builder/src/main.rs
0.0% <0.0%> (ø)

config/management/genesis/src/command.rs
19.7% <0.0%> (+5.2%)
:arrow_up:

config/management/genesis/src/json_rpc.rs
0.0% <0.0%> (ø)

config/management/genesis/src/storage_helper.rs
65.2% <ø> (+1.4%)
:arrow_up:

config/management/operational/src/command.rs
0.0% <0.0%> (ø)

config/management/operational/src/main.rs
0.0% <0.0%> (ø)

config/management/operational/src/test_helper.rs
0.0% <0.0%> (ø)

...management/operational/src/validate_transaction.rs
0.0% <0.0%> (ø)

... and 446 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8f80c93...abb0223. Read the comment docs.

@bors-libra r=sausagee

:pushpin: Commit 159a36d has been approved by sausagee

:hourglass: Testing commit 159a36d3e6b259d67fc500d16fbedc5d53dad10b with merge 698a244449dd3405ab185838b74eb0916fe7bb96...

:sunny: Test successful - checks-actions_land_blocking_test, checks-circle_commit_workflow
Approved by: sausagee
Pushing 698a244449dd3405ab185838b74eb0916fe7bb96 to master...


cleanup spotless workaround code and doc

This is a follow-up of #6441.

cc @HTHou

So, we can run mvn spotless:apply with jdk 17 now?

@HTHou I think with #6441 the spotless version already supports doing this. This PR is to clean up workaround settings and incorrect docs.
I ran mvn spotless:apply locally with JDK17 and it doesn't complain.


don't trust Peekable::count()

It could return zero in the overflow case (in release mode). Unsafe code can't trust it.

Fixed in 6c4f6e9.


Update Dockerfile

Hi @carlhiggs
I've been working on the #375 D.2 point to eliminate multiple URLs for running Jupyter (actually only the second URL works), I created a set default command to launch when the container is run (copied from the OSMnx Docker file set up ). I assume it could be tested only from the host's Docker daemon.  Let me know what you think about launching Jupyter without requiring authentication tokens and if we can proceed with it.

Hi @dapugacheva , its an interesting idea, thanks for looking into this!
I don't know about launching directly into Jupyter Lab, as some users will want to use the web app, and others may have different intentions so like the command line flexibility (although there are commandline options within jupyter lab too).
One possibility is that rather than launching directly into Jupyter Lab, we could update the existing alias for launching Jupyter lab using your approach.
So this line (in the Enhancements branch, the otherwise most up to date branch; we could merge this in with the 'update' branch to make later integration easier),
https://github.com/global-healthy-liveable-cities/global-indicators/blob/fb7b996773faa1b5b74bd193b96c22ea848bae68/docker/Dockerfile#L65
could be edited somehow so lab instead launches the following command and directions,
jupyter lab --no-browser --notebook-dir=/home/ghsci/process --allow-root --config=/home/ghsci/jupyter_notebook_config.py --IdentityProvider.token='' --ServerApp.password='' --ip='<IP_ADDRESS>' --port=8888 & printf '\n$(tput bold)Jupyter Lab$(tput sgr0)\nOpen http://localhost:8888/lab to create or open a Notebook (e.g. example.ipynb)\nTo shut down Jupyter Lab, go to File > Shut Down.\n\n'

So, in the above, I edited the options to Jupyter lab you suggested because of deprecation warnings I received when I ran it:

Running the above edited version looks a bit like this:

But, I think if it were added as an alias
it would look like

the output "[12] 135" would probably be a bit confusing for users, but I'm sure there'd be a way to hide it.
I like the idea of simplifying what is printed to screen, and I'm not necessarily against running Jupyter Lab as a background process.  It would be worth testing if/how it interferes with potential concurrent use of the GHSCI web app.
What do you think about the above suggestion, of not running Jupyter Lab automatically, but rather modifying the alias used to launch it?

The tokens provide authentication security for Jupyter running in a client-server environment. In a localhost only application the tokens are irrelevant for security because the server is only listening on localhost. See also https://blog.jupyter.org/security-release-jupyter-notebook-4-3-1-808e1f3bb5e2?gi=2f3a305c756c

Thanks @gboeing ; that sounds good.  I think this could be useful, but i'm still a bit uncertain about how best to implement this for users --- specifically, how to ensure they don't create duplicate servers using system resources that are inaccessible.  I found this could be a problem, and will explain below.  At the end, I propose a solution - love to hear your thour thoughts @dapugacheva and @gboeing .
I suggested earlier, I think its a good idea to not launch directly into Jupyter Lab.  The current approach provides directions and links for usage of the software, whereas Jupyter Lab is what it is -- different software, which some people will but others won't want to use.
I tried a sketch of how @dapugacheva 's suggestion above could be implemented with modification of the jupyter_notebook_config.py script that is referenced within the lab alias:
https://github.com/global-healthy-liveable-cities/global-indicators/blob/8d6861e2758adbf4be45ffa0d6038baa02d6d370/jupyter_notebook_config.py#L1-L8
I added this line on the end:
c.IdentityProvider.token = ''

So, the server launches when ran in this way, but without any directions:

I tested manually modifying the lab alias to display a print statement after running this, and this works --- launching the server, with directions:

So, that could be nice, but there is a problem.  If a user carelessly doesn't shut down a server, like I didn't apparently, they can spawn multiple server instances.  Despite specifying port=8888 in the config, subsequent instances are launched on incremental ports, yet these cannot be accessed locally (because we only open port 8888). The server on 8888 can be shut down in the browser, but the others can't.  Servers launched with tokens (that produce the output we are trying to hide) can be stopped by entering jupyter server stop XXXX, but when you attempt to shut down servers launched without a token, you get HTTP 403: Forbidden error.
I'll paste screen shots below of the server list -- you can see I was experimenting, and somehow accrued a couple with tokens (ports 8888 and 8889); those could be shut down from commandline.  But servers on ports 8890 and 8891 launched without tokens couldn't be terminated that I could see, except by exiting and restarting the Docker container.

But ... maybe I found a solution, kind of --- if the printf statement is put first, then the server isn't run in the background, which means multiple servers can't be run (or not easily/accidentally):
alias lab="printf '\n$(tput bold)Jupyter Lab$(tput sgr0)\nOpen http://localhost:8888/lab to create or open a Notebook (e.g. example.ipynb)\nTo shut down Jupyter Lab\n  - press Ctrl+C in this terminal window, or\n  - go to File > Shut Down in the Jupyter Lab interface.\n\n' & jupyter lab --no-browser --notebook-dir=/home/ghsci/process --allow-root --config=/home/ghsci/jupyter_notebook_config.py"

So, in the above image, useful guidance is displayed and the user can exit either via Ctrl+C or from Shut down in browser.
The down side is, when they do this, a nonsense echo of the printf code appears to be echoed; I don't know why:

But perhaps that's no big deal, and is an improvement on the current state of things

useful directions are given for accessing and closing Jupyter Lab
the unneccessary Jupyter Server logging is hidden
a few artifacts appear, but mostly only after exiting when maybe it doesn't matter

Do you think this could be a useful approach then?

adding the extra line in jupyter_notebook_config.py, c.IdentityProvider.token = ''
changing the echo 'alias lab="printf \"\n$(tput bold)Jupyter Lab$(tput sgr0)\nOpen http://localhost:8888/lab to create or open a Notebook, e.g. example.ipynb\nTo shut down Jupyter Lab\n  - press Ctrl+C in this terminal window, or\n  - go to File > Shut Down in the Jupyter Lab interface.\n\n\" & jupyter lab --no-browser --notebook-dir=/home/ghsci/process --allow-root --config=/home/ghsci/jupyter_notebook_config.py"' >> ~/.bashrc

If you know how to get rid of the artifact printf statement when shutting down the server, I'd love to know!

Just closing this one as the changes discussed above have now been implemented as part of other changes; thanks @dapugacheva for the suggestion


Cost estimation needed

Hello,
Can I have an estimation about pricing of this scenario?
Thanks,
Edi

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 73dc00bc-c571-72c0-c3de-8e1a35b9e27a
Version Independent ID: 424eb559-47a7-bcc5-e567-e428e59c11cb
Content: E-commerce website running in secured App Service Environment - Azure Solution Ideas
Content Source: docs/solution-ideas/articles/ecommerce-website-running-in-secured-ase.md
Service: architecture-center
Sub-service: solution-idea
GitHub Login: @adamboeglin
Microsoft Alias: pnp

Thanks for reporting - this issue is under review.  This is a Microsoft Internal DevOps Bug ID AB#199476

@edidako  in a similar architecture we have pricing that covers a small, medium and large scenario. https://docs.microsoft.com/en-us/azure/architecture/example-scenario/apps/ecommerce-scenario#pricing.  This one uses App Service, you were looking at App Services Environment (ASE).  Please make the appropriate changes in the cost calculator for your needs.


0.7.x Windows support

Windows support -> heading for world domination!
To use:

Install Windows 10 in a VM
Install Ruby+Devkit from rubyinstaller.org, make sure things are in the path
clone repo
open cmd, type: ridk enable
got to repo dir
type make test
see tests passing
celebrate

Remarks:
There is no ssl support.
There is no fork() on Windows, so only one worker will work.
There are no unix sockets on Windows, sadly. Named pipes would be the best solution, but semantics would change and make things complicated.
Instead i opted for using tcp ports for local communication, as it keeps things simple, keeps the socket semantics and its possible to use WSAPoll() for everything.
https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsapoll
The point of all of this is, to make iodine and thus isomorfeus development accessible to a broader audience. Patches for iodine follow some time soon.
Although tests work, an actual app does not respond ... need to fix.

Hi @janbiedermann ,
I love where you're going this, however, this week I'm super busy and won't have time to look it over (between my birthday obligations and a few projects I am fresh out of seconds, not to mention minutes or hours).
I'l review it soon :)
Cheers and THANKS!!!
Bo.

Hi @boazsegev ,
"I love where you're going" - you mean world domination? Sure, love it. ;)
No need to hurry, it probably still needs a week to ripen.
With dbf1151 finally the example app works.
So with -w1 -t1 in the example "Hello wordl!" app i get with Apache ab ab.exe -c10 -n 100000 -k http://localhost:3000/:
Requests per second:    53475.98 [#/sec] (mean)
Not bad at all, small birthday present - Happy Birthday!
 Still some work to do.

Fixed all issues i came across so far (not yet pushed), but there is one issue, that needs a bit more attention.
On windows there seems to be no determinable socket limit and socket fds (HANDLEs in winsock terminology) are all sorts of numbers and with multile threads often way beyond the fixed allocation of fio_data.
For example:
fio_data->info[1024]
allows for fd numbers up to 1023
Windows may for example provide a fd/HANDLE of 454654 as result of a accept() call, especially when using muliple threads. This will lead to a reference of
fio_data->info[454654]
and thus subsequently a crash
So for Windows a indirection would be required, that maps actual fds (HANDLEs) to fio_data->info[index].
I am thinking, how best to achieve that with as little as possible changes but did not arrive at a final solution yet.
I was considering the usage a fio hash map, maybe.
Any idea/inspiration very welcome. @boazsegev would be great, if you would have an idea?

To solve the above problem u just added a fio_data->info[i]->soket_handle. Works nicely, forward lookup is fast, reverse lookup scans fio_data>info[]. Fast enough. Single threaded it still shows excellent performance and after fixing another crash bug (not #105) it works very reliable. Socket handling seems to be correct.
However, there still is a performance issue when running multithreaded. It may be, because of the all the locking required, not sure yet. Dont pull this, Ill recreate this PR ans submit a new one.

Multi threaded with 1500 connections with a capa of 1024, 5 minute run:
  Reqs/sec      2081.54    1415.22   35504.73
  Latency      744.42ms   512.71ms     11.10s
  HTTP codes:
    1xx - 0, 2xx - 575632, 3xx - 0, 4xx - 0, 5xx - 0
    others - 29841
  Errors:
    dial tcp [::1]:3000: connectex: Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte. - 29764
    the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection - 77

Single threaded with 1500 connections and capa of 1024, 5 minute run:
Reqs/sec     42136.97    3294.27   56649.43
Latency       35.61ms   265.46ms     14.87s
HTTP codes:
1xx - 0, 2xx - 12609920, 3xx - 0, 4xx - 0, 5xx - 0
others - 31205
Errors:
dial tcp [::1]:3000: connectex: Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte. - 26851
the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection - 4354
Throughput:     8.65MB/s

Bombarding http://localhost:3000/ for 10s using 500 connection(s)
[=================================================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     82086.76   16529.77  235717.14
  Latency        5.99ms     3.62ms   557.00ms
  HTTP codes:
    1xx - 0, 2xx - 831271, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    17.13MB/s

Great performance and multithreaded! Finally! And very stable, stable enough for development.
Now need to check websockets ...

Hi @janbiedermann ,
First I want to thank you for all your amazing work. I love how minimal your code changes seem (though I think it would be even better if we could somehow abstract away some of the code changes to a centralized location using macros / inline functions).
I am still at the beginning of learning what needs to be done for Windows integration and how I could test the WinSock API code (I don't have the OS and don't really want to shell out money for an OS I don't believe in)... in the meantime I found myself asking a few core questions...:

I am assuming the "Linux Sub-System" wouldn't help us without forcing the user to compile/run the code within that system...? There's no way to enable that subsystem from within the code? - I ask because I assume the code will run "as is" on that subsystem, I mean, I read it runs Redis.

Is there an upper limit for the Windows fd value (i.e., 64K)? If so, is it reasonable to set capa to that value? - I ask because I doubt if scanning the connection data Array (the reverse lookup) will still work decently when using a 100K connection capacity...

To clarify point 2: With WebSockets and (maybe in the future) HTTP/2, facil.io supports tens of thousands of concurrent connections (some of whom may be dormant for long periods of time while waiting for push events)... on Linux facil.io would be limited mostly by machine resources, not by any internal implementation detail. I would love Windows users to have the same experience.

Do you have a recommendation how I might test the Windows code? Maybe we could add a Windows OS CI test? Is there a windows solution for developers of some kind?

Again, thanks for all the amazing work!
Cheers!
Bo.

Hi @boazsegev ,
my pleasure, i am too learning. Meanwhile, at several places the #ifdefs got a bit ugly. I agree, a beautiful solution would be nice to have.

The WSL2 is very well integrated into Windows. Its easy to access files from Win or Lin, one way or another. Even Visual Studio Code can open files or directories from within the WSL2 and its very nice to work with. Networking between those two still needs a few things to consider. But i dont intent to instruct users to install and learn about the WSL, how to upgrade it to WSL2, what to consider to get networking services running between the systems, adjust firewalls, etc.... I would prefer a easy to use Windows only way. facil.io runs without any changes in WSL, as WSL is just a standard Linux.
Memory is the limit for sockets on Windows too. I agree, scanning the array is not meant as a high performance solution, but as a way to get things running, without changing to much of facil.io. I chose 1024 sockets as capa, learning from xitami, which has internally the same limit and very high performance. But truly, without limits and random socket fds, the per socket task queue of facil.io needs a completely new way.
2.5. I also noticed, that the thread handling of facil.o isnt perfectly suited for poll/wsapoll, limiting performance and response time. But i am very pleased with the current performance and its very well suited for using Windows at least as development platform. However, with some more effort, rethinking the task queue, thread handling and using Window's completion api, better performance can be achieved for lots more sockets. We maybe can have a look at https://oatpp.io/ and see how it handles things on Windows. But for the moment, getting iodine running on Windows with good enough performance is my first priority.
You can get free Windows VMs from Microsoft, which work for 90 days, https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/ .  I use the HEB CI (hands, eyes, brains) ;-), otherwise i have no idea as of yet. I havent been a Windows user or developer until recently.

Ill try to get websockets running asap. Latest code is here https://github.com/janbiedermann/facil.io/tree/0.7.x_w
When everything works, ill recreate my PRs, to get things in order.

Latest instructions for Windows:
Install windows
Install Ruby30 with DevKit from rubyinstaller.org
Install git for Windows
clone repo, checkout 0.7.x_w
copy repo to new dir
cd to new dir
then ridk enable
then bash
then ./scripts/new/clean
then exit bash
then make DEBUG=1
then tmp\fioapp.exe
example app should work
curl http://localhost:3000/
for testing i used: https://github.com/codesenberg/bombardier/releases/tag/v1.2.5

Websockets work, however ... after using some and many with multiple threads the app busyloops doing nothing, just heating the room. No idea why. Debugger shows nothing of help.
I improves socket accuracy, meaning, as good as possible avoid passing invalid sockets to WSAPoll().
Added a simple websocket benchmark script, to be used with https://k6.io/

ok, poll() on Linux too has some issues dropping websocket connections and later on not taking any connection anymore. So there definitly is a bug in the common poll()/WSApoll() code. But on Linux it doesnt start busy looping. Anyway, it sometimes shows as this, on Linux and on Windows:
`AddressSanitizer:DEADLYSIGNAL
==7935==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x56219f509567 bp 0x7f0d401fda60 sp 0x7f0d401fd940 T4)
==7935==The signal is caused by a READ memory access.
==7935==Hint: address points to the zero page.
#0 0x56219f509566 in fio_flush lib/facil/fio.c:3661
#1 0x56219f5038d3 in deferred_on_ready lib/facil/fio.c:2668
#2 0x56219f4ffad3 in fio_defer_perform_single_task_for_queue lib/facil/fio.c:1256
#3 0x56219f4ffc12 in fio_defer_perform lib/facil/fio.c:1294
#4 0x56219f4ffd62 in fio_defer_cycle lib/facil/fio.c:1334
#5 0x7f0d44171608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
#6 0x7f0d43f49292 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV lib/facil/fio.c:3661 in fio_flush
Thread T4 created by T3 here:
#0 0x7f0d441c5805 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a805)
#1 0x56219f4fe209 in fio_thread_new lib/facil/fio.c:814
#2 0x56219f4fff2b in fio_defer_thread_pool_new lib/facil/fio.c:1366
#3 0x56219f50dc34 in fio_worker_startup lib/facil/fio.c:4478
#4 0x56219f50e407 in fio_sentinel_worker_thread lib/facil/fio.c:4568
#5 0x7f0d44171608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
Thread T3 created by T0 here:
#0 0x7f0d441c5805 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a805)
#1 0x56219f4fe209 in fio_thread_new lib/facil/fio.c:814
#2 0x56219f50e4fb in fio_sentinel_task lib/facil/fio.c:4582
#3 0x56219f50e9cf in fio_start lib/facil/fio.c:4640
#4 0x56219f5f899b in main src/main.c:254
#5 0x7f0d43e4e0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
==7935==ABORTING`
On Linux it restarts the worker, everything fine, until next time. On Windows, Windows kills the thread and starts spawning new ones and killing them again all the time, also spinlock waiting for something and thus causing the cpu usage and doing nothing.

There is no reason access to ->flush should fail, its never changed to something else than the default flush function. So there must be some stack/heap/whatever overrun or something. But at this time i am getting nowhere with this. How to find the culprit?

Actually this might be related to a bug I wanted to track down for a while that's probably related to race conditions in the protocol assignment logic and might be unrelated to the poll implementation.
However, I think the poll and bug tracking should take a slightly backstage as the issue might be resolved using a different concurrency IO design and a new poll wrapper (already written, see later on).
I want to dedicate the main thread to the system and then all user code will run on "user" threads. This will ease much of the lock contention and allow buffered data to be sent even when all user threads are busy and without stressing the locks on the IO buffers (new client requests will wait for user threads, while buffered responses will be sent with lower latency).
I'm very much behind on the new design for version 0.8.x and I don't want to pause the Windows support avenue, which is why I didn't point out before that there's additional future work in the pipeline. I thought I'll just port your code to the new design once it's ready.
You can have a look at the re-write for all the FIOBJ API and core type logic here (the API is much clearer, as the naming is more consistent). There's also a new poll engine in there... but there's no IO design just yet, this is just a library full of building blocks.
The future 0.8 design will eventually go into this repo which is currently full of junk that I need to replace (the beginning of the new design is partly on my system and mostly in my head).
One of the issues the new design raises is the question of the fd data array (which is great for POSIX OSs, but not portable). Replacing it with dynamic pointers might ruin memory locality (which is hugely important for some performance details) but could reduce some race conditions for multi-threaded memory access, so I have to write it all to test performance before I know if dynamic memory is a pass or fail.
I wish I knew what I'm doing, but right now I'm spread thin on a big number of projects and nothing is moving fast enough.

Hi @janbiedermann ,
I'm having some issues and I wanted your opinion.
For the last week or so I've been intensively attempting to work out a manageable approach to Windows development.
My main concerns are maintenance and testing.
If we're looking at getting iodine on Windows, then we need to get facil.io core functionality to support WinSock2 and windows IO polling (WSAPoll), as well as use the Windows file reading (and writing) API for the  FIOBJ data types.
Threads and fork-ing on iodine use the Ruby implementation, so we can delay working on that... though I think a threading API that allows a developer to choose between co-rutines (i.e., green threads) and native OS threads will help those using facil.io without a Ruby layer.
I think it would be better to code these few layers in a native Windows API rather than use MSYS2 / Cygwin / Mingw.
The facil.io custom memory allocator requires the ability to allocate memory on aligned addresses (it uses masks and pointer addresses to figure out the original allocation block and access the block's metadata)... I'm not sure I can rewrite it for Windows.
However...
Keeping an intel computer around just so I can run a virtual Windows dev machine and provide support for Windows sounds a little too heavy duty for me. I cannot test contributions without running the code and I don't want to run a windows machine (not even a virtual one) on my network. That's one high-risk OS where everyone (including Microsoft) it trying to mine personal data from both the machine and the local network.
So... I don't know what to do.
Maybe Windows support will (forever) remain "unofficial" and "untested", where I trust contributors without testing their code... or maybe Windows support will just fade away and Windows related bugs / issues / features will never get addressed.
I intend to try my hand at incorporating what you've already done so that we are as close to Windows compatible as we can be, but... this week that I spent learning the Windows API (and their types, they have to name all their types in capitals?)... it really made me feel that keeping up with Windows support over time might be impossible :(
What do you think? Will we be able to maintain support for Windows over time? Will there be enough contributors? Who will test Windows code? ...?
Sorry if I'm ranting, it's just that this Windows experience reminded me why I left that OS both as a user and as a developer.
Thanks for your input.
Bo.

Hi @boazsegev ,
ill give my best:

If we're looking at getting iodine on Windows, then we need to get facil.io core functionality to support WinSock2 and windows IO polling (WSAPoll), as well as use the Windows file reading (and writing) API for the FIOBJ data types.

Thats basically done in my code. There are some windows support functions, well, posixish wrappers for native win api.
Others did that work, libuv would be available, but not necessary.

(Threads) I think it would be better to code these few layers in a native Windows API rather than use MSYS2 / Cygwin / Mingw.
Sure, but i am not sure, if there would be any benefit, even performance wise. Sticking to pthreads of msys/mingw keeps thing simple and portable. Cygwin isnt a target for me, too slow.

I think it would be better to code these few layers in a native Windows ...
I agree. But honestly, that Windows tooling is beyond my understanding. All i need is a compiler and header files and libs. What you get is a web installer with lots of fancy things that i dont understand and gigs over gigs of software. No idea what this all is about. And it seems to change over and over again with each new VisualStudio release or Windows Release or whenever some new marketing guy is hired at Microsoft. I am unable to keep up with this ever changing vastness. However, i found out, that are "build tools" available and after installing >1G compiler, headers and libs are there, somewhere on the system ...
https://visualstudio.microsoft.com/de/downloads/#build-tools-for-visual-studio-2019

The facil.io custom memory allocator  ....
It seems Windows doesnt provide aligned memory allocation, so that must be done "manually". I omitted that part and just use malloc/free, good enough for now.

Keeping an intel computer around just so I can run a virtual Windows ...
What kind of computers do you have?
Sure, i understand your concern. Having facil.io as nice toolkit for VisualStudio Development, native with msvc, would be great. But no, i am not interested in that, all i want is iodine on Windows ;). And wouldnt need to have bells and wistles, just work, and fast enough, stable enough and all is good for me. Also it wouldnt need millions of ws connections, if it can handle around 300000 that would be fantastic.

What do you think? Will we be able to maintain support for Windows over time?
No, i think we dont need to. Once Isomorfeus is running on Windows the world will gradually switch to Linux or *BSD ;-D
because they all will see, that its a much better experience on Linux or *BSD ;-D

Will there be enough contributors? Who will test Windows code? ...?
Maybe, i could imagine that people might be interested in something fast and simple on Windows, not sure. But maybe not.
I was thinking about creating a facil.io-DevKit for Windows, complete with some simple opensource C IDE, compiler (tcc), headers (from mingw), libs (also from mingw, where necessary). And then implement real time development, with a file watcher/proxy daemon, that compiles superfast with tcc on file change and reloads the browser view via a proxy injected script ...
But probably i wont have time for that.

Sorry if I'm ranting, it's just that this Windows experience reminded me why I left that OS both as a user and as a developer.
Clearly, its the same for me, but i would like isomorfeus to be more accessible for a broader audience of developers/users because of reasons. As long as Windows is dominating the desktop, i think i can maintain and support iodine/isomorfeus on Windows in such a way, that isomorfeus development works nicely. So for that i am testing.
For super mega native Windows performance a lot more effort would be required.

And now for something completely different:

I think i found a bug, with the fix websocket things seem to work better on Linux:
in  fio_write2_fn i think, at the end it should look like this:
locked_error:   fio_unlock(&uuid_data(uuid).sock_lock);   fio_packet_free(packet);   /** fallthrough and free buffer */ error:   if (options.after.dealloc) {     options.after.dealloc((void *)options.data.buffer);   }   errno = EBADF;   return -1; }
Currently it doesnt fallthrough and just returns, and so the buffer passed in case of locked_error never gets freed.
Further on linux with poll() testing websockets, there seem to be "waves" of activity and sleep, like connections/data are coming in but facil.io does nothing for 5s and then is very busy for 1s and then again doing nothing for 5s. (times not accurate, just to explain). When i stop the debugger in such situation for example all threads are waiting for a lock to fio_postoffice.pubsub.lock or all threads are waiting for poll(). Still investigating.
Further on Linux now i am experiencing another memory error, where malloc fails with SIGSEGV. Still investigating.
Windows still is busylooping early with websockets and both linux and Windows still drop/loose ws connections. Still investigating.

Why do you use your custom locking and not pthread_rwlock?

I think i found the problem with the locking.
fio_filter_dup_lock_internal

locks the collection
doing stuff, getting channel
locks the channel
unlocks the collection

channel gets unlocked in fio_subscribe
fio_unsubscribe

locks the channel
doing stuff
locks the collection
...

Lets look at 2 threads running that interleaved by accident:
1 fio_filter_dup_lock_internal
2 fio_unsubscribe
1 lock collection
1 doing stuff getting channel
2 locking channel
2 doing stuff
1 locking channel --> waiting
2 locking collection --> waiting
--> deadlock, busy spinning

Alright, fixed all that and pushed latest code to 0.7.x_w but i got a new problem:
Thread 35 received signal SIGTRAP, Trace/breakpoint trap.
...
Thread 35 (Thread 16080.0x299c):
...
#6  0x00007ff8e8a19c9c in msvcrt!free () from C:\Windows\System32\msvcrt.dll
#7  0x00007ff759deeb72 in fio_ls_remove (node=0x191ac962670) at lib/facil/fio.h:3329
Only when using websockets.
No idea where that comes from now. Somewhere malloc/free is confused within websockets.
Hormal HTTP works like charm.


As a user, I want to retrieve the population of a city so that I can analyze demographic trends and understand the population size of a city

SQL:
select cty.Name , c.Name , cty.District , cty.Population  from city cty inner join country c on cty.CountryCode = c.Code
where cty.Name = '%city%';
Parameters:
%city%

report31.xml

report31.xml

Output with "Amsterdam" input as the city:

Name
Name
District
Population

Amsterdam
Netherlands
Noord-Holland
731200


Array#join is slower in Crystal than in Ruby

Ubuntu 16.04.1
1GB RAM (Free memory 400MB+)
On VPS machine
Each test is ran 3 times:
ruby -v
Ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]
crystal -v
Crystal 0.26.1 [391785249] (2018-08-27)
LLVM: 4.0.0
Default target: x86_64-unknown-linux-gnu
$ crystal run --release 1.rb
00:00:00.033265433
$ ruby 1.rb
0.015399671000011494
1.rb
require "benchmark"

cols = 10
rows = 1000
data = Array.new(rows) { Array.new(cols) { "x"*1000 } }

time = Benchmark.realtime do
  csv = data.map { |row| row.join(",") }.join("\n")
end

puts time

I'm surprise if an unopitimize code does affect Crystal to run slower than Ruby?
Is it due to LLVM 4.0 on Ubuntu being too old where LLVM 6.0 on macOS Homebrew is newer?
Tested latest Crystal 0.26.1 on macOS Mojave is definitely slower than Ruby.2.5 as other have reported.

Have you tried compiling it and running it separately? I'm not really sure why this would make a difference, but it produces completely different results for me.
With cols = 100 and rows = 10000:
$ ruby wat.cr
22.24634099297691
$ crystal run --release wat.cr
00:00:23.799129445

$ crystal build --release wat.cr

$ ./wat
00:00:07.604624759

@DestyNova Super strange, that shouldn't produce a difference.
@proyb6 You are totally right, Crystal is performing worse than Ruby here. At first I thought it was because maybe Ruby's memory allocator was faster than Crystal. Then I got curious and checked how join is implemented. I was sure join was defined in Enumerable, but just in case I checked whether its defined in Array too. And it is! I wondered why...
I turns out there's a possible optimization to do if the array consists entirely of strings. In the general case, join will use String.build and successively append the elements and the separators, reallocating memory if needed. But if all the elements are strings (and we convert the separator to a string if it's not a string already), we can compute the total memory needed for the final string: separator.bytesize * (array.size - 1) + array.sum(&.bytesize).
I will send a PR with this optimization soon.
@proyb6 Thank you so much for reporting this!  It's helpful if we can compare the benchmark based from the source code in Ruby Performance Optimization:
https://pragprog.com/book/adrpo/ruby-performance-optimization
Assuming Rubyists and newbies will evaluate the results from those source, I hope we get better and reliable!

I always keep adding ❤ to Ruby, it's incredible all the tweaks, optimizations and great thoughts that there are in the entire codebase. All of this usually goes un-noticed...

@asterite

Super strange, that shouldn't produce a difference.

Yeah it does seem counterintuitive. It might be because my laptop only has 8 GB of RAM and that's nearly all used by a couple of Elixir programs, Firefox and Chromium. Perhaps the compiler uses enough RAM that the system starts using swap memory, and a few seconds are needed to go back to normal... if I add a "sleep 10" to the program before the benchmark starts, it performs better (but still not as fast as running the executable created by crystal build --release).

Interestingly, if I add GC.disable to the beginning of the program, then both crystal build --release wat.cr && ./wat and crystal run --release wat.cr show similar performance.


Sending mail without authentication

Hi, I've just installed Alertmanager along with prometheus and grafana, and when setting up the mail to be able to receive email alerts I don't see an option for sending emails (in my case using the Gmail servers) without authentication.
Since we have Google mail services, we can just use their SMTP without authentication.
Does anyone know whether that is possible with alertmanager?
Thank you.

Please ask usage questions at https://groups.google.com/forum/#!aboutgroup/prometheus-users
Nowhere is it documented that you need to use authentication.


Allow accumulation to be not reset by writing to file in History

A suer @sdrabenh was asking if History could perform accumulation, I.E. do the accumulation for the time average but not divide the count. The motivation for this is that the stretched hazardous weather experiments effort needs accumulated precipitation (basically precipitation from 0-1z, 0-2z, etc...  for each day) as one of the products we must submit. I looked at it and indeed that in addition to min/max, a straight accumulation was allowed by the coupler.
However, there was a one problem. The coupler essentially has two alarms and receiver two frequencies. The history collection has two keywords "frequency" which controls how often the collection writes, and one I was unaware of "acc_interval" which defaults to the frequency and does not appear to be used, but does exist.
Both frequencies are passed to the coupler, and two alarms are made. One is a "time to couple" alarm which controls basically when the coupler puts the result in the output. The 2nd is "time to clear" which clears the accumulators and counts. Currently both alarms are set to the frequency of "frequency", passed to the coupler, even though it gets a 2nd frequency which is the acc_interval from history.
Essentially for all practical purposes the way we run history, the "time to couple" and "time to clear" run on the same alarm. So when you write, you clear, and if you wanted something like hourly accumulated precipitation and that reset once a day, you are out of luck.
The 2nd thing I discover is that History has this notion of a reference time for the collection which defaults to 0z, so the "frequency" of writing is relative to this ref time, but this ref time is never passed to the coupler, so if you do time average, unless you set the ref_time to the start time you intend to run, the time-averaging behavior is not right. I will make a separate issue for that.
However, the coupler looks like it was designed to do this, but almost but not quite there. I made some minor tweaks and I think I've fixed it so @sdrabenh could do what he wanted.
The changes that have to take place are:

the "time to zero" alarm in the coupler should be set to the acc_interval passed from history
some sort of reference time for this alarm needs to be passed. Since you can't pass this I highjacked the offset of the varspec which is not passed via history. I added an acc_ref_tinme, then I compute an offset so that the time-to-zero alarm can be set to ring at the time the user wants which may not be the start time which the time to couple alarm is hardwired to.
I removed 2 bits of logic the coupler, one was that when the couple is called it stops accumulation for that item even if you don't zero, but @atrayano could not think of a use case for that. 2nd for the time to zero alarm I removed the restriction that is has to start at or later than the model start time.

@bena-nasa Can we close this issue? or is there more work to be done?

I'm still not sure if this is fixed or not. Will contact @bena-nasa


#130 Fix displaying of border corner

Fix #130
All Submissions

[ ] Have you followed the guidelines in our Contributing document?
[ ] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions

[ ] Does your submission pass tests?
[ ] Have you lint your code locally prior to submission?

Changes to Core Features

[ ] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your core changes, as applicable?
[ ] Have you successfully ran tests with your changes locally?

@eliely Thanks for taking the initiative. Just a thing, can you please share a screenshot of this fix as well?

@Ashutosh00710 Sure. I've noticed that you add border radius to whole svg element, but border was inside on rect element and thets why is was cut on corners


gpg requires msys-sqlite3-0.dll

My installation broke after the update because the new gnupg requires libsqlite which I did not have. Also it seems that the PKGBUILD has more runtime dependencies than are actually used? Or am I missing osmething?
$ ldd $(which gpg)
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ff99b840000)
        KERNEL32.DLL => /c/Windows/system32/KERNEL32.DLL (0x7ff99b430000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7ff998d50000)
        msys-assuan-0.dll => /usr/bin/msys-assuan-0.dll (0x5a3090000)
        msys-bz2-1.dll => /usr/bin/msys-bz2-1.dll (0x461220000)
        msys-gcrypt-20.dll => /usr/bin/msys-gcrypt-20.dll (0x4aee70000)
        msys-gpg-error-0.dll => /usr/bin/msys-gpg-error-0.dll (0x45a000000)
        msys-2.0.dll => /usr/bin/msys-2.0.dll (0x180040000)
        msys-readline7.dll => /usr/bin/msys-readline7.dll (0x4da630000)
        msys-sqlite3-0.dll => /usr/bin/msys-sqlite3-0.dll (0x5798a0000)
        msys-z.dll => /usr/bin/msys-z.dll (0x522fe0000)
        msys-iconv-2.dll => /usr/bin/msys-iconv-2.dll (0x5603f0000)
        msys-intl-8.dll => /usr/bin/msys-intl-8.dll (0x430b30000)
        msys-ncursesw6.dll => /usr/bin/msys-ncursesw6.dll (0x5fcb10000)
        USER32.dll => /c/Windows/system32/USER32.dll (0x7ff998fb0000)
        GDI32.dll => /c/Windows/system32/GDI32.dll (0x7ff99a850000)

@jeroen ping

maybe it is better to build with  --disable-sqlite ?


Add Content-Type in header will cause a OPTION request

I want to use fetch to send a POST request to my server.
But when I set the "Content-Type": "application/json" in headers,
it will cause a OPTIONS request, if I remove  Content-Type in headers, it will be fine
The following is my code

//...
var f = fetch(host+"/", {
      headers: {
        "Accept": "application/json",
        "Content-Type": "application/json"
      },
      method: 'POST',
      body: 'name=Hubot'
});
f.then(json)
 .then(success)
 .catch(error)

Is your request cross-origin (i.e. spanning different hosts)? Then this is expected.

Which browsers have you tested with? Chrome has a native implementation of window.fetch that skips the polyfill. You might compare the behavior between Chrome and Firefox (which would use the polyfill code).

I've solved this problem, just like @mislav said. It's a problem with same-origin policy.
So allow cross domain on backend will solve this problem.
I'm sorry, this is my mistake and thanks your suggestions.

What is actually sending the OPTIONS request? Is it fetch itself, or is it the browser? If CORS is to be used, is the server supposed to support handling of OPTIONS requests? This is the first library I've used that has caused OPTIONS requests to be sent, so I'm just wanting to understand why and whether or not I need to be handling them.

Mozilla on CORS:

Additionally, for HTTP request methods that can cause side-effects on user data (in particular, for HTTP methods other than GET, or for POST usage with certain MIME types), the specification mandates that browsers "preflight" the request, soliciting supported methods from the server with an HTTP OPTIONS request method, and then, upon "approval" from the server, sending the actual request with the actual HTTP request method.  Servers can also notify clients whether "credentials" (including Cookies and HTTP Authentication data) should be sent with requests.


Piwik broken?

I talked to Edna about that. Apparently the account on uberspace ended and we need to reinstall and host it somewhere else. Checking out the possibilities on heroku.

I am at uberspace as well. Maybe I can do that?

@tyranja Edna said that it's probably too late to move it to another account. I will take care of it since you already have so many things to do.

OK. But it seems to be really easy.https://robtranquillo.wordpress.com/2015/06/29/piwik-installieren-auf-uberspace/

:) so do it if you want!

No, I just pasted it there, in case I want to find it again. Its your thing! I don't even know what to do with it.
Its raining in BCN :(

@tyranja humpf, we need a mysql database to store the piwik information. which means we can't use heroku for that. So what about you installing Piwik on your uberspace and I will implement everything afterwards?

OK

done. ouufff!


Add lock slot feature for sorting feature.

I would like to suggest add lock slot feature, locked slots on inventory, chest, ... will be ignore by sorting feature.

+1


It's unclear what role "sometool" is supposed to be playing

In this section it's not clear what "sometool" is supposed to be.  I'm… guessing… it's an executable target that is used in a build step? IMO it would be helpful if this guide were phrased in terms of the actual commands invoked.  It would be even more helpful if there was a tiny sample project that, say, preprocesses a .swiftx file into a .swift file with the identity transformation, or something.

I guess I begin to see that explained in the next section… but it would be nice if it weren't left a mystery until later.


Missing origins:random_teleport on 1.20.1-<IP_ADDRESS>

undying.json
As such, certain functionality available on fabric is inaccessible on forge.

Hi, this power type is not in the 1.20.1 version of Origins, thus we cannot reasonably add it to the 1.20.1 version of Origins Forge. Thanks for understanding.


rp2040.dtsi defines wrong number of IRQ priority bits

Describe the bug
The RP2040 dtsi (zephyr\dts\arm\rpi_pico\rp2040.dtsi) defines the wrong number of IRQ priority bits.
&nvic {
	arm,num-irq-priority-bits = <3>;
};

However RP2040 CM0+ implements only 2 bits. Thus IRQ priority settings are wrong in general. The RPI_PICO_DEFAULT_IRQ_PRIORITY (equals to 3) becomes 1.
From the RP2040 datasheet (build-date: 2023-03-02), section 2.4.5
M0PLUS: NVIC_IPR0 Register
Offset: 0xe400
Description
Use the Interrupt Priority Registers to assign a priority from 0 to 3 to each of the available interrupts. 0 is the highest
priority, and 3 is the lowest.
Note: Writing 1 to an NVIC_ICPR bit does not affect the active state of the corresponding interrupt.
These registers are only word-accessible

Expected behavior

number of IRQ priority bits = 2
RPI_PICO_DEFAULT_IRQ_PRIORITY (defined as 3) correctly configured in NVIC_ICP registers.
NVIC_IPR[0..3] Prio Values = 3, instead of 1

&nvic {
	arm,num-irq-priority-bits = <2>;
};

Impact
IRQ priorities don't work as expected. RPI_PICO_DEFAULT_IRQ_PRIORITY (defined as 3) becomes 1.
Logs and console output

**Environment

OS: Windows
Zephyr SDK

1a0b1124c6 - (13 days ago) MAINTAINERS: charger: Adds entry for "Chargers" - Ricardo Rivera-Matos (HEAD -> main, origin/main, origin/HEAD)

@renestraub Thanks for reporting this! Fixed here: #65785


Add package for lpeg

LPeg (http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html) is an expression
parsing library for lua created by the developer of lua.

@TheCycoONE,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

Thanks for the PR!
I think you missed a Build-Depends: lua :).
Is there a license available in the download itself? It's much better to get the license directly from the downloaded sources so that we won't accidentally miss a license change during an upgrade.

Ah yes, I'll fix that.
The copy of the license in the download is at the end of the included lpeg.html; formatted as html. They didn't include a separate copy. Seems to be Roberto's style, as  you can see https://github.com/Microsoft/vcpkg/tree/master/ports/lua had to include their own license as well.

Build-Depends: lua added.

Thanks for the PR!
The html-formatted version is still preferable to checking in a copy. Thanks for pointing out the lua port -- it probably just slipped through.


Add debug-bundle to the debug-pack

I just installed the debug-pack and was wondering, why my dump(...) wasn't showing in my webprofiler toolbar.
I would intuitively assume, that the integration code for dump() comes with the installation that adds dump().
Maybe one should add this to the profiler-pack as well?

See https://github.com/symfony/profiler-pack/pull/3

Thank you @apfelbox.


Setting cookies within getContext?

Is there any way of setting cookies within getContext? I'm trying to fetch an internal endpoint to refresh a session, which adds set-cookie headers but they're getting dropped. Furthermore, in load functions, I can use the passed in fetch to call /my/endpoint but when fetching in getContext this results in an invalid url error, so I'm guessing in load functions fetch is bound to a different context.
For now I'm using an ugly interval workaround in the $layout to accomplish the refresh but it'd be nicer if I could keep that out of the browser and only do it on demand.

If you need to adjust the headers that are being returned by SSR'd pages or by endpoints, the handle hook is the place to do that.

handle is called after getContext obviously, but I can work around that by attaching the set-cookie headers from the getContext internal fetch on the request as custom headers, and then setting them on the response in handle. A bit hacky, but it works.


Unexpected target/handler behavior with respect to blocks

I would expect the following two assertions to behave similarly, at least to the point that they would both pass, but they don't.
expect("foo").to eq("foo")
expect { "foo" }.to eq("foo")

In the same vein, I would expect these two assertions to behave differently but they are identical.
expect { "foo" }.to eq("foo")
expect(proc { "foo" }).to eq("foo")

It seems that the handoff between the target and the handler could be improved by somehow providing context/insight into whether the incoming "actual" value is something dynamic to be evaluated (a given block) or a static value (even a proc) given as an argument.
I'm hoping it's worth a conversation. Thank you!

How does one get the rspec matcher to implement supports_block_expectations??

How does one get the rspec matcher to implement supports_block_expectations??

Just define it as a method inside your matcher definition, e.g.
class MyMatcher
  def supports_block_expectations?
    true
  end

  # ...
end

If you are using the matcher definition DSL, you can just call supports_block_expectations:
RSpec::Matchers.define :my_matcher do
  supports_block_expectations

  # ..
end

hi @myronmarston , thanks for the reply - I'm fairly new to Ruby and rspec. I really appreciate the helpful info!

@myronmarston
You must pass an argument rather than a block to use the provided matcher (eq "foo"),
or the matcher must implement `supports_block_expectations?`.

is a really confusing message. It made me think eq got a block but needs an argument. Couldn't figure it out until googled up this issue.

@nicktime what is your code snippet that generated that message?

@myronmarston https://github.com/nicktime/tomograph/blob/add-json-support/spec/lib/tomograph/api_blueprint/yaml_spec.rb#L70
when put in a curly braces like this
      expect { subject.to_tomogram.map(&:to_hash) }.to eq(tomogram_hash)

So I was thinking the problem is with tomogram_hash.

Do you have a suggested wording improvement? It means expect was given a block but eg(tomogram_hash) needs a value

I'd suggest
You must pass an argument rather than a block to `expect` to use the provided matcher (eq "foo"),
or the matcher must implement `supports_block_expectations?`.

Confusion is only about which method it is that can't take a block, otherwise message is clear.

Thanks :)


extend OpenTelemetry module for Apache HTTP Server with arm support

The existing docs say currently apache otel module only works for  x86-64 instruction set and glibc version greater than 2.17
https://github.com/open-telemetry/opentelemetry.io/blob/main/content/en/blog/2022/instrument-apache-httpd-server/index.md
This feature is requesting support for arm as well.

I've built an autoinstrumentation image (ghcr.io/pl4nty/otel-operator-autoinstrumentation-apache-httpd) and it's been stable for a few months with httpd and nginx. Happy to send a PR but I haven't written tests or anything - it's just a few lines in build.gradle and common.gradle


build: unpack only named git deps

This is an I extension your patch in nix-community/naersk#167. It  no longer removes repeated git repos from the result and then uses this information to only unpack crates that are actually in the lock file. This decreased build time drastically for build with lots of git dependencies.

@yusdacra Any chance you were able to take a look yet?

Sorry, I didn't take a look at this yet... I can merge it if it works fine though :) I didn't really look at the naersk PR either because I have been working on other stuff and dream2nix.

@yusdacra Thanks, that would be great. I tried dream2nix on my rust project too, but it crashes due to some recursion issue, so I will stay with naersk for now.

It would be appreciated if you could file a bug report to dream2nix repo about the issue. I'm still trying to iron out issues with it (biggest one being git deps not being properly supported right now, but i have a PR for that), so it is nice to get feedback.


installation: public records, restricted fields?

After basic access control is done, see basic I3B-Access issues, we should decide whether there is still a need to restrict some fields in an otherwise fully public record.
If not, good.
If yes, prepare an example for the Invenio 3 demo site, and amend integration tests.

At INSPIRE legacy we were indeed hiding certain fields and making them available only to catalogers. I personally found this too complex of a requirement, so if it is too complex to implement I guess we can decide for a different technique (e.g. shadow records or similar).

I think this is fully the job of the serialiser. E.g. on Zenodo I have serialiser which only includes the files if a user actually have access to the files. So if we need it, then it's perhaps a "Restricted MARC21 serialiser" which is capable of remove specific tags. Note that indexing/searching should be taken into account as well.

The only issue with a serializer is that once you start to get many of them you risk to forget to filter away fields from one of them. Maybe we could have a facility that can be hooked to filter fields before passing the record on to the serializers?

Indeed, there are fields in records that should not be visible in general, but are required e.g. for cataloguers. While we don't have it that much in bibliographic records, we more likely have it in authority data, a classic being email addresses in people authorities. How it's solved, I admit, I don't care to much, but I agree with @kaplun that it should be defined in some central place to avoid forgetting the filtering.

Generic utilities to filter JSON yes, and also great if your instance can centralise the filtering of records. However, you may have serialisers that need access to the restricted data in order to do a correct serialisation (without exposing the restricted data). That's why I suggest e.g. that we can implement Invenio 1/2 behaviour via a Restricted MARC21 serialiser.
You can compare it to rendering a template for a record - the template has access to the full record, but can take decisions based on current_user and permission checks whether something should be displayed or not.
On Zenodo I simply stick all the internal/restricted data in an _internal property in the record, and remove it when I do e.g. the indexing.


⚠ Bofrost.es has degraded performance

In 7ef26ed, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1435 ms

Resolved: Bofrost.es performance has improved in 24f8c6a after 6 minutes.


Use proper config key to disable partition indexes

The integration tests and the example configuration file did not use the proper configuration key for disabling partition indexes, which we renamed from create-dense-index to partition-index late in the development process of that feature. This fixes that oversight.

:memo: Reviewer Checklist
Review this pull request by ensuring the following items:

[ ] All user-facing changes have changelog entries
[ ] User-facing changes are reflected on vast.io

I assume it does not actually assert the absence of indexes. Something we should fix eventually (not here).

You're correct. The test only checks whether the output is still correct with partition indexes disabled.


Enhance RouteConfig: Page title, breadcrumb and auth

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

Separate Title of page and title of menu
Support hide route from menu - just not add "menu"
Support add more breadcrumbs
Support overdrive page title and breadcrumb
Fix issue if user log-out and go to log-in, then back to dashboard, menu will show empty
Support check permissions via app.guard

What is the current behavior? (You can also link to an open issue here)

What is the new behavior (if this is a feature change)?

Other information:

Hi @TrungVuAnt, thanks much you made a huge work! The only thing is currently we also working on router/menu refactoring thus it would be really hard to merge this fully. So, I presume it's better to wait until our changes are out, and then we can either manually merge some parts of your PR or you will be able to submit some separate PRs based on the code version we release. Thanks again!

Hi @nnixaa, I understand it. I still left this pull request here, so you can take any useful part. I hope that you can finally the structure of this framework soon, so we can help to add more features or help you fix issues.
Thank for your work.


Add all_frames flag so that embedded videos are fixed, too

So, currently this addon does a great job of fixing videos on the YouTube site, but it has one flaw.  It doesn't work on embedded YouTube videos.  This Pull Request simply adds the necessary flag so that the script will get injected into the embedded video iframes, too.

Any chance this could get merged?

Sorry, I wasn't watching the repo. This looks good.

v1.2.1 submitted.

Thanks


Unclean Event Registration causes onChange to trigger incrementally

When using ember-sortable as described in the README, the onChange action is triggered multiple times and increases by 2 for every drag event.
I created a twiddle to outline this problem:
https://ember-twiddle.com/c75a926136ccbbb68963f3c2cf4be136?openFiles=controllers.application.js%2C
To reproduce open up the console and start dragging the items. As you can see "reorder" is logged first 2 times, then 4, then 6 and so on.
This is likely due to eventListeners being applied without being removed. I was able to fix this by replacing releaseEvents in mixins/sortable-item.js with removeEventListener without any noticeable problems.
https://github.com/heroku/ember-sortable/blob/master/addon/mixins/sortable-item.js#L396
I'd be happy to create a PR to address this issue. 🙂

A PR would be very welcome. Thanks for reporting!

Merged to master, closing.


Update: Posit.Positron version 2024.07.0 (2024.07.0-125)

Created by 🥟 Dumplings in workflow run #7009.

Log
Updated: 2024.07.0-113 → 2024.07.0-125
Submitting WinGet manifests
Found existing pull requests:
Update: Posit.Positron version 2024.07.0 (2024.07.0-107) - https://github.com/microsoft/winget-pkgs/pull/165070
Update: Posit.Positron version 2024.07.0 (2024.07.0-113) - https://github.com/microsoft/winget-pkgs/pull/165222
Update: Posit.Positron version 2024.07.0 (2024.07.0-100) - https://github.com/microsoft/winget-pkgs/pull/164700
The existing pull requests are ignored as the version is updated while the real version isn't
Creating manifests
Uploading manifests and making commits
Creating a pull request

Microsoft Reviewers: Open in CodeFlow

Close with reason: Duplicate of #165070;