[ci] Try to use GCP Bazel cache for more jobs

Summary
This change lets more jobs read from the cache, and also allows more jobs to write to the cache when running for the master branch.
It's being tested in this Azure run which will write to the cache. A second run will need to be triggered to see if we actually read from the cache.
Details
This change:

Switches usage of ./bazelisk.sh to ci/bazelisk.sh in pipelines.

This script connects Bazel to our GCP bucket containing a Bazel cache.
This allows CI to read from the cache.
If the GCP_BAZEL_CACHE_KEY variable is set, it also writes to the cache.

Adds downloads for the file containing cache write credentials to more jobs.

This file is stored encrypted in Azure and cannot be downloaded by PRs from forks.
The location to this file is automatically stored in the GCP_BAZEL_CACHE_KEY_SECUREFILEPATH environment variable.
The file is only downloaded for the master branch, however I have changed this for this test run to show that the cache can be written to.
In summary: having this downloaded allows the master branch to write to the cache.

Thanks @jwnrt for looking at this, this looks like a great idea. It seems that now we will have many instances of
  - task: DownloadSecureFile@1
    condition: eq(variables['Build.SourceBranchName'], 'master')
    name: GCP_BAZEL_CACHE_KEY
    inputs:
      secureFile: "bazel_cache_gcp_key.json"

which is quite error-prone and very specific to our workflow. What do you think about having a yaml file that we can include so we can do instead something like:
- template: ci/use-bazel-remove-cache.yaml

and maybe add a parameter to specific whether access is just read, or read-write?

I've extracted this step into a template but haven't added an r/rw parameter since these credentials are only for writing. The GCP bucket is always read if using the ci/bazelisk.sh.
I've tried to make this clearer by giving the template the name load-bazel-cache-write-creds.yml so it's more obvious they just for writing.

Oh also I should mention that even with this change, I don't think the Bazel cache is working in CI at all.
Before this PR, the sw_build job was set up correctly to use the cache (it had the GCP credentials to write and it was using ci/bazelisk.sh to read) but as far as I can tell has always rebuilt.
I tried to set up a local environment to read from the cache too, but no success. These are the (convoluted) steps I used (thanks to @nbdd0121 for helping with this):

Added the following to ~/.bazelrc to replicate what ci/bazelisk.sh builds:

build --remote_cache=https://storage.googleapis.com/opentitan-bazel-cache
build --remote_upload_local_results=false
build --remote_default_exec_properties=OSVersion="Ubuntu 20.04.6 LTS"

Set HOME="/root" - this is what the CI uses and the HOME env var is non-hermetic, so may have influenced the cache?
Set PATH=/tools/verilator/v4.210/bin:/tools/verible/bin:/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin which is what sw_build uses - we extracted this by running env in this PR's CI run.
Execute the same build command that sw_build does.

We did not observe the cache being used at all. There must be something else contributing to the cache that we didn't account for.

Thanks for reviewing, I asked @nbdd0121 to look over the Bash and we simplified it a bit more


Renderer log is missing

I ran into some trouble with Insiders that I wanted to trace
opened output
no renderer log.. 😕

After reload this back

After reload this back

@jrieken Is there any more info you can provide to reproduce this? Did you follow any specific steps?

@jrieken Is there any more info you can provide to reproduce this? Did you follow any specific steps?

No further steps, no log viewer 😆

No further steps, no log viewer 😆

This can happen only if the corresponding log file was not created yet. I think it is hard to speculate why it was not yet created. I am not sure if you checked the dev tools console at that time, have you seen any errors ?

This can happen only if the corresponding log file was not created yet. I think it is hard to speculate why it was not yet created. I am not sure if you checked the dev tools console at that time, have you seen any errors ?

No, I was busy drilling into another issue and didn't have patience to investigate

No, I was busy drilling into another issue and didn't have patience to investigate

I understand. Do you mind keeping an eye on this and if you get into this again, can you please try to get more info like:

Check if there are any errors in the console
Open logs folder and see if the log file for that window is created or not
Also any specific steps that lead to this

I understand. Do you mind keeping an eye on this and if you get into this again, can you please try to get more info like:

Check if there are any errors in the console
Open logs folder and see if the log file for that window is created or not
Also any specific steps that lead to this


add dashboard flash message

Is there a ticket which adds the front-end component of this? If not, please make one before closing this ticket.

@alexthemitchell  done


Algorithms in C++

I'd be happy to contribute algorithms in C++. @geekquad let me know what you think of starting contributions in C++.

I wanted to add Sorting & Searching Algorithms in C++

@RonLek @Siddhant-K-code awesome guys. Love the spirit.
We are just editing. We will be adding C++ track in a day. Please join us on Slack for any updates. Thankyou :)

Please check this issue #198
I have listed the Sorting Algorithms in C++

assignment me

I wanted to add some graph traversal, sorting & Searching Algorithms in C++(optimizedone)


03-Quering-AOpenAI.ipynb cell 6: Code fails if there are no answers

Needs to be changed to
display(HTML('
Top Answers
'))

for index,search_results in agg_search_results.items():
    if '@search_answers' in search_results:
        for result in<EMAIL_ADDRESS>            if result['score'] > 0.5: # Show answers that are at least 50% of the max possible score=1
                display(HTML('
' + 'Answer - score: ' + str(round(result['score'],2)) + '
'))
                display(HTML(result['text']))

this should be fixed now


Enhancement/optimal scroll position

Provides support for snapping back to highlighted text (bookmarks)!
Changes

Removed all direct assignments to self.view.scrollRow/Col. This is now all done in updateScrollPosition function now. You can control whether a function manually updates the view position by utilizing self.__skipCursorScroll and self.__skipUpdateScroll.
When you have a highlighted block of text and snap to it, the program will try to set that block of text at a position according to the optimal position preferences in default_prefs.py
Anything that causes the screen to snap to a position (not scroll up/down) should make use of the optimal cursor position.

I actually wanted to change updateScrollPosition from cursorPageUp/Down to cursorMoveScroll so it would be easy to keep track of what called it for future debugging, but it would be slightly less efficient. It wouldn't really provide any visible decrease in performance. What do you think about this change @dschuyler?

Maybe it would actually be better if we were to just make everything call updateScrollPosition when they want to move the scroll position (thus replacing the purpose of cursorMoveScroll). Any previous call to cursorMoveScroll could then call updateScrollPosition and then cursorMove.
I think maybe I'll work on refactoring/documentation sometime soon (although I don't really know all the functions quite yet).

Updated with new changes. Requested changes have been processed. Lots of refactoring and dividing functions into smaller ones!


Try to get rid of protoc begin required for golem-cli

This is probably only possible if TypeAnnotatedValue in wasm-rpc is converted back to a non-protobuf generated type.

Alternatively we can try to just publish cli binaries for some platforms, see https://github.com/bytecodealliance/cargo-component/blob/main/.github/workflows/publish-binaries.yml

We don't have time for this before 1.0 so removing from the milestone (but will attempt to publish binaries)

This and #1073 will be solved by the same set of changes that are refactoring the core crates and introducing optional features to them.


Synchronizing React State with Canvas state

Sorry about the weird title, couldn't think of anything better.
One thing that I noticed when extracting the components and functions was the usage of forceUpdate basically everywhere. I remember in reading one of the issues that there is an interest in adding elements array into application state. Now, there is a problem with how we use forceUpdate and adding elements array into application state. In majority of cases, we are mutating the elements array in-place, which causes the usage of forceUpdate to rerender the scene when things change.
By investigating the code, there are about 3-4 places where the elements array is being mutated outside the component. My suggestion is to convert these functions into utilities that return the final array instead of mutating them in-place. Then, perform array updates within the Application. This way, we can make the scene synchronized with app state.
Additionally, we can store selectedElements in a separate state instead of consistently looping through the elements array and comparing if an element is selected. This can improve performance because loops, mutations etc will be smaller. But mainly, this is going to separate logic and concerns from the elements. Because I personally look at selected elements as a UI tool instead of them being part of the scene (even if we do draw boxes around them, it still has nothing to do with the scene -- it is being only used by the Panels).
I suggest doing this in two (or three if you agree with the last change) parts:

[ ] Remove mutations from scene, element, and history functions and change App component code to use those functions (we can do something like scene.elements = newElements to update the global elements array)
[ ] Move elements array into application state and remove all the uses of forceUpdates
[ ] Separate selected elements state array from elements state array and update application code.

Yes to everything. That would be much better indeed :)

I can be of great help on this @GasimGasimzada, let me know if you want any help 👍
This will be tricky to pull off with a lot of people contributing features because we will most likely have to rebuild them on top at the same time we refactor, but it should definitely be done sooner or later.

@enzoferey I am currently working on a system that I call action system in this PR: #298. The goal of this PR is to modularize actions. If you want to, you can also get engaged in the discussion.


Band expression for MultiBaseReader

ref https://github.com/cogeotiff/rio-tiler/pull/523#discussion_r963304743
Right now when passing expression to MultiBaseReader (STAC) methods, we assume each asset is one band e.g:
with STACReader("stac.json") as stac:
    ima = stac.tile(701, 102, 8, expression="green/red")
    print(ima.count)
    >>> 1  # it returns a 1 band image

if one asset has more than one band and if no other option is set, the result could be unexpected.
The way expression works is by using
def apply_expression(
    blocks: Sequence[str], bands: Sequence[Union[str, int]], data: numpy.ndarray,
) -> numpy.ndarray:
    return numpy.array(
        [
            numpy.nan_to_num(
                numexpr.evaluate(bloc.strip(), local_dict=dict(zip(bands, data)))
            )
            for bloc in blocks
            if bloc
        ]
    )

where we assign a name to each band within the numpy array
data = numpy.zeros((3, 256, 256))
list(zip(["b1", "b2"], data))
>>> [('b1',
  array([[0., 0., 0., ..., 0., 0., 0.],
         [0., 0., 0., ..., 0., 0., 0.],
         [0., 0., 0., ..., 0., 0., 0.],
         ...,
         [0., 0., 0., ..., 0., 0., 0.],
         [0., 0., 0., ..., 0., 0., 0.],
         [0., 0., 0., ..., 0., 0., 0.]])),
 ('b2',
  array([[0., 0., 0., ..., 0., 0., 0.],
         [0., 0., 0., ..., 0., 0., 0.],
         [0., 0., 0., ..., 0., 0., 0.],
         ...,
         [0., 0., 0., ..., 0., 0., 0.],
         [0., 0., 0., ..., 0., 0., 0.],
         [0., 0., 0., ..., 0., 0., 0.]]))]

In ☝ we have a 3 bands data but a 2 bands list ("b1", "b2"), and we construct the dict that will be passed numexpr. We can see that no errors is raised while we have 3 bands and only 2 names.
Proposal
with #523 we now have bands names all the way during the processes, e.g
with STACReader("stac.json") as stac:
    ima = stac.tile(701, 102, 8, assets=("green", "red"))
    print(ima.band_names)
    >>> ["green_b1", "red_b1"]

It means that in theory expression could be in form of expression="green_b1/red_b1"
with STACReader("stac.json") as stac:
    ima = stac.tile(701, 102, 8, expression="green_b1/red_b1")
    print(ima.band_names)
    >>> ["green_b1/red_b1"]

internally rio-tiler will know that for expression "green_b1/red_b1" we want to read assets green and red. Then we will have an image with band_names="green_b1", "red_b1". Those names will be used in the apply_expresion directly because apply_expression is a new method of the ImageData class
https://github.com/cogeotiff/rio-tiler/blob/55ae89749204de11eef26e06c962d9744241f132/rio_tiler/models.py#L278-L283
potential issue
Another question is what happens when we use asset_expression?
with STACReader("stac.json") as stac:
    ima = stac.tile(701, 102, 8, assets=("green", "red"), asset_expression={"green": "b1+2"})
    print(ima.band_names)
    >>> ["green_b1+2", "red_b1"]

☝ we have then a band name that is "green_b1+2" so if we wanted to use expression we would have to write
expression= "green_b1+2/red_b1" but that won't work because rio-tiler will try to parse the main expression and re-apply! +2 to green_b1+2!!!
One solution would be to use _b1 as the first band of the asset even if there is expression applied.
Status Quo ?
While I feel ☝ is better, I understand that it's a big breaking change.

In ☝ we have a 3 bands data but a 2 bands list ("b1", "b2"), and we construct the dict that will be passed numexpr. We can see that no errors is raised while we have 3 bands and only 2 names.

I've added a check in https://github.com/cogeotiff/rio-tiler/pull/523/commits/d883df00f134aed83ad5c1097e4952ff827b812e to make sure the number of bands and data shape match

It means that in theory expression could be in form of expression="green_b1/red_b1"

I like this. We are breaking things anyways so might as well get it right now. Representing the full hierarchy of asset::band seems like the best way to support applying expressions across multi-band assets.

One solution would be to use _b1 as the first band of the asset even if there is expression applied.

I think appending _b1 is a good way to normalize applying of an expression to single and multi-band images.

Another question is what happens when we use asset_expression?

Can asset expressions be removed entirely if a user can pass assets to the expression?  It seems to me the difference between expression and asset_expression is the former may be used to combine data from multiple assets while the latter applies one expression to each asset individually.  Which means everything you can do with asset_expression you could accomplish with multiple calls to expression:
# one asset_expression
asset_expression={"green": "b1+2", "red": "b1+3"}

# two expressions
expression="green_b1+2"
expression="green_b1 3"

Applying multiple expressions like this would be much easier if ImageData.apply_expression did not mutate the ImageData instance but instead created a new ImageData instance with modified data.  This way a user could do something like:
with STACReader("stac.json") as stac:
    ima = stac.tile(701, 102, 8)
    first_expression = ima.apply_expression("green_b1+2")
    second_expression = ima.apply_expression("green_b1+2")
    third_exression = ima.apply_expression("green_b1+2/red_b1")

This example ☝ got me thinking that maybe including expression in the COGReader.tile function call is overloaded now that it is an instance method on the object returned by that call 🤷

🙏 thanks @geospatial-jeff

I like this. We are breaking things anyways so might as well get it right now. Representing the full hierarchy of asset::band seems like the best way to support applying expressions across multi-band assets.

+1

Can asset expressions be removed entirely if a user can pass assets to the expression

I think yes! +1

Applying multiple expressions like this would be much easier if ImageData.apply_expression did not mutate the ImageData instance but instead created a new ImageData instance with modified data.

Yes, just one thing: in your example you still need a way to tell the tile method which assets to read
with STACReader("stac.json") as stac:
    ima = stac.tile(701, 102, 8, assets=("green", "red"))
    first_expression = ima.apply_expression("green_b1+2")
    second_expression = ima.apply_expression("green_b1+2")
    third_exression = ima.apply_expression("green_b1+2/red_b1")

This example ☝ got me thinking that maybe including expression in the COGReader.tile function call is overloaded now that it is an instance method on the object returned by that call 🤷

Sadly we still need to pass the expression to tile so it parse it and know which asset to read


How to assert if string is lexicographical after/before ?

Is there a smarter (more verbose) way of asserting
should.ok("zzz" > "aaa")

or is there a way to register own assertion methods?
Thanks.

Hi.
First of all take a look on first line of readme (repo was moved).
yes you can add own assertions:
var should = require('should');

should.Assertion.add('method', function(val) {
this.params = { operator: 'to be lexicographically less', expected: val };

this.assert(this.obj > val);
});

'zzz'.should.method('aaa');

If you need more help recreate issue in correct repo.


Implement Chat MeMessage

Reference: https://api.slack.com/methods/chat.meMessage
Acceptance Criteria:

[ ] Spring Slack Client is extended with given method
[ ] Spring Slack Test Client is extended with given method

resolved by #280


Properly detect Asterisk instances built from git

Asterisk from git uses a different version string to indicate
that it is not from a release tarball.
This commit updates the regular expressions to support:
    Asterisk 13...
    Asterisk SVN-branch-13...
and Asterisk GIT-13

thanks for the patch.


Brackets are misinterpreted

This is the bibentry:
{
  title = {{In (Co)Inductive Type Theory}},
}

I'm getting:
In the 'title', put a space after the closing bracket

Hey, I will do it, can you assign it to me?


nix-copy-closure problem

Hello,
I'm trying to copy a closure from a nixos box to another but I'm unable to make it work.
I've built a new haskellPackages.xmonad (0.12) on my lenovo machine and I want to copy the closure on my dagobah machine.
# tony at lenovo in ~/repo/perso/nixpkgs on git:update-xmonad x [20:05:23]
$ nix-copy-closure --to tony@dagobah  /nix/store/d5vgamnf1hhqd8pz8xp9vnasj4kv7xz8-haskell-xmonad-ghc7.8.4-0.12-shared
zsh:1: command not found: nix-store
unable to connect to ‘tony@dagobah’; falling back to old closure copying method
zsh:1: command not found: nix-store
Died at /nix/store/ffig6yaggbh12dh9y5pnf1grf5lqyipz-nix-1.8/lib/perl5/site_perl/5.20.1/x86_64-linux-thread-multi/Nix/CopyClosure.pm line 100.

Note:

both machines are on nixos 14.12
lenovo and dagobah are on the same network and see each other
I connect from lenovo to dagobah
I use ssh-agent and ssh connection is ok:

# tony at lenovo in ~/repo/perso/nixpkgs on git:update-xmonad x [20:09:58]
$ ssh tony@dagobah
Last login: Wed Mar 18 19:38:31 2015 from <IP_ADDRESS>
Syndaemon already running...
 ____________________________________
 / I wonder if I should put myself in \
 \ ESCROW!!                           /
  ------------------------------------
          \   ^__^
           \  (oo)\_______
              (__)\       )\/\
                   ||----w |
                   ||     ||

# tony at dagobah in ~ [20:10:04]
$

Is there something I do wrong?
Thanks.
Cheers,

Make sure you can run nix commands through ssh:
ssh tony@dagobah nix-env -h

The problem might be related to the fact that the default shell is zsh (apparently).

Ahh I didn't see the zsh. @ardumont, simply source .nix-profile/etc/profile.d/nix.sh from ~/.zshenv. zshrc is only used for interactive shells, and as such is not sourced before running remote commands through ssh non-interactively (as nix-copy-closure does).

Make sure you can run nix commands through ssh:
ssh tony@dagobah nix-env -h

Right, baby steps!

If the above fails make sure you're sourcing .nix-profile/etc/profile.d/nix.sh in your shell init file (bashrc)

# tony at lenovo in ~ [22:57:51]
$ ssh tony@dagobah nix-env -h
zsh:1: command not found: nix-env

Indeed, it fails.

The problem might be related to the fact that the default shell is zsh (apparently).

Yes, I use zsh as my default shell.
It seems to be the reason indeed.

zshrc is only used for interactive shells, and as such is not sourced before running remote commands
through ssh non-interactively (as nix-copy-closure does).

I did not know that, thanks!

Ahh I didn't see the zsh. @ardumont, simply source .nix-profile/etc/profile.d/nix.sh from ~/.zshenv.

I do not have such file as ~/.nix-profile/etc/profile.d/nix.sh (I did not install nix, I use the one that comes along nixos, so nix 1.8).
There exists a file on my system though /nix/store/ffig6yaggbh12dh9y5pnf1grf5lqyipz-nix-1.8/etc/profile.d/nix.sh.
--
After forcing a nix install on my dagobah box using nix-env:
nix-env -i nix

The link nix.sh appears where it's supposed to (not sure this is the way though).
Also after adding a new file ~/.zshenv file in my dagobah box with this content:
[ -f ~/.nix-profile/etc/profile.d/nix.sh ] && . ~/.nix-profile/etc/profile.d/nix.sh

I have a new error which seems promising.
# tony at lenovo in ~ [23:17:41]
$ ssh tony@dagobah nix-env --help
error: command ‘man nix-env’ failed: No such file or directory

--
Trying out nix-copy-closure again, I got also another error:
# tony at lenovo in ~ [23:23:56]
$ nix-copy-closure --to tony@dagobah /nix/store/d5vgamnf1hhqd8pz8xp9vnasj4kv7xz8-haskell-xmonad-ghc7.8.4-0.12-shared
error: Nix database directory ‘/nix/var/nix/db’ is not writable: Permission denied
unable to connect to ‘tony@dagobah’; falling back to old closure copying method
error: Nix database directory ‘/nix/var/nix/db’ is not writable: Permission denied
Died at /nix/store/vgs2asxq5ldha99s98jx6d4wldj9wdpv-nix-1.9pre4045_bd91064/lib/perl5/site_perl/5.20.1/x86_64-linux-thread-multi/Nix/CopyClosure.pm line 100.

Thanks for your time guys.

Ok.
I found something that worked.
What I understood from the error I showed you is that the PATH is not rightly set once ssh connected to the dagobah box.
So, I updated my dagobah's ~/.zshenv:
source /etc/profile

[ -f ~/.nix-profile/etc/profile.d/nix.sh ] && . ~/.nix-profile/etc/profile.d/nix.sh

And now, ssh tony@dagobah nix-env --help works (I won't copy paste the too long result).
And also nix-copy-closure command is now working (running right now).
# tony at lenovo in ~ [23:46:18]
$ nix-copy-closure --to tony@dagobah /nix/store/d5vgamnf1hhqd8pz8xp9vnasj4kv7xz8-haskell-xmonad-ghc7.8.4-0.12-shared
copying 13 missing paths (854.87 MiB) to<EMAIL_ADDRESS>exporting path ‘/nix/store/c6qx0qffanfzrygw26k3wh8xn586dn5h-ghc-7.8.4’

I suppose I can close this as this is finally a problem of box setup.
What do you think?
Thanks for the help!

I didn't realize this was a NixOS machine. I foolishly assumed it wasn't because I'd encountered the same problem with non-nixos installs. Are you using program.zsh.enable = true in your configuration.nix?  If yes, this could be a bug in that NixOS module. If no, try it out and you might no longer need ~/.zshenv

Are you using program.zsh.enable = true in your configuration.nix?
If no, try it out and you might no longer need ~/.zshenv

No.
Trying right now!

Are you using program.zsh.enable = true in your configuration.nix?
If no, try it out and you might no longer need ~/.zshenv

No.
Trying right now!

It works!
Here are my steps, on dagobah box:

remove ~/.zshenv
update configuration.nix to add the attribute you mention.
rebuild nixos configuration

On lenovo, trigger nix-copy-closure command.
And it works!
So definitely a problem on my side!
Thank!
Definitely a better approach!


Implement the new Template Schema

Implement the new schema for Templates in the scaffolder.
Allowing for step definitions inside the template schema rather than in code in the scaffolder.
TBD.
#2771

Fixed by #4718


libkv: update godeps, fix wrong error check on zookeeper List

This should fix the transient errors we were seeing on zookeeper tests.
/cc @dongluochen @nishanttotla
Signed-off-by: Alexandre Beslic<EMAIL_ADDRESS>

:green_apple:

LGTM

LGTM


Wrong reference of amount of used ports

In Chapter: https://learn.microsoft.com/en-us/microsoftteams/teams-security-guide#udp-3478-3481-and-tcp-443
The sentence: "A client uses these two ports to allocate UDP and TCP ports respectively to enable these media flows."
Should be changed to: "A client uses these ports to allocate UDP and TCP ports respectively to enable these media flows."
Since there are more than two (2) ports involved.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 33613b2d-d607-ec47-1254-ae540d07ad59
Version Independent ID: aba728f0-0d3c-73fd-c7fe-68e69b6a753f
Content: Security guide for Microsoft Teams overview - Microsoft Teams
Content Source: Teams/teams-security-guide.md
Service: msteams
GitHub Login: @MSFTTracyP
Microsoft Alias: tracyp

To make it easier for you to submit feedback on articles on learn.microsoft.com, we're transitioning our feedback system from GitHub Issues to a new experience.
As part of the transition, this GitHub Issue will be moved to a private repository. We're moving Issues to another repository so we can continue working on Issues that were open at the time of the transition. When this Issue is moved, you'll no longer be able to access it.
If you want to provide additional information before this Issue is moved, please update this Issue before December 15th, 2023.
With the new experience, you no longer need to sign in to GitHub to enter and submit your feedback. Instead, you can choose directly on each article's page whether the article was helpful. Then you can then choose one or more reasons for your feedback and optionally provide additional context before you select Submit.
Here's what the new experience looks like.
Note: The new experience is being rolled out across learn.microsoft.com in phases. If you don't see the new experience on an article, please check back later.
First, select whether the article was helpful:

Then, choose at least one reason for your feedback and optionally provide additional details about your feedback:

Article was helpful
Article was unhelpful

Finally, select Submit and you're done!


Deepmedic Throws Error On ppworker.py at training

Hi ,
I have been trying to work with the deepmedic package on HPC using gpu. The example with the full network runs absolutely fine. However, now I am trying our set of TCIA GBM and LGG cases with the same algorithm i.e. that is replacing the training and validation .cfg files with our cases. I think done this correctly, but the training phase throws an error at ppworker.py showing a problem with the index size. I'm copying the error below for your reference. I would appreciate your help on this.
-Ayush Karnawat
=============================================================
============== Training the CNN model =================
~~~~~~~~~~~~~~~~~~~~Starting new Epoch! Epoch #0/35 ~~~~~~~~~~~~~~~~~~~~~~~~~

************* Starting new Subepoch: #0/20 *************

:=:=:=:=:=:=:=:=: Starting to extract Segments from the images for next Validation... :=:=:=:=:=:=:=:=:
Out of the 4 subjects given for Validation, it was specified to extract Segments from 4 per subepoch.
Shuffled indices of subjects that were randomly chosen: [1, 0, 2, 3]
SAMPLING: Starting iterations to extract Segments from each image for next Validation...
SAMPLING: Going to load the images and extract segments from the subject (iteration) #0/4
Loading subject with 1st channel at:/home/axk840/deepmedic/examples/dataForExamples/patientTestingData/validation/TCGA-06-0125/MNISpace/FLAIR_BiasCorrect_Normalized_Reg_MNI_MeanDivStd.nii.gz
DEBUG: Original index of this case in list of subjects: 1
Images for subject loaded.
From current subject #0 extracting that many Segments centered on : Foreground=0 Background:1250
WARN: No mask was provided for the foreground (Ground Truth Labels). In this case, Background-Centered segments are actually extracted from the whole ROI in a UNIFORM manner (thus extracting Foreground-Centered too). This is a hacked implementation, which will be re-written cleaner soon.
End of the function that extracts segments. Segments extracted centered on: Foreground=0 Background=1250
SAMPLING: Going to load the images and extract segments from the subject (iteration) #1/4
Loading subject with 1st channel at:/home/axk840/deepmedic/examples/dataForExamples/patientTestingData/validation/TCGA-02-0054/MNISpace/FLAIR_BiasCorrect_Normalized_Reg_MNI_MeanDivStd.nii.gz
DEBUG: Original index of this case in list of subjects: 0
Images for subject loaded.
From current subject #1 extracting that many Segments centered on : Foreground=0 Background:1250WARN: No mask was provided for the foreground (Ground Truth Labels). In this case, Background-Center
ed segments are actually extracted from the whole ROI in a UNIFORM manner (thus extracting Foregroun
d-Centered too). This is a hacked implementation, which will be re-written cleaner soon.
End of the function that extracts segments. Segments extracted centered on: Foreground=0 Background=
1250
SAMPLING: Going to load the images and extract segments from the subject (iteration) #2/4
Loading subject with 1st channel at:/home/axk840/deepmedic/examples/dataForExamples/patientTestingData/validation/TCGA-FG-5964/MNISpace/FLAIR_BiasCorrect_Normalized_Reg_MNI_MeanDivStd.nii.gz
DEBUG: Original index of this case in list of subjects: 2
Images for subject loaded.
From current subject #2 extracting that many Segments centered on : Foreground=0 Background:1250
WARN: No mask was provided for the foreground (Ground Truth Labels). In this case, Background-Centered segments are actually extracted from the whole ROI in a UNIFORM manner (thus extracting Foreground-Centered too). This is a hacked implementation, which will be re-written cleaner soon.
End of the function that extracts segments. Segments extracted centered on: Foreground=0 Background=1250
SAMPLING: Going to load the images and extract segments from the subject (iteration) #3/4
Loading subject with 1st channel at:/home/axk840/deepmedic/examples/dataForExamples/patientTestingData/validation/TCGA-FG-6692/MNISpace/FLAIR_BiasCorrect_Normalized_Reg_MNI_MeanDivStd.nii.gz
DEBUG: Original index of this case in list of subjects: 3
Images for subject loaded.
From current subject #3 extracting that many Segments centered on : Foreground=0 Background:1250
WARN: No mask was provided for the foreground (Ground Truth Labels). In this case, Background-Centered segments are actually extracted from the whole ROI in a UNIFORM manner (thus extracting Foreground-Centered too). This is a hacked implementation, which will be re-written cleaner soon.
End of the function that extracts segments. Segments extracted centered on: Foreground=0 Background=1250
TIMING: Extracting all the Segments for next Validation took time: 6.15(s)
:=:=:=:=:=:=:=:=: Finished extracting Segments from the images for next Validation. :=:=:=:=:=:=:=:=:
Loading Validation data for subepoch #0 on shared variable...
DEBUG: For Validation, loading to shared variable that many Segments: 5000
TIMING: Loading sharedVariables for Validation in epoch|subepoch=0|0 took time: 0.19(s)
PARALLEL: Before Validation in subepoch #0, the parallel job for extracting Segments for the next Training is submitted.
-V-V-V-V-V- Now Validating for this subepoch before commencing the training iterations... -V-V-V-V-V-
Validated on 0/104 of the batches for this subepoch...
Validated on 20/104 of the batches for this subepoch...
Validated on 40/104 of the batches for this subepoch...
Validated on 60/104 of the batches for this subepoch...
Validated on 80/104 of the batches for this subepoch...
Validated on 100/104 of the batches for this subepoch...
++++++++++++++++ Reporting Accuracy over whole subepoch for (whole-foreground)Class-0 ++++++++++++++++
VALIDATION: Epoch #0, Subepoch #0, (whole-foreground)Class-0: mean error: 0.996995
VALIDATION: Epoch #0, Subepoch #0, (whole-foreground)Class-0: mean accuracy 0.00300481
VALIDATION: Epoch #0, Subepoch #0, (whole-foreground)Class-0: mean accuracy for Positive Samples(voxels): 1.0. True Predicted Pos=37/37
VALIDATION: Epoch #0, Subepoch #0, (whole-foreground)Class-0: mean accuracy for Negative Samples(voxels): 0.00282543. True Predicted Neg=14/4955
VALIDATION: Epoch #0, Subepoch #0, (whole-foreground)Class-0: mean Dice: 0.0147557
VALIDATION: Epoch #0, Subepoch #0, Class-1: mean error: 0.996995
VALIDATION: Epoch #0, Subepoch #0, Class-1: mean accuracy 0.00300481
VALIDATION: Epoch #0, Subepoch #0, Class-1: mean accuracy for Positive Samples(voxels): 0.0. True Predicted Pos=0/14
VALIDATION: Epoch #0, Subepoch #0, Class-1: mean accuracy for Negative Samples(voxels): 1.0. True Predicted Neg=4978/4978
VALIDATION: Epoch #0, Subepoch #0, Class-1: mean Dice: 0.0
++++++++++++++++ Reporting Accuracy over whole subepoch for Class-2 ++++++++++++++++
VALIDATION: Epoch #0, Subepoch #0, Class-2: mean error: 0.996995
VALIDATION: Epoch #0, Subepoch #0, Class-2: mean accuracy 0.00300481
VALIDATION: Epoch #0, Subepoch #0, Class-2: mean accuracy for Positive Samples(voxels): 0.0. True Predicted Pos=0/8
VALIDATION: Epoch #0, Subepoch #0, Class-2: mean accuracy for Negative Samples(voxels): 1.0. True Predicted Neg=4984/4984
VALIDATION: Epoch #0, Subepoch #0, Class-2: mean Dice: 0.0
++++++++++++++++ Reporting Accuracy over whole subepoch for Class-3 ++++++++++++++++
VALIDATION: Epoch #0, Subepoch #0, Class-3: mean error: 0.996995
VALIDATION: Epoch #0, Subepoch #0, Class-3: mean accuracy 0.00300481
VALIDATION: Epoch #0, Subepoch #0, Class-3: mean accuracy for Positive Samples(voxels): 1.0. True Predicted Pos=1/1
VALIDATION: Epoch #0, Subepoch #0, Class-3: mean accuracy for Negative Samples(voxels): 0.00280505. True Predicted Neg=14/4991
VALIDATION: Epoch #0, Subepoch #0, Class-3: mean Dice: 0.000401687
++++++++++++++++ Reporting Accuracy over whole subepoch for Class-4 ++++++++++++++++
VALIDATION: Epoch #0, Subepoch #0, Class-4: mean error: 0.996995
VALIDATION: Epoch #0, Subepoch #0, Class-4: mean accuracy 0.00300481
VALIDATION: Epoch #0, Subepoch #0, Class-4: mean accuracy for Positive Samples(voxels): 0.0. True Predicted Pos=0/14
VALIDATION: Epoch #0, Subepoch #0, Class-4: mean accuracy for Negative Samples(voxels): 1.0. True Predicted Neg=4978/4978
VALIDATION: Epoch #0, Subepoch #0, Class-4: mean Dice: 0.0
TIMING: Validating on the batches of this subepoch #0 took time: 41.61(s)
UPDATE: In this last epoch (cnnTrained) #0 the CNN achieved a new highest mean validation accuracy of :0.00015024
:=:=:=:=:=:=:=:=: Starting to extract Segments from the images for next Training... :=:=:=:=:=:=:=:=:
Out of the 10 subjects given for Training, it was specified to extract Segments from 50 per subepoch.
Shuffled indices of subjects that were randomly chosen: [7, 8, 3, 0, 4, 9, 2, 1, 6, 5]
SAMPLING: Starting iterations to extract Segments from each image for next Training...
SAMPLING: Going to load the images and extract segments from the subject (iteration) #0/10
Loading subject with 1st channel at:/home/axk840/deepmedic/examples/dataForExamples/patientTestingData/train/TCGA-DU-7309/MNISpace/FLAIR_BiasCorrect_Normalized_Reg_MNI_MeanDivStd.nii.gz
/home/axk840/anaconda2/lib/python2.7/site-packages/pp-1.6.4-py2.7.egg/ppworker.py:78: FutureWarning: comparison to None will result in an elementwise object comparison in the future.
try:
DEBUG: Original index of this case in list of subjects: 7
Images for subject loaded.
From current subject #0 extracting that many Segments centered on : Foreground=50 Background:50
/home/axk840/anaconda2/lib/python2.7/site-packages/pp-1.6.4-py2.7.egg/ppworker.py:32: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
import sys
/home/axk840/anaconda2/lib/python2.7/site-packages/pp-1.6.4-py2.7.egg/ppworker.py:33: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
import os
/home/axk840/anaconda2/lib/python2.7/site-packages/pp-1.6.4-py2.7.egg/ppworker.py:39: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
version = "1.6.4"
/home/axk840/anaconda2/lib/python2.7/site-packages/pp-1.6.4-py2.7.egg/ppworker.py:40: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
/home/axk840/anaconda2/lib/python2.7/site-packages/pp-1.6.4-py2.7.egg/ppworker.py:43: VisibleDeprecationWarning: boolean index did not match indexed array along dimension 0; dimension is 528 but corresponding boolean dimension is 209
fname, fsources, imports = pickle.loads(msg)
An error has occured during the function execution
Traceback (most recent call last):
File "/home/axk840/anaconda2/lib/python2.7/site-packages/pp-1.6.4-py2.7.egg/ppworker.py", line 90, in run
__result = __f(*__args)
File "", line 150, in getTheArraysOfImageChannelsAndLesionsToLoadToGpuForSubepoch
File "", line 43, in get_positive_and_negative_imageParts
IndexError: index 130 is out of bounds for axis 2 with size 96
Traceback (most recent call last):
File "/home/axk840/deepmedic/deepMedicRun", line 102, in
deepMedicTrain.deepMedicTrainMain(absPathToTrainConf, absPathToCnnModelGivenInCmdLine, cnnInstanceLoaded, filenameAndPathWhereNewlyCreatedModelWasSaved)
File "/home/axk840/deepmedic/deepmedic/frontEndModules/deepMedicTrain.py", line 440, in deepMedicTrainMain
do_training(*trainSessionParameters.getTupleForCnnTraining())
File "/home/axk840/deepmedic/deepmedic/trainValidateTestVisualiseParallel.py", line 1434, in do_training
subsampledImagePartsChannelsToLoadOnGpuForSubepochTraining] = parallelJobToGetDataForNextTraining() #fromParallelProcessing that had started from last loop when it was submitted.
TypeError: 'NoneType' object is not iterable

Hello Ayush and thanks a lot for the feedback and interest in our work. Hope you find it useful.
About the problem:
The ppworker is a parallel process that is handling the extraction of image-patches/segments for the training and validation. If anything goes wrong during this process, it will be breaking and complaining.

I think the explanation about the problem lies in the line: "IndexError: index 130 is out of bounds for axis 2 with size 96".
I believe that for this subject: /home/axk840/deepmedic/examples/dataForExamples/patientTestingData/train/TCGA-DU-7309/MNISpace/FLAIR_BiasCorrect_Normalized_Reg_MNI_MeanDivStd.nii.gz
the input images are not all of the same dimensions. Check that the actual images (MRI/CT/whatever), the ground-truth labels and the rois are all of the same dimensions (probably one of them is at axis 2 of size 96, and all others are bigger?). The software needs all of them to be of the same size for a certain subject (not required across all subjects. Per-subject basis only).

Please check if this was the problem and let me know.
ps: This warning "FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison" is ugly and my bad. I will open an issue for this.
Thanks again for your interest and feedback. Let me know how it goes.
Kind regards,
Konstantinos

Hi Kamnitsask,
Sorry for getting back to you so late. The fix you described worked perfectly as the brain mask was not of the same dimensions as the Flair,T1,GT, and ROI files for a per-subject basis. After modifying the brain masks to be the same dimension, the training and testing models ran successfully. Thank you once again for your help.
-Ayush

Wonderful to hear your problem is solved Ayush. Hope you get some helpful results for your work from it.
Sorry for the fuss it might have caused you. I know that I should have placed better checks and error messages with feedback, but there are quite some things to check for and this would be quite some work, which is lower in my priorities. Quite some things to do before this.
Please feel free to let me know how your experience with it goes, or ask anything further at<EMAIL_ADDRESS>Kind regards,
Kostas


Google feedback on TypeScript 4.4-beta

Feedback to the TypeScript team for TypeScript 4.4-beta
This GitHub issue contains feedback on the TS 4.4-beta release from the team
that is responsible for keeping Google's internal software working with the
latest version of TypeScript.
During the 4.4-beta migration, we have ran into some issues. Especially some
changes made in lib.dom.d.ts will make it diffilcut for Google to migrate. We've
listed the breaking changes which we weren't able to easily migrate in the
following section.
Breakages which are complicated to migrate
Each subsection in this section contains a diagnosis of the breakage and a link
to the github issue if available.
canvas API changes in WebGL
WebGL2RenderingContext no longer seems to have a .canvas field, however the
WebGLRenderingContext does have one. TS now doesn't list one on either API.
There seems to be a github issue
related to the failure, and it seems it has been fixed already.
Readonly attributes in DOMPoint, DOMMatrix, DOMRect
Some properties of DOMPoint, DOMMatrix, DOMRect became readonly, and this no
longer match the specification. This is because these types no longer override
properties from the extended types to be read/write.
Compare(DOMPoint):

https://github.com/microsoft/TypeScript/blob/release-4.3/lib/lib.dom.d.ts#L3979
https://github.com/microsoft/TypeScript/blob/release-4.4/lib/lib.dom.d.ts#L3886

Compare(DOMMatrix):

https://github.com/microsoft/TypeScript/blob/release-4.3/lib/lib.dom.d.ts#L3854
https://github.com/microsoft/TypeScript/blob/release-4.4/lib/lib.dom.d.ts#L3783

Compare(DOMRect):

https://github.com/microsoft/TypeScript/blob/release-4.3/lib/lib.dom.d.ts#L4026
https://github.com/microsoft/TypeScript/blob/release-4.4/lib/lib.dom.d.ts#L3929

Removed Types
SpeechRecognition, RTCError got removed from lib.dom.d.ts. There are existing
code which use this API, and it would be difficult to migrate without proper
type support either from the compiler or from definitelyTyped.

https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognitionEvent
https://developer.mozilla.org/en-US/docs/Web/API/RTCError

Offscreen Rendering API changes
OffscreenCanvas still exists, but it's marked as deprecated. Though it is no
longer a value, i.e. code calling new OffscreenCanvas no longer works, it's
just an interface. We think this might be unintentional. Though, we would still
be able to migrate via referring to the types defined in
DefinitelyTyped.

https://github.com/microsoft/TypeScript/blob/release-4.3/lib/lib.dom.d.ts#L11087
https://github.com/microsoft/TypeScript/blob/release-4.4/lib/lib.dom.d.ts#L10832

ShadowRoot interface changes
ShadowRoot is a subtype of DocumentFragment. DocumentFragment has a
getElementById which returns HTMLElement. In 4.4-beta, DocumentFragment no
longer defines its own getElementById method, meaning
HTMLElement.prototype.shadowRoot.getElementById returns Element, not
HTMLElement. This causes hundreds of breakages of code using shadowRoot.

https://github.com/microsoft/TypeScript/blob/release-4.3/lib/lib.dom.d.ts#L4985
https://github.com/microsoft/TypeScript/blob/release-4.4/lib/lib.dom.d.ts#L4717

ChildNode interface changes
In 4.4, ChildNode no longer extends Node. This seems like a bug.

https://github.com/microsoft/TypeScript/blob/release-4.3/lib/lib.dom.d.ts#L3531
https://github.com/microsoft/TypeScript/blob/release-4.4/lib/lib.dom.d.ts#L3487

DocumentOrShadowRoot interface changes
DocumentOrShadowRoot no longer has getSelection, elementFromPoint,
elementsFromPoint, caretFromRange. We also couldn't find in the repo whether
this was intentional or not.

https://github.com/microsoft/TypeScript/blob/release-4.3/lib/lib.dom.d.ts#L4995
https://github.com/microsoft/TypeScript/blob/release-4.4/lib/lib.dom.d.ts#L4725

Impact summary (trivial fix unavailable)

Change description
Libraries affected

type of ariaLabel is string rather than string/null
0.079%

ShadowRoot interface changes
0.034%

OffscreenCanvas Rendering API changes
0.014%

Removed Types
0.009%

Newly readonly attributes
0.008%

DocumentOrShadowRoot interface changes
0.005%

ChildNode interface changes
0.004%

canvas API changes
0.004%

Impact summary (trivial fix available)

Change description
Libraries affected

window.parent nullability changes
0.088%

misc and other changes
0.088%

Various Media Related API changes
0.019%

window.location.reload interface changes
0.014%

Navigator API changes
0.012%

Promise always evaluate to true or false
0.011%

CustomElementConstructor interface changes
0.010%

Property has no initializer
0.008%

Infinite depth type inference
0.002%

property specified more than once in object literal
0.002%

document.createTreeWalker interface changes
0.001%

@h-joo thank you very much for this summary! It sounds like Google didn't run into any issues with new strictness flags being introduced. Do you all run with --strict on, or with a manually-specified set of strictness flags?

Do you all run with --strict on, or with a manually-specified set of strictness flags?
Thank you for actually mentioning this. We've been doing the latter. That'd be the reason that we had no issues with the strictness flags. We will discuss this internally whether we want to turn the new flags on 😄

we use getSelection from shadowRoot since long time and now we get this error:


build failure following 372674fca3250069d8f4943b572474ce077849fd

Describe the bug
Vim fails to build from source following 372674fca3250069d8f4943b572474ce077849fd
To Reproduce
./configure --prefix=$HOME/usr/vim --disable-darwin --disable-selinux --enable-perlinterp --enable-pythoninterp --enable-luainterp --disable-netbeans --with-lua-prefix=/usr --with-luajit
make -j8

Gives the error:
gui.c: In function ‘gui_init’:
gui.c:687:9: warning: implicit declaration of function ‘gui_mch_maximized’; did you mean ‘gui_mch_mousehide’? [-Wimplicit-function-declaration]
     if (gui_mch_maximized())
         ^~~~~~~~~~~~~~~~~
         gui_mch_mousehide

Expected behavior
A successful build.
Screenshots
n/a
Environment (please complete the following information):

Vim version v8.1.1092
OS: Ubuntu 18.04.2 LTS, amd64
Terminal: XTerm(330)

Additional context
Please let me know if you need any more details about my setup.

I suppose you are not building with GTK.  Motif or Athena perhaps?

Thanks for the fix. I was only looking to build a non-GUI, terminal version. So I think my answer to the above is none of them?

Thanks for the fix. I was only looking to build a non-GUI, terminal
version. So I think my answer to the above is none of them?

The error is in gui.c, thus you must have picked up some GUI library.
Configure finds it automatically.
--
Did you ever see a "Hit any key to continue" message in a music piece?
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

Configure finds it automatically.

Ah, ok thanks. I've actually switched to explicitly disabling the GUI now.


about view desktop of connected users

This wiki shows us how to view desktop of connected user (Xvnc on?). But it's not well documented. Another solution, one could use pgrep -a vnc to see all  vnc related processes. We could get something like this:
34711 Xvnc :11 -auth .Xauthority -geometry 1920x1052 -depth 32 -rfbauth /home/USER//.vnc/sesman_passwd-USER@lab-linux-06:11 -bs -nolisten tcp -localhost -dpi 96

OK, xrdp or xrdp-sesman start Xvnc on port :11 in localhost (to the remote machine).
Not sure how to check xorgxrdp connection, maybe journalctl -u xrdp-sesman could tell us something.

The wiki page is not created by we xrdp developers. We do not gurarantee if it works.


Add gitignore file for bazel generated files

Bazel is Google's new build system for compiled
languages. At present there is no documentation on what files are
generated that need to be ignored but these are the ones that I've
noticed. A link to the documentation can be found
here. Bazel is a really
awesome tool and it thankfully does not generate my extra files. It is
also incredibly easy to get started with so I imagine that it will
become pretty popular in the near future.

If this is a tool rather than a language, should it live under the Global namespace?

Bazel is a build system, like Maven, Gradle and CMake. Those three all live in the root (not Global). If a project uses Bazel (for example https://github.com/mbrukman/autogen/), then all users of that project will need the gitignore in the project, not installed globally. So I think as is this patch is good.

@durandj apologies for the delay in replying to this.
I had a lot of these templates requested, but I couldn't quite figure out the right place for them. I ended up settling on #2857, and the Bazel template now lives at community/Bazel.gitignore.
Feel free to add any further changes there, and thanks again for the contribution!


the function return pointer in cpp

Language: cpp
AerialInfo:
Filetype: cpp
Configured backends:
lsp (not supported)
treesitter (supported) (attached)
markdown (not supported)
Show symbols: Class, Constructor, Enum, Function, Interface, Method, Struct
The minimal amount of code that demonstrates the missing symbol:
int* bar() {}

Expected symbol: bar

Thanks for the report!


test: add duplicate symbol test

On OSX symbols are exported by default, and they overlap if two
different copies of the same symbol appear in a process. This change
ensures that, on OSX, symbols are hidden by default.
Re: https://github.com/nodejs/node-addon-api/pull/456

Checklist

[x] npm install && npm test passes
[x] tests are included
[x] commit message follows commit guidelines

Description of change

@richardlau addons are not built with -fvisibility=hidden because originally it was impossible to do so, because that would also hide the node::nm_module data symbol that early versions of Node.js were searching for when loading addons. However, it has been possible to load native modules via DSO constructors since 76b98462e589a69d9fd48ccb9fb5f6e96b539715 (released in v0.11.11), meaning that it would have been possible to introduce -fvisibility=hidden all this time. In fact, in bd8a5755dceda415eee147bc06574f5a30abb0d0 (released in v0.11.12) @bnoordhuis presciently defined NODE_MODULE_EXPORT as __attribute__((visibility("default"))) instead of the empty value it had had up to that point in anticipation of the ability to actually make use of it. Native addons explicitly wishing to expose symbols should prefix them with NODE_MODULE_EXPORT.
It seems that the involuntary exposure of all the symbols has so far caused few problems. Nevertheless, the test in this PR and the issue this PR references indicate that there are problems with unwittingly exposing symbols on OSX.
The problem in the wild can arise from having a large application that has two dependencies, each of which depends on (perhaps a different version of) a native addon. When this addon is loaded from two different places in the dependency tree into the same process, and if this addon exposes symbols using this static inline storage class, then the two different symbols will not be distinguishable on OSX. I have at least one case where that causes a segfault.
I guess the point of discussion should be whether we should takes this opportunity to start building addons with -fvisibility=hidden, at least on OSX, or whether we should take the advocacy approach and evangelize the addition of this flag to each addon's binding.gyp.
Can we test this change with CITGM?

@mcollina @kkoopa @addaleax do you think it would be feasible to add -fvisibility=hidden to addon build flags on OSX given the error I found because the flag is absent?

@bnoordhuis thanks for the review! I have addressed the nits.

Rebased to fix conflicts.

I'm wondering if setting this by default could be breaking for any existing addons?

Can we test this change with CITGM?

I think we'll need to tweak the CITGM CI script, but @gabrielschulhof you can run CITGM locally on mac. It's as simple as:
npm i -g citgm
citgm-all

But you'll need to transplant a fixed version of node-gyp into npm's node_modules...

@refack the Mac machine I have access to is behind a proxy server, so I'm seeing some failures that might be related to that. I'll request access to a CI mac. Let's see if I can run the citgm there.

@refack I tried running citgm-all on the Mac to which I received access, but, unfortunately, it fails in lots of places even without replacing node-gyp. Thus, I have no firm basis against which to perform such a test.

Thus, I have no firm basis against which to perform such a test.
this is the citgm problem, there's no concrete baseline except in the heads of people that use it regularly.

I'm fine with getting this merged but could someone come up with a semver-* label for it? I don't really know. Seems potentially semer-major to me, but maybe not?

@rvagg for this to be semver-major there would have to be addons which, upon loading, assume the existence of symbols provided by other addons, symbols which are not explicitly marked NODE_MODULE_EXPORT.

I changed the commit msg to have a "build,test" prefix and landed in 8b7fdd1. I'm still having trouble with the semveriness of this. semver-minor at least? I don't think I'll include this in 5.0.1 but maybe it should be in a 5.1.0?

@gabrielschulhof would you mind revisiting this please? I've just noticed the tests fail with Node 12 on macOS, the addon is using a bare GetFunction().

This works:
diff --git a/test/node_modules/duplicate_symbols/binding.cc b/test/node_modules/duplicate_symbols/binding.cc
index bc5f6f5..a0999b7 100644
--- a/test/node_modules/duplicate_symbols/binding.cc
+++ b/test/node_modules/duplicate_symbols/binding.cc
@@ -2,9 +2,9 @@
 #include "common.h"

 void Init(v8::Local<v8::Object> exports) {
-  exports->Set(Nan::New("pointerCheck").ToLocalChecked(),
-               Nan::New<v8::FunctionTemplate>(Something::PointerCheck)
-      ->GetFunction());
+  Nan::Set(exports, Nan::New("pointerCheck").ToLocalChecked(),
+    Nan::GetFunction(
+      Nan::New<v8::FunctionTemplate>(Something::PointerCheck)).ToLocalChecked());
 }

 NODE_MODULE(NODE_GYP_MODULE_NAME, Init)

since this is already merged, maybe push a new commit here and I'll merge that on top of master.

oh, and update the "nan" version in package.json when you're in there too I think.

@rvagg I made the changes you asked for. I'm not sure if the change I made to the version of nan requested in the test's package.json is the one you had in mind. After all, the existing ^2.0.0 should result in the latest version of nan getting pulled, right?

great, thanks. and yes you're right about the ^2, my local version was old. landed in af0d7b8


Postman: Error inserting user into database

Postman: Error inserting user into database
POST

Endpoint: http://localhost:3000/api/register
Body:{
  "username": "test1",
  "password": "testPwd1",
  "role": "buyer"
}

Error log:
npm.cmd run start

><EMAIL_ADDRESS>start
> npx nodemon server.js

[nodemon] 3.0.1
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,cjs,json
[nodemon] starting `node server.js`
Server is listening on 3000
You have tried to call .then(), .catch(), or invoked await on the result of query that is not a promise, which is a programming error. Try calling con.promise().query(), or require('mysql2/promise') instead of 'mysql2' for a promise-compatible version of the query interface. To learn how to use async/await or Promises check out documentation at https://www.npmjs.com/package/mysql2#using-promise-wrapper, or the mysql2 documentation at https://github.com/sidorares/node-mysql2/tree/master/documentation/en/Promise-Wrapper.md
Error: You have tried to call .then(), .catch(), or invoked await on the result of query that is not a promise, which is a programming error. Try calling con.promise().query(), or require('mysql2/promise') instead of 'mysql2' for a promise-compatible version of the query interface. To learn how to use async/await or Promises check out documentation at https://www.npmjs.com/package/mysql2#using-promise-wrapper, or the mysql2 documentation at https://github.com/sidorares/node-mysql2/tree/master/documentation/en/Promise-Wrapper.md
    at Query.then (C:\Users\vomin\Projects\rmit\rmit-isys2099-group9-app\server\node_modules\mysql2\lib\commands\query.js:43:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
error: Error: You have tried to call .then(), .catch(), or invoked await on the result of query that is not a promise, which is a programming error. Try calling con.promise().query(), or require('mysql2/promise') instead of 'mysql
2' for a promise-compatible version of the query interface. To learn how to use async/await or Promises check out documentation at https://www.npmjs.com/package/mysql2#using-promise-wrapper, or the mysql2 documentation at https://github.com/sidorares/node-mysql2/tree/master/documentation/en/Promise-Wrapper.md
    at Query.then (C:\Users\vomin\Projects\rmit\rmit-isys2099-group9-app\server\node_modules\mysql2\lib\commands\query.js:43:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Resolved by @LaansDole


Let users see clip thumbnail same as regular thumbnail (#386)

Issue https://github.com/whyboris/Video-Hub-App/issues/386 - Let users see clip thumbnail same as regular thumbnail

Added new toggle under View Settings -> Clips View
Setting only appears in settings menu, not in ribbon
If enabled,  it uses the same thumbnail as the other views
Works with Folder view or Single Clip
If enabled, it also respects the returnToFirstScreenshot setting

Thank you for this PR 🤝
Code seems great -- I'll test the changes today and will merge before releasing 2.2.3 🎉

And thank you for the angular-specific (mouseout) fix 😉 (replacing the onmouseout event)


Inspect docker container memory

No need to update docker configuration as github actions already provides 7G of RAM memory (link)

However, one can experience timeout issues with E2E tests if the Docker memory limit is too low.
Just increase the docker memory limit to something bigger than 2g on your machine


x86 Early Boot Fixes

Various fixes and cleanups discovered while dealing with early boot on PC hardware.

I'm seeing something weird when I test this.
If I build and run a test locally (any test, doesn't seem to matter) I get a hung console:
SeaBIOS (version rel-1.12.1-0-ga5cab58-dirty-20200516_152813-e7ee184eb100-zephyr
)
Booting from ROM..

Reminds me of this, might be related:
https://github.com/zephyrproject-rtos/sdk-ng/issues/168
https://github.com/zephyrproject-rtos/zephyr/pull/21733

Sanitycheck issue addressed by https://github.com/zephyrproject-rtos/zephyr/pull/26223, I used this patch to help test it.

Yeah, that disassembly is showing a port of "5", which is the register offset.  I clearly broke the ioport generator, will fix.  This was sort of a rush job to get it into a PR-worthy state and almost all my testing was on MMIO hardware.

Rebase, address comments, fix one goof that had broken irq_connect_dynamic() (which apparently we only enable for one test?)

Close/open again to kick CI.  Seems like the buildkit error is stale?  It's reflecting a failure the previous push corrected.

Just for clarity:

We should get this from DTS:

Under absolutely no circumstances.  I'll go all the way to the mattresses on this.  If you want all x86 console drivers to use DTS then submit a patch to remove this one.

Under absolutely no circumstances. I'll go all the way to the mattresses on this. If you want all x86 console drivers to use DTS then submit a patch to remove this one.

the driver was already using DTS and you are changing it to use a value that is hardcoded in the drive, what is the rationale behind all of this?
diff --git a/arch/x86/core/early_serial.c b/arch/x86/core/early_serial.c
index c4d9cc4f29..58dcebef5b 100644
--- a/arch/x86/core/early_serial.c
+++ b/arch/x86/core/early_serial.c
@@ -4,19 +4,19 @@
  * SPDX-License-Identifier: Apache-2.0
  */

-#include <devicetree.h>
 #include <kernel.h>
 #include <sys/util.h>
+#include <drivers/pcie/pcie.h>

 #include <soc.h>

-#if DT_PROP(DT_CHOSEN(zephyr_console), pcie)
-BUILD_ASSERT(IS_ENABLED(CONFIG_PCIE), "NS16550(s) in DT need CONFIG_PCIE");
-#define UART_NS16550_PCIE_ENABLED
-#include <drivers/pcie/pcie.h>
-
-#define UART_PCIE_BDF  (DT_REG_ADDR(DT_CHOSEN(zephyr_console)))
-#define UART_PCIE_ID   (DT_REG_SIZE(DT_CHOSEN(zephyr_console)))
+#if defined(X86_SOC_EARLY_SERIAL_PCIDEV)
+#define UART_PCIE_BDF X86_SOC_EARLY_SERIAL_PCIDEV
+#define UART_NS16550_PCIE_ENABLED 1
+#elif defined(UART_NS16550_ACCESS_IOPORT)
+#undef UART_NS16550_PCIE_ENABLED
+#else
+#error "Incomplete x86 SoC early serial config"
 #endif

@nashif Please don't start a fight on this.  The reasoning is called out explicitly in the commit message, and I explained it above again.
I won't make that change to this PR, it's just technically wrong.  This driver doesn't have value if it's coupled to devicetree.  If that's your position and you won't budge, then just remove it.  We have another console driver.  We only need this one if we need to bring up more boards that defeat analysis.

Could we try to get this moving forward? @andyross there seems to be conflicts with arch/x86/core/intel64/locore.S - possibly due to the zefi PR getting merged?

Oh right, I actually uncollided these to submit them separately and one of the PRs was always going to need a rebase.  Will fix ASAP.  Note that there's a rebased variant of these patches in my ehl-wip branch if you need it sooner.

New version includes an extra patch to zefi to workaround a fundamental race with HPET.  Please re-review as needed. Existing comments should be addressed.


Add saga pattern support in Dapr workflow

Related issue is https://github.com/dapr/proposals/issues/48

While this proposal sounds useful I would like to see the core implementation / common interface, API methods outlined in pseudocode.
It's very hard to interpret Java code and think through how this would work in Python for example.
Could you update the proposal to identify the generic structure that all SDKs must implement?
In this proposal we can then discuss whether this will make sense. After well we want to have a unified implementation across all SDKs, so we will want to agree on the same implementation strategy in this proposal.

While this proposal sounds useful I would like to see the core implementation / common interface, API methods outlined in pseudocode.
It's very hard to interpret Java code and think through how this would work in Python for example.
Could you update the proposal to identify the generic structure that all SDKs must implement?
In this proposal we can then discuss whether this will make sense. After well we want to have a unified implementation across all SDKs, so we will want to agree on the same implementation strategy in this proposal.

I agree. I think the implementation detail will evolve as the Java implementation serves as a reference. I am not blocking the Java implementation waiting for this proposal to be merged but knowing the implementation can change (ok for version 0.x of the workflow SDK).

While this proposal sounds useful I would like to see the core implementation / common interface, API methods outlined in pseudocode.
It's very hard to interpret Java code and think through how this would work in Python for example.
Could you update the proposal to identify the generic structure that all SDKs must implement?
In this proposal we can then discuss whether this will make sense. After well we want to have a unified implementation across all SDKs, so we will want to agree on the same implementation strategy in this proposal.

Good suggestion, I will update it soon.

I agree with both @halspang's and @kaibocai's comments.
Providing an interface which can be implemented by an activity for adding the compensation feature, would be great.
Also moving the logic of calling compensation function into the workflow runtime will make the business logic code more clear.
Should we add another function like compensateOn(Predicate<T Result> result), where if an exception is not thrown, and the compensation must be triggered due to a null Result or some value in result, the Predicate function can be used?
Additionally, should a hook be provided so that users can look into the error/exception if thrown before the compensation is triggered i.e. onError() is called before compensate() is called?

I would love to see first-class support for Sagas in Workflows across all languages.
FWIW I had the same idea a good few months ago (which was based on the same idea I had of using Sagas for Azure Durable Functions 2 years ago) so it's nice to see that you've arrived in a place along the same line as where I was thinking!

The proposal LGTM overall, thanks @skyao. Agree with the comments here, specifically (1) not having to register a compensation separately, and (2) if we can offload certain things to runtime, to avoid having to duplicate this to all SDKs and maintain them.

I would love to see first-class support for Sagas in Workflows across all languages.

We'll start with java, and hopefully this proposal will be accepted soon so that the implementation of saga pattern in the dapr java sdk can be released in the dapr v1.13 release. Then I plan to add python and .net support in next dapr release v1.14.
Chris has agreed to do some optimization work in workflow for saga support, as you see, "first-class support".
This should all happen soon.

Sharing my thoughts on the feedback received so far:

Main comment is about the registering of the activity and its compensation separately, I'd prefer to not do that if we can avoid it.

@halspang (and others) if I understand the concern correctly, this is about reducing boilerplate. While I sympathize with that, I worry about how this can make the saga pattern overly opinionated and less useful. For example, I don't think there will always be a 1:1 relationship between an activity and its compensation. Furthermore, the exact compensation strategy (which activity to call) and parameters may also need to be different for the same activity executed at different points in the workflow. I would prefer that we start by erring on the side of a loosely coupled design.

In an ideal world I'd love to see the runtime be able to walk back the entire workflow itself

I understand the appeal of this idea but have two major concerns:

This would require non-trivial changes in the workflow runtime and I'm not convinced that the benefit justifies the cost at this point, especially given that this is a new feature that hasn't been developer tested. Though redundant, an SDK implementation will be cheaper to build, maintain, and change in the future based on user feedback.

This could necessarily make the compensation model even more inflexible because the runtime will have to make specific assumptions about when and how to invoke compensation logic.

I'll also point out that the activity registration verbosity problem can be solved in other ways, such as what @mukundansundar has proposed in the Python SDK. In other words, we may be able to treat these concerns separately.
My overarching hope is that the saga implementation can be loosely coupled to the overall SDK logic, especially given that it's a brand-new thing that we want to get more real-world feedback on.

if the user enables Saga, we handle the compensation logic for them in the catch block for the SDK, what if the user has some specific logic when an exception happens, in other words, they need to do some work (mostly logging) before trigger compensation. It seems in the current proposal we don't provide a chance for them to do this. They can do it in their catch block but that is just opposed to the proposed solution.

@kaibocai I agree that the ability to have some custom logic is important, which is partly why I strongly hesitate against tightly coupling an activity's execution to its compensation. In the proposed model, which is loosely coupled, developers can execute custom logic (e.g., logging) by catching exceptions, logging in the catch block, and then rethrowing the exception (or use manual compensation) to trigger the compensation. I don't think this approach is opposed to the proposed solution. I think the auto-compensation done by the SDK should be seen as a convenience feature.

Additionally, should a hook be provided so that users can look into the error/exception if thrown before the compensation is triggered i.e. onError() is called before compensate() is called?

@mukundansundar similar to my previous responses, this isn't necessary with the current proposal because you can control if/when the compensation gets triggered. These hooks that you're suggesting are only required if we tightly couple an activity invocation with its compensation logic, which I'm arguing we shouldn't do because it creates inflexibility (and requires us to complicate the design with behavior customizing hooks, etc.).

Wen through the proposal today. @skyao I think it's a great proposal, that would immensely benefit users.
I too agree with comments regarding keeping registration of compensation separate than activity registration.
@skyao I also wanted to understand a bit more on lines of how compensation will work in different scenarios i.e. An Activity A's compensation may need to be called, if workflow fails at an activity X's level, but Activity A's compensation may NOT be needed to be called if workflow fails at Activity Y's level.
Will there be some rules/ filters that need to be designed accordingly, as a part of infra here OR will user need to explicitly mention in code, depending on language specific error/exception semantics?

@skyao I also wanted to understand a bit more on lines of how compensation will work in different scenarios i.e. An Activity A's compensation may need to be called, if workflow fails at an activity X's level, but Activity A's compensation may NOT be needed to be called if workflow fails at Activity Y's level. Will there be some rules/ filters that need to be designed accordingly, as a part of infra here OR will user need to explicitly mention in code, depending on language specific error/exception semantics?

Currently we have not considered such a complex compensation logic; this judgment of whether to compensate is made across activities, and it requires some global data across activities. Of course, if the activiy x/y failed with output, it can be such a simple judgment:
Object output-x = ctx.callActivity("activity-x");
Object output-y = ctx.callActivity("activity-y");

......
Object output-a = ctx.callActivity("activity-a");
if (!output-x.isOK() && output-y.isOK) {
   ctx.registerCompensation("compensation-b")
}

But if the activiy x/y failed with exceptions, users have to do try/catch to let the workflow continue to execute activity-a when activity-x and activity-y are failed:
boolean isXFailed = true;
boolean isYFailed = true;
try {
    Object output-x = ctx.callActivity("activity-x");
    isXFailed = false;
} catch {...}
try{
    Object output-y = ctx.callActivity("activity-y");
    isYFailed = false;
} catch {...}

......
Object output-a = ctx.callActivity("activity-a");
if (isXFailed && !isYFailed) {
   ctx.registerCompensation("compensation-b")
}

I have to say that the flexibility of current proposal is very high, and users can always combine compensation strategies that meet their requirements.
From this point of view, I agree with Chris' suggestion: flexibility is more important.

While this proposal sounds useful I would like to see the core implementation / common interface, API methods outlined in pseudocode.
It's very hard to interpret Java code and think through how this would work in Python for example.
Could you update the proposal to identify the generic structure that all SDKs must implement?
In this proposal we can then discuss whether this will make sense. After well we want to have a unified implementation across all SDKs, so we will want to agree on the same implementation strategy in this proposal.

I think it can be done without another set of pseudo-code. After the java-sdk implementations are merged, I will contact the maintainers of the python sdk and .net sdk to come together and implement saga mode in the python sdk and .net sdk. I will then help the maintainers define the python and .net APIs and implementations directly and update them to this proposal.
I expect saga support for python-sdk and .net sdk to be available in dapr v1.14, and I promise I'll support it.
Translated with www.DeepL.com/Translator (free version)

I agree with @cgillum to keep the activity and compensation loosely coupled to begin with. We can always add opinionated Facades on top if needed.
+1 binding

@artursouza Can we merge this PR now to add this proposal into main branch.

Would it be helpful to add references to Saga support in other workflow tools?
In Temporal, for example, there is a helper class in Java but in other languages there is only sample code.
https://temporal.io/blog/saga-pattern-made-easy

In Java, the Saga class keeps track of compensations for you ... In other language SDKs you can easily write the addCompensation and compensate functions yourself.

(Followed by links to examples for Go, PHP, Python, TypeScript.)
In Golang there is no try/catch hell and defer is built into the language.
Even in Java, consider whether this should be part of the SDK or whether it should be part of a contrib library or even a DSL built on top of (one of) the Dapr SDKs.
Questions:
In a workflow-as-code solution such as Dapr, why can't different custom solutions and patterns be accomplished "in code" using the features of the programming language and its libraries?
Is Saga such a singularly useful pattern that it should to be built into every Dapr SDK?


Select - keyboard home button click choose the first option

Reproduction
See in Angular Material examples:
https://material.angular.io/components/tooltip/examples
Steps to reproduce:

choose an option from the select (don't lose focus).
press the Home button on the keyboard.

Expected Behavior
Default home button behavior - scroll to top of the page.
Actual Behavior
The first option of the select being selected.
Environment

Angular: 7.2.13.
CDK/Material: 7.3.7.
Browser(s): Chrome.
Operating System (e.g. Windows, macOS, Ubuntu): Ubuntu.

This matches the behavior of the native select element which also selects the first option when you press home. Closing since it works as expected.


chore: release main

:robot: I have created a release beep boop
internal/themes: 0.0.1
0.0.1 (2024-05-11)
⚠ BREAKING CHANGES

rewrite of library to be simpler. This is not something that is usually done but this library is still a bit of work in progress. The major version will not be bumped since there is no-one using it.

Features

catppuccin: bump version (c7a7624)
catppuccin: bump version (1ea6b99)
catppuccin: bump version (ad1a9fb)
catppuccin: bump version (db94821)
release 1.2 (c2ec968)
rewrite of library (36408df)

Bug Fixes

add gallery section (4da21ab)
bump version (ef2d306)
internal/logtheme: bump version (c86f76a)
internal/logtheme: bump version (2a158eb)
internal/logtheme: bump version (7ff9f0e)
new version with license (5809ad9)
require slog (40e9556)
update go.mod (a72a7f1)
update go.mod (8c342d4)
versions (4a2abfb)

Miscellaneous Chores

release 0.0.1 (edcc76f)

pkg/catppuccin: 0.0.1
0.0.1 (2024-05-11)
⚠ BREAKING CHANGES

rewrite of library to be simpler. This is not something that is usually done but this library is still a bit of work in progress. The major version will not be bumped since there is no-one using it.

Features

add Default to catppuccin (f81efbd)
catppuccin (3846146)
catppuccin: bump version (c7a7624)
catppuccin: bump version (1ea6b99)
catppuccin: bump version (ad1a9fb)
catppuccin: bump version (db94821)
release 1.2 (c2ec968)
rewrite of library (36408df)

Bug Fixes

add gallery section (4da21ab)
bump version (ef2d306)
create hex color (388c271)
image for catppuccin (f21e2d9)
image sizes (b1342ae)
internal/logtheme: bump version (2a158eb)
internal/logtheme: bump version (7ff9f0e)
new version with license (5809ad9)
require slog (40e9556)
touch catp (28082bb)
update go.mod (a72a7f1)
update go.mod (8c342d4)
versions (4a2abfb)

Miscellaneous Chores

release 0.0.1 (edcc76f)

This PR was generated with Release Please. See documentation.

:rocket: This description was created by Ellipsis for commit f9b54351cb9c74c9b223454219298492a9e91712

Summary:
This PR updates the internal/themes and pkg/catppuccin modules with new features, bug fixes, and prepares for the release of version 0.0.1.
Key points:

Release update for internal/themes and pkg/catppuccin.
Includes version bumps, feature additions, bug fixes, and miscellaneous chores.
Significant library rewrites marked as breaking changes.
Updates to CHANGELOG.md files for both modules.

Generated with :heart: by ellipsis.dev

:robot: Release is at https://github.com/op/redlog/releases/tag/internal/themes/v0.0.1 :sunflower:

:robot: Release is at https://github.com/op/redlog/releases/tag/pkg/catppuccin/v0.0.1 :sunflower:


Lazy load and loading only on the page that it needed

Hi,
After adding this modul to nuxt.config all Stripe scripts and HTML element were rendered on all pages. I don't even use it. Is it possible to only add it to pages where I use it and with lazy loading?

It all comes down to this:
https://github.com/fuentesloic/nuxt-stripe/blob/main/src/module.ts#L73-L74
But I'm having an hard time trying to trying to fix this.

In #19 I wanted to fix a double call for the useClientStripe, but in doing so I was able to just remove that line, this could potentially fix both issues.

@Sandros94 I don't understand your approach. So you finally fixed it? Cause there's nothing I can do about this problem

So you finally fixed it? Cause there's nothing I can do about this problem

Yes, I did manage to fix it, but it is still under review in that PR. I should update the description to mention this issue too.

I don't understand your approach.

The issue was just that the stripe-js (the one that is responsible to load Stripe script and HTML elements client side) not only it was forcefully loaded as a global plugin, always present and available, but it was double called when used with useClientStripe.
I know its not a good practice (with security concerns), but you could clone the repo I use for the PR and do yarn && yarn prepack && yarn pack to create a local tgz to use test in your project.

Hey hope you doing good I will have a look this weekend of your work :)

Hey I have done a PR that should handle the behavior you are expecting. It will be loaded only the first time you will use it. Check the PR. This will be merge this week. It will create a breaking change as we are going to harmonize the api config

I'll mainly respond in the PRs #19 and #22. But long story short IMO:
#22 currently just delays when the stripe-js client is loaded, but doesn't make it lazy load. Relying on addPlugin it will make it load app/website wide, degrading user performance once it has loaded it regardeless of the page where the user is.
Why open up a new PR when there is already another one that is currently discussed on with the maintainers?


tests: add tests to test suite

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Files
Coverage Δ

crates/archlinux/src/lib.rs
35.10% <ø> (+26.59%)
:arrow_up:

crates/archlinux/src/test/mod.rs
72.72% <100.00%> (ø)

... and 3 files with indirect coverage changes
:loudspeaker: Thoughts on this report? Let us know!


Redirect results in double extensions

I ran into an issue recently when trying to configure redirects for URL's that included file extensions. The file extensions was appended an additional time to the URL. This is an issue that seems to have been introduced in afc18236c2d43907b96577272dff517ddc037640. There's no link to an issue or anything in the changelog, so I'm not sure what scenarios that change covers, but there looks to have been a small regression.
To reproduce the problem, configure a redirect like this for instance:
^/files(/.*)$: http://example.com$1

Then visit the URL http://grav.dev/files/example.zip (switching out the hostname as appropriate of course). You'll be automatically redirected to http://example.com/example.zip.zip.

I'll take a look at this.

Sorted.

Thanks!


How to get a minimal configuration json ?

Problem Statement
The doc says

The config files are based on the keycloak export files. You can use them to re-import your settings. But keep your files as small as possible. Remove all UUIDs and all stuff which is default set by keycloak.

Our legacy Keycloak export is 3000 lines long, so I'ld like to do exactly that, but I don't see how I can get there. Tried to make some three way diff between

An export from our legacy keycloak
An export from an empty keycloak, same version as legacy
An export from an empty keycloak, same major as legacy but latest minor

Couldn't achieve a result : list are unordered in the export, for a given realm I don't know if I can remove all the roles that seems to be default or not ... and  you end up with an invalid JSON because always end up removing a ) or a , somewhere you shouldn't.
Any recommendation, a script, any tools here ? Just an export without the default included would be enough
Proposed Solution
I've no idea of a solution atm
Environment

Keycloak Version: 21
keycloak-config-cli Version: from the Helm chart

  image:
    registry: docker.io
    repository: bitnami/keycloak-config-cli
    tag: 5.6.1-debian-11-r22

Java Version: ?

Additional information
No response
Acceptance Criteria
No response

In my project, I also have similar concerns, the full export of Keycloak always involves full data (with thousands of lines of property and values) and cannot be re-applied multiple times.
Unfortunately, I also don't have a good solution for that yet, we are doing it manually.
What I did was:

Remove all "id" and "containerId" properties by using find/replace by regex functionality (I think most IDEs support that).
Remove all properties and data that you know/believe have NO impacts on the clients that rely on Keycloak. (This would usually take a lot of time if the Keycloak has a lot of dependent services/clients).
Always use validate JSON online tools (ex: jsonlint) per changes. In case you might have sensitive data that you don't want to expose to the world, feel free to use any local tools.
If possible, store the full version for backup in case you miss any important configuration in step 2.

I would also love to know if anyone has suggestions on this. (This could be hard to implement since Keycloak could introduce breaking changes about their default properties).

I also does that manually. I could not find any other better approach.

I get some help using this jq command
jq 'del(.id, .realm, .accessTokenLifespanForImplicitFlow,
   .accessTokenLifespanForWebApps, .accessTokenLifespan,
  .offlineSessionIdleTimeout, .accessTokenLifespanInSeconds,
  .ssoSessionIdleTimeout, .ssoSessionMaxLifespan,
  .ssoSessionIdleTimeoutRememberMe, .ssoSessionMaxLifespanRememberMe,
  .accessCodeLifespan, .accessCodeLifespanLogin, .accessCodeLifespanUserAction,
  .accessCodeLifespanMobile, .notBefore, .registrationAllowed,
  .registrationEmailAsUsername, .rememberMe, .verifyEmail, .resetPasswordFlow,
  .editUsernameAllowed, .bruteForceProtected, .permanentLockout, .maxFailureWaitSeconds,
  .minimumQuickLoginWaitSeconds, .waitIncrementSeconds, .quickLoginCheckMilliSeconds,
  .maxDeltaTimeSeconds, .failureFactor, .requiredCredentials, .otpPolicyType,
  .otpPolicyAlgorithm, .otpPolicyInitialCounter, .otpPolicyDigits, .otpPolicyLookAheadWindow,
  .otpPolicyPeriod, .otpSupportedApplications, .webAuthnPolicyRpEntityName,
  .webAuthnPolicyAttestationConveyancePreference, .webAuthnPolicyAuthenticatorAttachment,
  .webAuthnPolicyRequireResidentKey, .webAuthnPolicyUserVerificationRequirement,
  .webAuthnPolicyCreateTimeout, .webAuthnPolicyAssertionTimeout,
  .webAuthnPolicyRegistrationRecoveryEnabled, .webAuthnPolicyRegistrationRecoveryCodesQuantity,
  .webAuthnPolicyRegistrationTokenBindingRequired, .webAuthnPolicyRegistrationAttestationConveyancePreference,
  .webAuthnPolicyRegistrationAuthenticatorSelectionCriteria,
  .keys)' < keycloak-realm-export.json > keycloak-realm-export-new.json

Nice, this jq deserve a spot in the documentation :)

I am also thinking of an approach how to do this.
This is the manual approach I see atm:

Replace Id-Ref with alias-Ref for authenticationFlowBindingOverrides (if not empty)
From the jq command from  @KarstenSiemer remove the ones which you have explicitely modified and therefore don't have its default value anymore, add "containerId". Apply it.
Remove the roles and clients that match the checks done here https://github.com/adorsys/keycloak-config-cli/blob/main/src/main/java/de/adorsys/keycloak/config/util/KeycloakUtil.java#L56 unless you have made changes to them.

Hello @rgarrigue
For your issue on getting a minimal configuration json, here is a script that can do so.
#!/bin/bash

INPUT_FILE="realm-config.json"
OUTPUT_FILE="keycloak-realm-export-minimal.json"

jq 'del(
  .id, .containerId, .accessTokenLifespanForImplicitFlow,
  .accessTokenLifespanForWebApps, .accessTokenLifespan, .offlineSessionIdleTimeout,
  .accessTokenLifespanInSeconds, .ssoSessionIdleTimeout, .ssoSessionMaxLifespan,
  .ssoSessionIdleTimeoutRememberMe, .ssoSessionMaxLifespanRememberMe,
  .accessCodeLifespan, .accessCodeLifespanLogin, .accessCodeLifespanUserAction,
  .accessCodeLifespanMobile, .notBefore, .registrationAllowed,
  .registrationEmailAsUsername, .rememberMe, .verifyEmail, .resetPasswordFlow,
  .editUsernameAllowed, .bruteForceProtected, .permanentLockout, .maxFailureWaitSeconds,
  .minimumQuickLoginWaitSeconds, .waitIncrementSeconds, .quickLoginCheckMilliSeconds,
  .maxDeltaTimeSeconds, .failureFactor, .requiredCredentials, .otpPolicyType,
  .otpPolicyAlgorithm, .otpPolicyInitialCounter, .otpPolicyDigits, .otpPolicyLookAheadWindow,
  .otpPolicyPeriod, .otpSupportedApplications, .webAuthnPolicyRpEntityName,
  .webAuthnPolicyAttestationConveyancePreference, .webAuthnPolicyAuthenticatorAttachment,
  .webAuthnPolicyRequireResidentKey, .webAuthnPolicyUserVerificationRequirement,
  .webAuthnPolicyCreateTimeout, .webAuthnPolicyAssertionTimeout,
  .webAuthnPolicyRegistrationRecoveryEnabled, .webAuthnPolicyRegistrationRecoveryCodesQuantity,
  .webAuthnPolicyRegistrationTokenBindingRequired, .webAuthnPolicyRegistrationAttestationConveyancePreference,
  .webAuthnPolicyRegistrationAuthenticatorSelectionCriteria, .keys
)
| walk(if type == "object" then del(.id) else . end)' < "$INPUT_FILE" > "$OUTPUT_FILE"

echo "Minimal export saved to $OUTPUT_FILE"

You can customize it to your like and it'll work fine.


stat: fix test on travis

See also #996 for diagnostics.
The travis builds on linux run inside a docker container. It looks like travis has changed the containers some time ago. Now all builds on linux fail because gnu stat does not detect the bind-mount of /dev/zero. This causes the output to be different, and the test to fail.
I assume test_char is meant to test a character special file in /dev. This PR uses /dev/pts/ptmx instead, both stat versions agree about it's output.
See https://travis-ci.org/wimh/coreutils/builds/177738828 where I added some additional logging.

Thanks for figuring this out. Though if coreutils stat works with bind mounts i
think we should too. Removing the test hides the missing parts. Maybe add
#[cfg_attr(not(feature="test_unimplemented"),ignore)] to the test so it
doesn't pollute travis results until someone wants to work on it and open a
issue for the problem?

I am not sure what the expected behaviour is, I guess I have to file a bugreport at gnu to find out. But even if we handle bind mounts wrong, test_char was not written to test that. I would suggest to add an additional test in that case to avoid testing two different things in a single test. Unless of course test_char was redundant anyway.
And to clarify, we handle bind mounts, gnu does not:
# mount -o bind /dev/zero /root/zero
# mount -o bind /root /home/root
# /usr/bin/stat -c %m /root/zero /home/root
/
/

Oh ok, then i mixed it up. Not sure what's the best path then.

From the docs:
The mount point printed by ‘%m’ is similar to that output by df, except that:

(...)
stat outputs the alias for a bind mounted file, rather than the initial mount point of its backing device. One can recursively call stat until there is no change in output, to get the current base mount point

The way I interpret this, both versions are wrong. I would expect:
# mount -o bind /dev/zero /root/zero
# stat -c %m /root/zero
/dev/zero
# stat -c %m /dev/zero
/dev
# stat -c %m /dev
/dev

# mount -o bind /root /home/root
# /usr/bin/stat -c %m /home/root
/root
# /usr/bin/stat -c %m /root
/
# /usr/bin/stat -c %m /
/

I also found this ticket discussing this.

Merged this as is for now, we still need to  figure out how to deal with %m.


Add afterBuild and afterDist options

This commit adds two hooks to allow to run external tasks after {target}:build and {target}:dist.

Codecov Report

Merging #152 into master will increase coverage by 0.33%.
The diff coverage is 81.25%.

@@            Coverage Diff             @@
##           master     #152      +/-   ##
==========================================
+ Coverage    41.6%   41.93%   +0.33%
==========================================
  Files          36       36
  Lines         971      980       +9
==========================================
+ Hits          404      411       +7
- Misses        567      569       +2

Impacted Files
Coverage Δ

src/lib/targets/base.ts
61.9% <66.66%> (+0.17%)
:arrow_up:

src/lib/targets/lib.ts
62.83% <90%> (+1.14%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef18201...0505e86. Read the comment docs.


[Snyk] Upgrade nuxt-windicss from 2.5.0 to 2.5.5

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade nuxt-windicss from 2.5.0 to 2.5.5.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 5 versions ahead of your current version.
The recommended version was released a month ago, on 2022-11-17.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Fixed #244


Added Auto-Save

Due to an issue, I saw multiple people requested auto-save so I went ahead and added it into the main program. (I figured it would be easiest this way as I had issues making this feature a plugin because it would save before you actually made the change to the document).
I hope this helps, let me know if there are any conflicts to this 👍

Hm, instead of every character/edit, is there a way for you to do it every second/variable time interval after the last edit, and on app close? I think it would be a bit better than calling a command and putting if statements everywhere. This method is from VSC, and I think it's pretty suitable for lite, plus a tiny bit more customizable and efficient. The current solution could also maybe get laggy for big files, if that's a concern.

In that case I could easily make this into a plug-in instead. I thought that making it every character update could be useful, especially for having to compile continuously, but I guess the plug-in could let you find tune your cool-down and such

Ohh yeah a plugin would be better, might still require modifications to the core module(detecting last edit) but it would be a lot less intrusive and more customizable! If it's possible, I would love for it to happen :D

Will there be any progress on this in the future? Sorry for bothering you, I just wanted this feature to be properly implemented someday :D

There definitely will be progress made in the future (I've been really busy recently) but all I need to do is make a simple sleep function and call the doc-save command after it. It should only take one coding session but I'm really busy so it will probably be out this weekend. I'm going to keep this PR open just so I can update you when I make a PR to the lite-plugins repo

Oh, wait did you make the plugin...?