Bugfix - not sending the custom endpoint to parseQueued method

Description
parseQueued function is not receiving the custom endpoint, so it is trying to construct an OTS and throws an exception: "Please create an endpoint manually before sending requests to a custom build."
How Has This Been Tested
Calling my generated API endpoint
Types of changes

[ ] Docs change / refactoring / dependency upgrade
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Requires a change to the official Guide documentation.

Well spotted, thanks :+1:


Added selftext to user submitted

There are only a small sampling of the fields returned from the submitted query currently parsed by serde.
The selftext (body text of the post), would be a useful field to include.

Thanks for that. There seem to be quite a few fields generally everywhere that aren't parsed, so I'll make an issue to note it.


Add: Undo git rm

Added functionality to undo git rm command
Sample : undo git rm
https://user-images.githubusercontent.com/38408670/186435882-701e455d-d052-44b4-b6c2-be2bd7f7f7f8.mp4

@all-contributors please add @nalin-singh for code


update from origin

merge from origin

Can one of the admins verify this patch?

I signed it!

I just what to pull origin to my trunk

This doesn't make any changes, so I'm closing it out. Please provide a better description in the pull request for future pulls.


UI scaling on high-dpi screen

Hi,
The UI does currently not seem to scale with higher DPI screens. I am using Arch Linux (KDE) and a 4K monitor. The KDE user interface scales nicely, but the tev interface becomes tiny.
I am not sure whether the underlying issue here is a nanogui problem, so feel free to close the issue if you think it should be fixed in nanogui itself. ;)
Delio

Hi Delio,
this indeed is a problem at nanogui's end. Currently, on linux, it only obtains the pixel ratio via gsettings, so if you are not using gnome it probably won't work. Since tev is already using a fork of nanogui with some similar fixes implemented, I'll have a look at how easy scaling is to add for KDE. In case you want to look into it yourself, given that you are able to replicate the problem, here is the file and function in question.
Cheers,
Thomas

For reference, I commented on this in the nanogui main repository
https://github.com/wjakob/nanogui/issues/202

Thanks for the help! I will test this locally on my machine and will merge afterwards. :)


Fix complete() flick problem

I have ported https://github.com/neovim/neovim/pull/4315 patch from neovim.
The original patch author: I and @junsinmk (Justin M. Keyes)
It fixes the flicker problem in complete().

Shougo wrote:

I have ported https://github.com/neovim/neovim/pull/4315 patch from neovim.
The original patch author: I and @junsinmk (Justin M. Keyes)
It fixes the flicker problem in complete().

Thanks!
--
Life is a gift, living is an art.		(Bram Moolenaar)
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

Thank you!


Minor BJData fixes

These are the previously discussed minor fixes to the BJData binary_reader code.

Replace the std::vector/std::map lookup tables with std::arrays.
This is meant to be an optimization (mostly avoiding unnecessary, repeated heap allocations). Whether std::lower_bound() is actually faster than std::find/std::find_if in this scenario is unknown.

Replace string_t::npos with binary_reader::npos.

Coverage remained the same at 100.0% when pulling 79e9357514b1339d7810f4f58c323e6cee7ccea4 on falbrechtskirchinger:bjdata-fixes into 9aafcbe96578f40276d7f779ab5421a908cea611 on nlohmann:develop.

Wow! This improves performance parsing a 136M document by 0.3%. This was truly worth it! :rofl:

Looks good to me.


Insertion Sort Code

Insertion Sort Algorithm in C++
Easy algorithm for sorting arrays.

Please move this code into proper sub folder


Show toolbar permission awareness

In Plone Classic you can add the following to rolemap.xml and hide the toolbar for some roles:
    <permission name="Show Toolbar" acquire="False">
      <role name="Manager"/>
      <role name="Site Administrator"/>
    </permission>

Would be nice to be able to hide toolbar also in Volto for some roles.

Is there any RestAPI endpoint to get that permissions info?

@ionlizarazu AFAIK, no :thinking:

So we have 2 ways, create an endpoint and get the configuration from the backend or add a setting in the frontend to manage it.
I think that the good one is the first one. @erral @sneridagh ?


Add App: Hamlet

Hamlet app. Mod to Consequencer adding two voice tracks.

Thanks for the comments and review.

Nice work, I'm looking forward to using this script in my patches! Thanks for another great contribution.


Extensions viewlet: delete key should uninstall the extensions

Refs: #20440

Focus the list of installed extensions
Click on delete key - nothing happens. I would expect the extension to get uninstalled

So, there should be a command associated to a key binding, to uninstall the extension on focus.


Fix parsing of record & list bodies.

Brief Description
Change the node structure of the record & list bodies to better represent the actual syntax of the language. This fixes some bugs as well as make the syntax more consistent & easier to read.
Fixes

TS can't properly detect durantion values in records. It is instead deteced as integer & a command node that create an error:

{
    key: 5sec
}

TS can't properly detect when value & key begins & allow invald syntax with value & key having no space or any separator in between. The following example is considered valid by TS:

{
    key: 1536another_key: 8549
}

What is changed

Records use a new body structure which manages entries & separators in between. This fixes the separator bugs described above & creates new nodes that can be used for highlighting & indenting.
Lists also have their structure changed in the same manner (both records & list body acts the same in nushell syntax, with exception to what is actually a record entry key: value vs a list entry value.
Record tests are added for the bugs described above. All tests have been updated with the new node syntax.

Details

New function is added to grammar js for general body structure. It consist of repeated statement all seperated by a mandotory separator, followed by a final statement with an optional separator.
This function is applied to the bodies of lists & records with the corresponding entries.
List has an added node for the list_entry itself (before it was just hardcoded into the list rule)
Separator is added for entries [,\s\n\r]
Comment is added on top of the code to improve LSP parsing of code with tsserver (commonly used by other TS projects for grammar.sj)
Tests are updated with new new node syntax - I've individually checked the diffs of all tests to make sure they are working as indented
New test has been added to check if records can accept duration input.
New test has been added to test if records accept invaild input with no separator between value & key.

It must have been a mistake on my part. Thanks for catching that - I re added the lost tests.

Thanks


Marking unused parameters with AWS_UNUSED_PARAM macro.

Description of changes:
This PR adds an AWS_UNUSED_PARAM(param) macro and respectively replaces all usages of the (void)param pattern. This makes identification of unused parameters simpler.
For context, I've been migrating a large database proxy system (of interest to AWS) from libevent/libuv over to AWS-C-IO. The code uses a number of portability/convenience macros, similar to those found in Hedley, and I'm looking to add the commonly-useful ones to AWS-C-COMMON. Accordingly, I'll update the other common-based libraries (io, http, mqtt, etc.) to use any new addition as well.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ColdenCullen - I'm interested in your opinions/suggestions on how I should proceed with not only this PR, but also any potential additions to this (and related) libraries. Prior to any larger changes, other than bug-fixes or test additions, should I want to make a change, would you prefer I put it in an issue for discussion pre-development? I have no desire to bloat your libraries, I just want to add things I think are generally useful to other developers and that will be used/needed within my codebase.

Not sure why my opinion is worth bonus points, but my feelings are:
Pros:

The macro is slightly clearer

Cons:

Canceling out a parameter now requires significantly more text
This change involves LOTS of churn

When in doubt, I tend to lean towards not making changes, but if anyone else on @awslabs/aws-sdk-common-runtime-team has feelings (looking at you @JonathanHenson), I could be swayed.

Not sure why my opinion is worth bonus points

I went off the contributor graph and the cursory observation that I've seen the latest responses from you in terms of communication regarding issues and such :)

I don’t really care one way or the other as long as it is consistent. We did something similar in the c++ sdk years ago. One benefit of the macro is that it’s explicit about intent, and reduces cognitive load. If we do move forward with this, let’s update the coding guidelines as well. Also keep in mind, these header files will be included in the C++ SDK at some point and this macro will likely clash with its macro of the same name. Maybe put an ifdef check in there as well.

This looks fine to me. I’ll talk with the team today and make sure everyone wants to move forward with the change.

Cool. Thanks!

talked to team about it this morning. We can't find a compelling reason why this is better than it currently is. And while the change is fine, since we're neutral on it, we prefer to provide Occam's razor on changes.
Can you provide us with an argument for beyond the subjective argument I made about "explicit intent?"

The code I'm working on will soon be handed-over to AWS. To that end, I've already been making a considerable number of changes to switch from third-party libraries over to AWS-specific ones along with performing general clean-ups. Trust me, I get it: my position is similar to your team's in terms of, where possible, making the fewest changes required. In this case, for those things in my codebase that already seem cleaner, I'd just prefer not to make them less-so.
While I can always keep custom definitions within the codebase, it seemed this would be a good candidate for common, which I assume is why it was also added to the aforementioned C++ library.
While I get the default stance, aside from churn, I don't see a negative. Having made the related update to all but one other (public) library based on common, I've ensured 100% passing test coverage.
Regardless of this PR, I guess my question going forward is: how do we avoid this in the future? I'd like to hand over code that is relatively-clean and I'm happy to contribute to the base (-common and -io libraries on which this will be based), but I don't want to spend a couple hours working on something that isn't likely to be accepted. Any suggestions you or your team has would be appreciated.

We'd obviously always love your contributions. They're very much welcome. We also have a long roadmap of things that need improvement/implementation if you're looking for something in your spare time.
Regarding your last paragraph, I'd suggest opening an issue for us to dialog on to avoid churn for a large change.
I'm still not against this, we're just mulling it over. I agree there's not really a negative here, we just also don't see the positive.

Cool. I'll open an issue on any future proposed change. This was a simple one, but I was definitely wondering how you would like to handle anything not-straight-forward. As an example: https://github.com/awslabs/aws-c-common/issues/123#issuecomment-495848822

While going through the IOT library samples I noticed it also has a similar macro, IOT_UNUSED.

I now have three full database protocol implementations rewritten with aws-c-common and aws-c-io; is this still iffy in terms of being merged? If the team if pro-merge, I'll run through and make sure it's up-to-date with master. If not, I'll just update my local definitions to use something other than AWS_UNUSED.


function parameters always same order

The order of for example helper and item or helper and index, should always stay the same in the different function parameters in the helper function. Every helper should become a leading comment with the defined order.

Hi! I am new to open source so I'm not fully sure how I can start working on this, Can I please be assigned this issue?

Hi! I am new to open source so I'm not fully sure how I can start working on this, Can I please be assigned this issue?

Hey, sorry, for late answer, i am also new as repo owner. If you cant assign yourself, you maybe just clone the repo, make a branch with changes, and then do a pull request here?

Hi! I am new to open source so I'm not fully sure how I can start working on this, Can I please be assigned this issue?

I invited you as collaborator, this should give you more options.

@aqil18
Assigned you also. I try to work from now on also strictly in branches, so it should be fine when you clone the repo only one time and work on the code, and only if i merged another branch you have to pull the new code into.

@aqil18
Steps are always: make a copy from the main branch, clone it. From there one make a own branch, for example with the naming 27-, if you finished your work, this branch has also to exist remote, not only on your local pc, and with this remote branch you can open a pull request that i merge your changes into main.


V2/logout

Summary
Implementing Logout
Description for the changelog
Implements logout functionality
Other info
Because V2 is using JWT as opposed to sessions and cookie auth, the server implementation is a bit different.  In this case, our only knowledge of long-term user authentication is via refresh tokens (managed by JWTs, with a separate signing key from the main jwt signing key).  Refresh tokens can be used to re-generate a JWT after the JWT expires. Logout on the server only invalidates the refresh token, forcing the user to complete authentication again should they request a protected resource.
On the front-end, all states are cleared, and if not a local instance, the logout method is called on the server.  By clearing the states, we are clearing all meaningful data from the sessionStorage as well.  There is still a schema left behind in the session storage, but I see no security implications there as this is an open source product, and anyone can view the schema.

Codecov Report

Merging #184 (51a3019) into v2-development (73fc257) will increase coverage by 0.28%.
The diff coverage is n/a.

@@                Coverage Diff                 @@
##           v2-development     #184      +/-   ##
==================================================
+ Coverage           97.47%   97.75%   +0.28%
==================================================
  Files                  60       60
  Lines                 831      847      +16
  Branches               13       14       +1
==================================================
+ Hits                  810      828      +18
+ Misses                 21       19       -2

Impacted Files
Coverage Δ

src/service/loginApi.js
100.00% <0.00%> (ø)

src/components/Navbar.vue
100.00% <0.00%> (ø)

src/store/actions/auth.js
100.00% <0.00%> (ø)

src/store/modules/auth.js
100.00% <0.00%> (ø)

src/config/routes.config.js
100.00% <0.00%> (ø)

src/controllers/homecontroller.js
100.00% <0.00%> (ø)

src/controllers/auth.js
100.00% <0.00%> (+5.71%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 73fc257...51a3019. Read the comment docs.


Refactor algorithm example for clarity

A slightly clarified algorithm example emphasizing the failure conditions.

Great simplification thank you.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target_5357bf7c-6cc9-419c-b9e1-6fad776f5eff/pull/1 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-target_5357bf7c-6cc9-419c-b9e1-6fad776f5eff.git

Docs Build status updates of commit 2a95a2e:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error: GitBranchDeletedOrForcePushed] Cannot sync git repo to specified commit because branch Release_Archive_main_2022-01-06-08-01-51 has been deleted or has been force pushed

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Update colorport to 2.0.5

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

There’s a very old commit mixed in.


Add option to have an Abandon release plan state

When there is a plan that has errors or something, we want to disable it. Right now we don't need UI because it is not a common scenario so we can change it in the backend.
This issue will change UI too to show that

https://github.com/Azure/azure-sdk-tools/assets/9868623/2c5805dc-e7a3-489c-a81c-b40ac452d965


Docs: broken styles

Hi,
There is a documentation issue.
It looks messed up (in FF 59.0.1 & chrome 65.0.3325.181)
and in chrome you cannot scroll the main content.

Thanks! Try mousing over the main content before scrolling, does that help?
There's a PR that's a work in progress to fix this: https://github.com/embermap/ember-data-storefront/pull/36

@ctjhoa could you see if the new site has fixed it? https://embermap.github.io/ember-data-storefront/latest/

Yeah the new one is better.
However, I have found one bug on the "LoadableStore" page.

The main content is truncated

Thoses bugs are now fixed so I close this issue.

Thanks!


Epilog detection unreliable when ip is on jmp with an 0x58 immediate before it

This is a very rare case of epilog analysis false-positive I found when checking epilog detection robustness for the WIP SEH backend, but I thought it was worth reporting.
The sample landed on an in-function jump:
0x35911b800      488b4258       mov rax, qword [rdx + 0x58] ; 0x58 looks like a pop but is actually an offset
0x35911b804      eba9           jmp 0x35911b7af             ; sample ip here

The function starts at 0x35911b6c0 and ends at 0x35911ba8c.
It looks like the current heuristic can break if you stress it with millions of samples. :P
https://github.com/mstange/framehop/blob/1517df9f51ca0b022e4db683948750982c7a5c35/src/x86_64/instruction_analysis/epilogue.rs#L33-L43
Wine uses something different: It checks if the jump lands within function bounds.
https://github.com/wine-mirror/wine/blob/d0ce5a77c60e0a9613f1af03d67d13b89816441b/dlls/ntdll/signal_x86_64.c#L720
This could be potentially more robust, but needs testing.

After analyzing more cases, I've realized that the jump target heuristic can't work for indirect jumps. (In general, the indirect jump identification is much harder, and there doesn't seem to be a reliable way to tell between indirect tail-calls and indirect switch-table jumps.) So the pop byte heuristic needs to stay at least for those cases.

I'm definitely in favor of adding a check for "is within function bounds" for the case where the target address is known.


Use regular cursor

It's a lot harder to target small streets with the hand cursor:
new UI:

old UI:

Maybe this is why: https://github.com/graphhopper/graphhopper-maps/blob/8515346ed76dd23ede5bbc082ea8be0e53d02d11/src/layers/UseBackgroundLayer.tsx#L54


oneOf was not implemented

Have a json schema:
...
"ip-address": {
"description": "",
"type": "string"
},
"unnumbered-id": {
"description": "",
"type": "integer",
"format": "int64"
},
"oneOf": [
{ "required": [ "ip-address" ] },
{ "required": [ "unnumbered-id" ] }
],
...
But in the Java code, "oneOf" is a generic Java Object, and so is "required"

This library does not support "oneOf" (#392).


feat adding Contributing Guidelines in a separate folder

This will be helpful for the contributor to understand better and face less difficulty while working on this project

Hey, this issue is already opened in issue #2


The "toggle-able" logger is broken in one critical way

The superclass ControlledLogger is great, except that it breaks the utility of the %(filename), %(funcName), and %(lineno) reporting. Surely there's a more elegant solution than re-invoking the frame inspection.

Fixed in 18 Jul commits (b8b22... - be8aa).


Bumping webhook version to v0.7.0-rc1

What type of PR is this?
/kind cleanup
What this PR does / why we need it:
Last step before v0.7.0-rc1 release, missed this in #1919. (Caught this by following release instructions and running make generate with this version). This also updates verify-examples-kind to load local test image for webhook since apparently that wasn't working properly since we moved to kustomize.
Does this PR introduce a user-facing change?:
NONE

/lgtm


question about how to set data Parallelism

Thanks for your contributions. Tutel is definitely a great work!
But I have difficulty applying this Tutel to a new framework.
If I have 8GPUs, And I want to set the number of experts to 4 while there are 2 local experts on a device (GPU).
Thus the GPUs will be divided into 4 groups, which each has 2 GPUs and contain 4 experts.
I think the GPU0, GPU2, GPU4, and  GPU6 have the experts with the same parameters.
How can I implement this setting?

Maybe I could this can be implemented with deepspeed.

Hi, thanks for your question. I'd like to confirm what you request: do you need the setting that uses 8 GPU to support 4 experts so that on average, each expert is stored on 2 GPUs?

Thanks for your reply. This is indeed what I require.
Now I have known how to do this with deepspeed. But I still have no idea about how to implement this with tutel only.

@Lechatelia The existing version yet doesn't include this feature when |GPU| > |Experts|. Please see this PR which will be merged once it passes regression tests: https://github.com/microsoft/tutel/pull/53/files, and it will support what you need by configuring: experts = {'type': 'ffn', 'count_per_node': 0.5, ..} which means each GPU will hold a half slice of an expert. If you concern about the precision of float value when it is large, you can also replace 0.5 with -2 which donates 1 / 2.
Note that we still see some optimization potential for the case |GPU| > |Experts| and it is yet not included in this PR.
We'll make it up soon after we have more evaluation about that.

Thanks a lot for your help.
I have another question. what should I do If 2 GPUs that hold the same expert constitute a data parallel process group, where each GPU will hold an entire expert.

I am curious about why you need that since it is more memory consuming.
I think this is a usage question about how you schedule group communicators with torch.distributed. Tutel's moe layer allows you specific custom-scheduled group communicators.
Do you mean by the following format?
GPU0 (Expert-0 Full)  GPU1 (Expert-1 Full) GPU2 (Expert-2 Full)  GPU3 (Expert-3 Full)
GPU4 (Expert-0 Full)  GPU5 (Expert-1 Full) GPU6 (Expert-2 Full)  GPU7 (Expert-3 Full)

Yeah,  This is just what I want to do.
I think there will be more data (higher batch size) to train an expert in this format.
Maybe I got this wrong.
Thanks for your help again.

You may have interests in this example: https://github.com/ghostplant/tutel/blob/shared_expert/tutel/examples/helloworld_data_model.py

Thank you!
I think I have figured out how to do this with your help ~ _ ~

@Lechatelia  In v0.1.5, using data parallelism mode you request is much simplified, and there is no any programming differences with other MoE normal cases, e.g. You no longer need to create custom group and concern which params to communicate on data_group/model_group or global group.
You just need to set count_per_node = -4 for you case which means every adjacent 4 GPUs will maintain one expert. Each GPU will hold a unique slice of an expert params, and you don't need further all_reduce them on the parallel_env.model_group. The original example helloworld_data_model.py have been replaced by a new simpler example helloworld_sharded_experts.py for your reference.

Hi, @ghostplant,
Thank you for your excellent repo. I have a question about setting only distributed data parallel mode using tutel. For example, I want to train a model with total 16 experts on 2 GPUs. The model size is acceptable so I want each GPU hold the whole parameters of the 16 experts. I don't want 8 experts in each gpu. I just want each GPU to feed different data into the model. Such training mode is similar to the original distributed data parallel of PyTorch. However, I could not figure out how to set the arguments in tutel. I find in your code the global_expert_count is normally computed as num_local_experts * world_size, that is num_local_experts * gpu_number. If I set num_local_experts as 16 / 2 = 8, there will be a model parallel. So could you give me some hints about how to set the arguments so that I could use only distributed data parallel?
Thank you very much.
Zhenxing Mi

@MiZhenxing To MoE parameters, data parallelism should be the worst choice. If you insist on using this inefficient approach. You need:
from tutel import net
single_data_group = net.create_groups_from_world(group_count=1).data_group

self._moe_layer = tutel_moe.moe_layer(
            gate_type = {'type': 'top', 'k': top_value, 'fp32_gate': args.fp32_gate},
            experts = {'type': 'ffn', 'count_per_node': num_local_experts, 'hidden_size_per_expert': hidden_size, 'activation_fn': lambda x: F.relu(x)},
            model_dim = model_dim,
            seeds = (1, dist_rank + 1, 1),
            a2a_ffn_overlap_degree = a2a_ffn_overlap_degree,
            parallel_type = args.parallel_type,
-            scan_expert_func = lambda name, param: setattr(param, 'skip_allreduce', True),
+           group = single_data_group,
        )

Comparing with moe_layer creation in helloworld.py, you need to remove scan_expert_func argument and add group argument.

Hi, @ghostplant, thank you very much for your help. I will test on these setting.

Hello, I am using DeepSpeed to implement data parallelism + pipeline parallelism + tutel moe. My node has 6 GPUs, and I started training with the command:
              python -m torch.distributed.launch --nproc_per_node=6 train.py
The model was split into 2 parts, so the data parallelism is 3 and the model parallelism is 2.

(1) If I set num_local_experts to 4 in moe_layer, the code would be as follows:
self._moe_layer = tutel_moe.moe_layer(
gate_type=_gate_type,
model_dim=in_features,
experts={'type': 'ffn',
'count_per_node': self.num_local_experts,
'hidden_size_per_expert': hidden_features,
'activation_fn': lambda x: self._dropout(F.gelu(x))},
scan_expert_func=lambda name, param: setattr(param, 'skip_allreduce', True),
seeds=(1, self.dist_rank + 1, self.dist_rank + 1),
batch_prioritized_routing=use_bpr,
normalize_gate=normalize_gate,
is_gshard_loss=is_gshard_loss
)
At this point, following Tutel's idea, global_expert_count = num_local_experts * world_size=24, each GPU has 4 experts, and there are a total of 24 experts on 6 GPUs, and all 24 experts are different. The code execution will be stuck in the communicate.py in the tutel source code:
https://github.com/microsoft/Tutel/blob/main/tutel/impls/communicate.py
line 450:     output, f_wait = simple_all_to_all(reshaped_input, group, background=True)
Perhaps it's because I use pipeline parallelism that the model is split into two parts and placed on two GPUs, but only one GPU can train the layer of the moe, causing communication issues? If I use data parallelism+tutel moe, the code can be executed successfully.
（2）Based on problem 1, I have modified the moe_layer code as follows：
my_processing_group=tutel.net.create_groups_from_world(group_count=1).data_group
self._moe_layer = tutel_moe.moe_layer(
gate_type=_gate_type,
model_dim=in_features,
experts={'type': 'ffn',
'count_per_node': self.num_local_experts,
'hidden_size_per_expert': hidden_features,
'activation_fn': lambda x: self._dropout(F.gelu(x))},
#scan_expert_func=lambda name, param: setattr(param, 'skip_allreduce', True),
seeds=(1, self.dist_rank + 1, self.dist_rank + 1),
batch_prioritized_routing=use_bpr,
normalize_gate=normalize_gate,
is_gshard_loss=is_gshard_loss,
group=my_processing_group
)
I set num_local_experts to 4 in moe_layer，Start training with the same command:
python -m torch.distributed.launch --nproc_per_node=6 train.py
and the code can be executed successfully.
At this point, there are still 4 experts on each GPU,  The model was split into 2 parts and moe_layer is divided into the first part. Therefore,  during the training process, the expert parameters for GPU 0, GPU 1, and GPU 2 have been updated, while the expert parameters for GPU 3, GPU 4, and GPU 5 have not been updated. Due to Deepspeed's all reduce operation on data parallelism, the expert parameters for GPU 0, GPU 1, and GPU 2 are the same, and the expert parameters for GPU 3, GPU 4, and GPU 5 are the same.
Here, I actually trained a total of 4 experts using Tutel Moe, right?

Hello, I am using DeepSpeed to implement data parallelism + pipeline parallelism + tutel moe. My node has 6 GPUs, and I started training with the command:
python -m torch.distributed.launch --nproc_per_node=6 train.py
The model was split into 2 parts, so the data parallelism is 3 and the model parallelism is 2.
(1) If I set num_local_experts to 4 in moe_layer, the code would be as follows:
    self._moe_layer = tutel_moe.moe_layer(
        gate_type=_gate_type,
        model_dim=in_features,
        experts={'type': 'ffn',
                 'count_per_node': self.num_local_experts,
                 'hidden_size_per_expert': hidden_features,
                 'activation_fn': lambda x: self._dropout(F.gelu(x))},
        scan_expert_func=lambda name, param: setattr(param, 'skip_allreduce', True),
        seeds=(1, self.dist_rank + 1, self.dist_rank + 1),
        batch_prioritized_routing=use_bpr,
        normalize_gate=normalize_gate,
        is_gshard_loss=is_gshard_loss
    )

At this point, following Tutel's idea, global_expert_count = num_local_experts * world_size=24, each GPU has 4 experts, and there are a total of 24 experts on 6 GPUs, and all 24 experts are different. The code execution will be stuck in the communicate.py in the tutel source code:

https://github.com/microsoft/Tutel/blob/main/tutel/impls/communicate.py
line 450:     output, f_wait = simple_all_to_all(reshaped_input, group, background=True)

Perhaps it's because I use pipeline parallelism that the model is split into two parts and placed on two GPUs, but only one GPU can train the layer of the moe, causing communication issues? If I use data parallelism+tutel moe, the code can be executed successfully.
(2) Based on problem 1, I have modified the moe_layer code as follows：
    my_processing_group=tutel.net.create_groups_from_world(group_count=1).data_group
    self._moe_layer = tutel_moe.moe_layer(
        gate_type=_gate_type,
        model_dim=in_features,
        experts={'type': 'ffn',
                 'count_per_node': self.num_local_experts,
                 'hidden_size_per_expert': hidden_features,
                 'activation_fn': lambda x: self._dropout(F.gelu(x))},
        #scan_expert_func=lambda name, param: setattr(param, 'skip_allreduce', True),
        seeds=(1, self.dist_rank + 1, self.dist_rank + 1),
        batch_prioritized_routing=use_bpr,
        normalize_gate=normalize_gate,
        is_gshard_loss=is_gshard_loss,
        group=my_processing_group
    )

I set num_local_experts to 4 in moe_layer，Start training with the same command:
python -m torch.distributed.launch --nproc_per_node=6 train.py
and the code can be executed successfully.
At this point, there are still 4 experts on each GPU,  The model was split into 2 parts and moe_layer is divided into the first part. Therefore,  during the training process, the expert parameters for GPU 0, GPU 1, and GPU 2 have been updated, while the expert parameters for GPU 3, GPU 4, and GPU 5 have not been updated. Due to Deepspeed's all reduce operation on data parallelism, the expert parameters for GPU 0, GPU 1, and GPU 2 are the same, and the expert parameters for GPU 3, GPU 4, and GPU 5 are the same.
Here, I actually trained a total of 4 experts using Tutel Moe, right?

Hello, I am using DeepSpeed to implement data parallelism + pipeline parallelism + tutel moe. My node has 6 GPUs, and I started training with the command: python -m torch.distributed.launch --nproc_per_node=6 train.py The model was split into 2 parts, so the data parallelism is 3 and the model parallelism is 2.
(1) If I set num_local_experts to 4 in moe_layer, the code would be as follows:
    self._moe_layer = tutel_moe.moe_layer(
        gate_type=_gate_type,
        model_dim=in_features,
        experts={'type': 'ffn',
                 'count_per_node': self.num_local_experts,
                 'hidden_size_per_expert': hidden_features,
                 'activation_fn': lambda x: self._dropout(F.gelu(x))},
        scan_expert_func=lambda name, param: setattr(param, 'skip_allreduce', True),
        seeds=(1, self.dist_rank + 1, self.dist_rank + 1),
        batch_prioritized_routing=use_bpr,
        normalize_gate=normalize_gate,
        is_gshard_loss=is_gshard_loss
    )

At this point, following Tutel's idea, global_expert_count = num_local_experts * world_size=24, each GPU has 4 experts, and there are a total of 24 experts on 6 GPUs, and all 24 experts are different. The code execution will be stuck in the communicate.py in the tutel source code:

https://github.com/microsoft/Tutel/blob/main/tutel/impls/communicate.py
line 450:     output, f_wait = simple_all_to_all(reshaped_input, group, background=True)

Perhaps it's because I use pipeline parallelism that the model is split into two parts and placed on two GPUs, but only one GPU can train the layer of the moe, causing communication issues? If I use data parallelism+tutel moe, the code can be executed successfully.
(2) Based on problem 1, I have modified the moe_layer code as follows：
    my_processing_group=tutel.net.create_groups_from_world(group_count=1).data_group
    self._moe_layer = tutel_moe.moe_layer(
        gate_type=_gate_type,
        model_dim=in_features,
        experts={'type': 'ffn',
                 'count_per_node': self.num_local_experts,
                 'hidden_size_per_expert': hidden_features,
                 'activation_fn': lambda x: self._dropout(F.gelu(x))},
        #scan_expert_func=lambda name, param: setattr(param, 'skip_allreduce', True),
        seeds=(1, self.dist_rank + 1, self.dist_rank + 1),
        batch_prioritized_routing=use_bpr,
        normalize_gate=normalize_gate,
        is_gshard_loss=is_gshard_loss,
        group=my_processing_group
    )

I set num_local_experts to 4 in moe_layer，Start training with the same command: python -m torch.distributed.launch --nproc_per_node=6 train.py and the code can be executed successfully.
At this point, there are still 4 experts on each GPU, The model was split into 2 parts and moe_layer is divided into the first part. Therefore, during the training process, the expert parameters for GPU 0, GPU 1, and GPU 2 have been updated, while the expert parameters for GPU 3, GPU 4, and GPU 5 have not been updated. Due to Deepspeed's all reduce operation on data parallelism, the expert parameters for GPU 0, GPU 1, and GPU 2 are the same, and the expert parameters for GPU 3, GPU 4, and GPU 5 are the same. Here, I actually trained a total of 4 experts using Tutel Moe, right?

Hi. For case (1), I didn't see any settings related to enable pipeline parallelism but data parallelism of size 3. Do you pipeline parallelism and model parallelism are 2 different concepts. Can you explain the pipeline setting clearly? Anyway, not setting dedicated group=xxx is of course not true. For case (2),  if you choose tutel.net.create_groups_from_world() to create groups, it could simplify a lot in groups maintaining, however, DeepSpeed's group maintaining will be not useful and even introduce more troubleshooting since you hyrbid 2 different group creation maintainers. I think you should firstly decide which group maintainer to choose while disabling another one.

@luuck Considering pipeline parallelism is always slow. For your requirement, what's the problem for you not considering "3 data parallelism + 2 expert parallelism"?

For case (1), I use the following code to implement pipeline parallelism：
    layer = torch.nn.Sequential(
        model.input_layer,
        model.encoder0,
        model.downsample1,
        model.encoder1,
        model.downsample2,
        model.encoder2,
        model.decoder2,
        model.upsample2,
        model.decoder1,
        model.upsample1,
        model.decoder0,
        model.output_layer
    )
    pipeline_model = PipelineModule(layers=layer, loss_fn=self.criterion, num_stages=2, partition_method="uniform")

    data_set = WeatherData(args, "train")
    ds_config = self.get_ds_config()
    self.model, _, self.data_loader, _ = deepspeed.initialize(
        #args=self.args,
        model=pipeline_model,
        #model_parameters=[p for p in pipeline_model.parameters() if p.requires_grad],
        model_parameters=pipeline_model.parameters(),
        config=ds_config,
        training_data=data_set
    )

In this code，num_stages=2  means that I have split the model into two parts, and moe_lay is placed in encoder2, so it will be divided into the first part.
I started training with the command: python -m torch.distributed.launch --nproc_per_node=6 train.py，the code executed Failed, The 6 GPUs of the node are occupied as follows：
|    0   N/A  N/A     13353      C   ...iforge3/envs/awakenpy/bin/python3.9       6862MiB |
|    1   N/A  N/A     13354      C   ...iforge3/envs/awakenpy/bin/python3.9       6862MiB |
|    2   N/A  N/A     13355	 C   ...iforge3/envs/awakenpy/bin/python3.9       6862MiB |
|    3   N/A  N/A     13356	 C   ...iforge3/envs/awakenpy/bin/python3.9       8324MiB |
|    4   N/A  N/A     13357	 C   ...iforge3/envs/awakenpy/bin/python3.9       8324MiB |
|    5   N/A  N/A     13358	 C   ...iforge3/envs/awakenpy/bin/python3.9       8324MiB |
In this way，The data parallelism is 3, which are GPU 0+GPU 3, GPU 1+GPU 4, GPU 2+GPU 5. The first part with moe_layer of the model is on GPU 0，GPU 1 and GPU 2, while the second part is on GPU 3，GPU 4 and GPU 5.
If I set  num_stages=1, it means that only use data parallelism+tutel moe, the code can be executed successfully,The 6 GPUs of the node are occupied as follows：
|    0   N/A  N/A     23301      C   ...iforge3/envs/awakenpy/bin/python3.9      12920MiB |
|    1   N/A  N/A     23302      C   ...iforge3/envs/awakenpy/bin/python3.9      12920MiB |
|    2   N/A  N/A     23303	 C   ...iforge3/envs/awakenpy/bin/python3.9      12920MiB |
|    3   N/A  N/A     23304	 C   ...iforge3/envs/awakenpy/bin/python3.9      12920MiB |
|    4   N/A  N/A     23305	 C   ...iforge3/envs/awakenpy/bin/python3.9      12920MiB |
|    5   N/A  N/A     23306      C   ...iforge3/envs/awakenpy/bin/python3.9      12920MiB |
In this way，The data parallelism is 6，There is a whole model on each GPU.
Based on the above analysis, so I said that perhaps it's because I use pipeline parallelism that the model is split into two parts and placed on different GPUs, but only GPU 0，GPU 1 and GPU 2 can train the layer of the moe, causing communication issues?
For case (2), in order to solve the problem，The model size is acceptable so I want each GPU hold the whole parameters of the experts. I don't want a part of experts to each gpu. I just want each GPU to feed different data into the model. Such training mode is similar to the original distributed data parallel of PyTorch. So I choose data parallelism for MoE parameters. I have chosen your following opinion：
https://github.com/microsoft/Tutel/issues/52#issuecomment-1105144317

My model cannot fit on a single GPU, so I have to use pipeline parallelism to divide the model into two parts.
Additionally, I use Deepspeed Zero1, so I choose Choice-2

My model cannot fit on a single GPU, so I have to use pipeline parallelism to divide the model into two parts.
Additionally, I use Deepspeed Zero-1 + data parallelism + pipeline parallelism + tutel moe , so I choose Choice-2

My model is too large to fit a single GPU, so I have to use pipeline parallelism to divide the model into two parts.
Additionally, I use Deepspeed Zero-1 + data parallelism + pipeline parallelism + tutel moe , so I choose Choice-2

@luuck Sure, please use this setting when creating moe_layer:
        self._moe_layer = tutel_moe.moe_layer(
            gate_type = {'type': 'top', 'k': top_value, ...},
            experts = {'type': 'ffn', 'count_per_node': 4, 'hidden_size_per_expert': hidden_features, ...},
            model_dim = in_features,
            scan_expert_func = None,
            seeds = (None, None, None),
            group = dist.new_group(ranks=[dist.get_rank()]),
        )

Keep in mind the following items:

set scan_expert_func = None since it will work in tradition data-parallel method, instead of Zero-2/3 method.
set seeds = (None, None, None) to let its expert parameter initialized and maintained by external framework like Deepspeed.
set group = dist.new_group(ranks=[dist.get_rank()]) so that Tutel Moe layer will internally fallback to legacy data parallel layout.


feature: provide refresh permissions

Description
This PR provides and exposes function named refreshPermissions in auth context that can be used to get latest permissions after actions made in admin panel that effects those permissions. refreshPermissions is called automatically by Payload CMS only when the user changes, but can be used by plugin or project developers for other means. The feature is needed to overcome problems presented in dicussion #2191 and is related to an issue in plugin I'm developing: https://github.com/joas8211/payload-tenancy/issues/2.

[x] I have read and understand the CONTRIBUTING.md document in this repository

Type of change

New feature (non-breaking change which adds functionality)
This change requires a documentation update

Checklist:

[x] I have added tests that prove my fix is effective or that my feature works
[x] Existing test suite passes locally with my changes
[x] I have made corresponding changes to the documentation

I wasn't sure where to put the test in the folder structure, so I added new folder to root test/ folder. That's for example something I would want to get feedback for, if there's better place for it.


Add an optional 'image' field to the dataset spec?

So STAC browser lets us do nice displays of catalogs, like http://planet-stac.netlify.com/
But the pages end up pretty bare. I was thinking it could be nice to add the ability to reference an image that represents either the provider or the dataset. I'm mostly thinking of a logo - like on the one above I'd put the Planet logo. You could also see a thumbnail representation of the dataset?
Lots of ways to do this. Could have a 'provider image' in the provider section. Could have just an image link in the main section that people could use as they want. Could even just use a special 'rel' in the 'links' section, since it should be a link to an image.

Hmm, not sure whether that's really a core thing. I'd more see this as part of an extension. Not even sure whether that would be a general extension, which we could call 'design extension', 'customize extension' or so. It could also just be a "vendor" extension for/by the stac-browser.

Yeah, you're right, an extension makes sense. And you remind me of my philosophy I try to get in to all extensions (which we've slipped from some), which is to start in implementations.
I'll close this and add to STAC-Browser, and can play with adding some there.


Mark PWM and Servo Pins

Uses hasPWM boolean field to address if a pin within an array supports PWM, or is Analog-Input Only (or an input-only sensor is connected to the pin, such as a light sensor).
The pwm Wippersnapper_Components on the Adafruit IO interface must only display the hasPWM: true pins.
Examples added for ESP32, ESP8266 and SAMD51 platforms.
Resolves https://github.com/adafruit/Wippersnapper_Boards/issues/68
cc @ladyada  - does this make sense?

After the PR has been reviewed, I'll add this to all boards within this repo.

Marking this property false by default

note: possibly switch to hasPWM to remove ambiguity around analog pins without and with PWM support.

@lorennorman  Re-requesting a review. I've covered the Adafruit boards here and added both pwm and servo flags.
We'll want to make an issue for adding these flags to the non-adafruit boards (possibly a good hacktoberfest issue).

@brentru woops i didnt see the tag above, do you still need me to 👁

@ladyada  nope! worked it out with Loren


Add new grid gutter options

With #22940 I unshipped the Sass map and code used to loop over those values. It generated the wrong code and isn't the approach I'd like to use moving forward. Looking at .no-gutters, I think this is the method I'd like to use—modifier classes on the row that can be customized by breakpoint.

Very Right decision @mdo... Best of luck in your progress dude :+1:

Sorry for asking, but is this option will be available in the final release?
BTW keep up the good work!

Otherwise, please explain how to add custom gutters for mobile devices.
I was searching for a way to have 5px gutter on mobile, 15 on tablet and 30 on desktop (just an example).
What is the right way foreward?

@x-christian-romeni I made a seperate scss file with custom gutters, but not tested it, so it might brake something. Not sure if it will work fine and reading this repo, maybe there was a good reason why this feature has been unshipped
` $my-gutters: (
xs: 5px,
sm: 5px,
md: 15px,
lg: 30px
);
.row.custom-gutter {
@media (min-width: map-get($grid-breakpoints, lg)) and (max-width: map-get($grid-breakpoints, xl) - 1px) {
margin-left: map-get($my-gutters, lg) *(-1);
margin-right: map-get($my-gutters, lg) *(-1);
}
@media (min-width: map-get($grid-breakpoints, md)) and (max-width: map-get($grid-breakpoints, lg) - 1px) {
margin-left: map-get($my-gutters, md) *(-1);
margin-right: map-get($my-gutters, md) *(-1);
}
margin-left: map-get($my-gutters, sm) *(-1);
margin-right: map-get($my-gutters, sm) *(-1);
}
[class^='col'].custom-gutter,
[class*=' col'].custom-gutter {
@media (min-width: map-get($grid-breakpoints, lg)) and (max-width: map-get($grid-breakpoints, xl) - 1px) {
padding-left: map-get($my-gutters, lg);
padding-right: map-get($my-gutters, lg);
}
@media (min-width: map-get($grid-breakpoints, md)) and (max-width: map-get($grid-breakpoints, lg) - 1px) {
padding-left: map-get($my-gutters, md);
padding-right: map-get($my-gutters, md);
}
padding-left: map-get($my-gutters, sm);
padding-right: map-get($my-gutters, sm);
} `

It was removed because the file size was getting too large.

@myorangeca Now I understand! Thanks!

Hi all, thanks for this.
Totally understand it's been removed, however it's still in the documentation. I spent ages trying to work out what the hell was going on!
Could remove it? Happy to do it myself if that helps?
Cheers,
Paul

@pauldcollins If i'm understand it correctly, then it is in the Alpha documentation, what you are reading. For the Beta it is here.

Hi @Tyutyesz - my bad, thanks for pointing that out!

any news on this?

would really want this in bs4.

It's also still in the documentation: https://getbootstrap.com/docs/4.0/layout/grid/#columns-and-gutters

...while $grid-gutter-width allows breakpoint-specific widths that are divided evenly across padding-left and padding-right for the column gutters.

@aubricus I'm sorry, but I still don't get how you could achieve breakpoint-specific gutter widths?

@ahennie
I wasn't indicating that you could. In fact, as far as I can tell it's no longer possible.
The current documentation needs to be updated to reflect that.

Can someone please update the docs if this is not implemented? I went down a rabbit hole today trying to get this to work :(

Coming back to this for v4.2 with #26825.


Does remark-html work only with command line and .remarkrc

I encounter an issue when trying to use only command line to execute remark. This is my .remarkrc
{
  "output": true,
  "plugins": {
    "lint": {
      "no-shell-dollars": false
    },
    "github": true,
    "comment-config": true,
    "toc": true,
    "validate-links": true,
    "html": true
  },
  "settings": {
    "commonmark": true,
    "bullet": "*"
  }
}

I then execute
remark history.md --output history.html

This does not seem to work. Is remark-html intended to be used in conjunction with .remarkrc or do I have to do this programmatically.
Thanks.

Please provide more information, such as version numbers, contents of files.
remark-html works with .remarkrc files on the CLI, and programatically on the API.
P.S. Read the docs/

I upload my files to GitHub: https://github.com/kahwee/remark-test2
The exact error message is:
> remark testing.md --output testing.html

testing.md
        1:1  error    Error: Invalid value `true` for setting `options`
    at raise (/Users/kahwee/projects/remark-test2/node_modules/remark/lib/utilities.js:37:11)
    at Of.compilerPrototype.setOptions (/Users/kahwee/projects/remark-test2/node_modules/remark/lib/stringify.js:881:9)
    at Of.Compiler (/Users/kahwee/projects/remark-test2/node_modules/remark/lib/stringify.js:822:10)
    at new Of (/Users/kahwee/projects/remark-test2/node_modules/unherit/index.js:58:22)
    at Of.HTMLCompiler (/Users/kahwee/projects/remark-test2/node_modules/remark-html/index.js:57:26)
    at Processor.stringify (/Users/kahwee/projects/remark-test2/node_modules/unified/index.js:226:16)
    at Ware.stringify (/Users/kahwee/projects/remark-test2/node_modules/remark/lib/cli/file-pipeline/stringify.js:56:35)
    at Ware.<anonymous> (/Users/kahwee/projects/remark-test2/node_modules/wrap-fn/index.js:75:16)
    at Ware.<anonymous> (/Users/kahwee/projects/remark-test2/node_modules/wrap-fn/index.js:57:27)
    at next (/Users/kahwee/projects/remark-test2/node_modules/ware/lib/index.js:85:20)

✖ 1 error

To execute the test, run npm run build
Pardon me if I missed some details in remark docs.
Thanks.

Oh I see, don't use 'true' as the value for the keys in '.remarkrc', instead use 'null', in which case they default to an empty object!

Hey thank you @wooorm, that explains.
Do you mind if I contact you through email? I want to improve Node.js guides generator based on https://github.com/nodejs/node/pull/4866
I want to know what your thoughts on how I should be implementing this where there is a template involved. Should I fork remark-html to do this?


Annual Review - Open Review Initiative

Renewed for another year by TAC vote on 8/23/2023

@erikstrauss FYI for 2024 date.

Deck for 2024 annual review - https://docs.google.com/presentation/d/1aC-PXOWwoCJ45gPSejx59RO3FBugs_6L/edit#slide=id.p1
Project would like be considered for moving from Sandbox to Incubation

Approved via LFX TAC vote for renewal - Sept 5, 2024


Rails installation using importmaps

Initial guide for explaining the setup of import maps in Rails.

Hey @sebscholl, thank you so much for working on this!
Just curious, is this derived from the importmap-rails README? If so, we should maybe declare it as such.

No problem @marcoroth . I wrote this while testing the rails importmap:install. That said, I'll update the guide with a reference to the Working with Javascript in Rails guide (https://guides.rubyonrails.org/working_with_javascript_in_rails.html)

Happy to help.


Smart Contract Support

Enable https://github.com/CosmWasm/cosmwasm plz :) I want to write smart contracts on Akash.

Why Wasm? I find Atmega 8-bit assembler to be very pleasant to write.


Update to use M1 CircleCI machines

Intel machines are deprecated by CircleCI. This updates our machines to use the M1 machines

          1 Message

:book:
Size increase: 0.00 KB

  Generated by :no_entry_sign: Danger


Parametrize public ingress gateway in-cluster hostname

This removes the hardcoded istio-ingressgateway.istio-system.svc.cluster.local hostname used in VirtualServices. Instead, the hostname is fetched from an annotation on the Namespace.
Testing
Follow the updated quickstart: https://github.com/israel-hdez/odh-model-controller/blob/1cd1ef6e082714236c3aacac5bb137c4187e8923/docs/quickstart-traffic-splitting.md
Merge criteria:

[X] The commits are squashed in a cohesive manner and have meaningful messages.
[X] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[X] The developer has manually tested the changes and verified that the changes work

@bartoszmajsak @aslakknutsen FYI. Related to https://github.com/maistra/odh-project-controller/pull/17


Writes fail (RPC timeouts) whenever the DB is not available in the current region

Attempting to use the Fly repo from any region in which a replica is not yet running, seems to work perfectly fine...
... that is, until any mutating calls (insert, update etc), or calls like Fly.Postgres.rpc_and_wait are made. Instead of transparently falling back to directly using the sole available DB connection (and maybe logging a warning?), these RPC calls will always time out.

At first I thought this was just a 'principle of least surprise' kind of issue that users which follow the guide to the letter would not stumble across. But actually the problematic situation can also arise during normal operation: When the read replica in the local region fails, DB traffic from that region is again transparently rerouted to the primary DB and the same situation occurs!

Interesting! Thanks for reporting this and detailing the failure situations!


Update webhook for new alertmanager.

As we've a new data format the version number needs a bump.
Also make alerts with no annotations easier to handle for
users on the receiving end.
@fabxc

:+1:


build cluster from linux AMIs?

It'd be nice to be able to build cfncluster from standard AMIs. Currently, the StarCluster AMIs are very out of date (stuck on Ubuntu 12 series) which causes problems with software installation. An AWS customer representative suggested cfncluster as an alternative, but I'm worried that the cfncluster AMIs will go out of date too in the future.

You can take the code from GitHub and install onto an AMI, then probably update the CloudFormation, however there is work ongoing to support more OS choices and make it easier for customers to use their own AMI and customize existing.

I see, thanks for the clarification. What would need to be done in addition to cd bootstrap; make install to get this installed on an AMI? I'd be happy to help with documentation or be a guinea pig. I'm interested in getting this working on Ubuntu 14.04 and autodeploying an ansible configuration upon node instantiation.

@dkoppstein Looking over the coder, it is not clear to me that cd bootstrap; make install is enough to build an AMI. For example where is SGE installed?

With the move to Chef for AMI building and bootstrapping, it should now be simpler to build a custom AMI. You can use the build_ami.sh script as part of the https://github.com/awslabs/cfncluster-cookbook project.

Nice! Can I use the Packer config file as a basis for my install directly?

Yes, although the build_ami.sh script sets some required variables used by the packer json files.
Also the goal of CfnCluster is to provide an environment that you can quickly launch and tear down. As such, customers typically don't customize an AMI but rather add simple customizations using post_install and then additionally add an application environment using an EBS snapshot.

Thanks. We're using it as a longer running cluster with many users giving input to what should be installed, then tearing down and re-launching when new software is required. But I might be able to 'retrain' my collabs to use it more in the fashion as designed.


Mismatch with code

Hi team,
I moving this from here: https://github.com/MicrosoftDocs/azure-docs.nl-nl/issues/549
the user reports an issue with the code
@nicowitteman commented 2 days ago — with docs.microsoft.com
Running the code returns a different set of data in the Token Info pane:
Name, Token Expires, Access token, Id Token, Tenant Id, whereby Name and TenantId are empty

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 90602ccd-184e-bb74-504c-5b06eb8eb4ab
Version Independent ID: 3d4865d9-f91f-9aeb-16a8-9789e647c068
Content: Tutorial - Authenticate users in a native client application - Azure Active Directory B2C
Content Source: articles/active-directory-b2c/active-directory-b2c-tutorials-desktop-app.md
Service: active-directory
Sub-service: b2c
GitHub Login: @mmacy
Microsoft Alias: marsma

@CeciAc Thanks for your feedback! We will investigate and update as appropriate.

Hey @nicowitteman thanks for letting us know about this! Can you confirm if you had changed the tenantid to match your tenantid?
Per the section : https://docs.microsoft.com/en-us/azure/active-directory-b2c/active-directory-b2c-tutorials-desktop-app?tabs=applications#configure-the-sample
You have to replace the values in the code with your values otherwise the sample will not run properly.

Hello Frank,
Thanks for your reply. Yes I have replace the TenantId, and following the instruction I looked for AzureAdB2CHostname to replace that as well, but that is not in the code.
The set of fields returned in the actual code is differing from the set as shown in the screen print, too (with Job Title among other fields).
Kind regards,
Nico
Van: Frank Hu<EMAIL_ADDRESS>Verzonden: dinsdag 26 november 2019 00:51
Aan: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>CC: nicowitteman<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Onderwerp: Re: [MicrosoftDocs/azure-docs] Mismatch with code (#43514)
Hey @nicowitteman https://github.com/nicowitteman  thanks for letting us know about this! Can you confirm if you had changed the tenantid to match your tenantid?
Per the section : https://docs.microsoft.com/en-us/azure/active-directory-b2c/active-directory-b2c-tutorials-desktop-app?tabs=applications#configure-the-sample
You have to replace the values in the code with your values otherwise the sample will not run properly.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/MicrosoftDocs/azure-docs/issues/43514?email_source=notifications&email_token=AF5JEBUZ7JARPXXGE4DSBRLQVRQEVA5CNFSM4JROT6FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFEGTIQ#issuecomment-558393762 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AF5JEBSM6HZMQ5OM5LMVHIDQVRQEVANCNFSM4JROT6FA .

Hi @nicowitteman, thanks for reporting this. I suspect that what has happened is that you downloaded a ZIP archive of the sample using the link we provide in the article, which is now incorrect. The link is to the master branch archive, which is now an incorrect because it appears that the default branch was changed to msalv3.
Please use this download link instead, and try the running through the tutorial again--it should work better for you:
https://github.com/Azure-Samples/active-directory-b2c-dotnet-desktop/archive/msalv3.zip
Sorry for the trouble, I'll get the article updated with the correct link immediately.
#reassign:mmacy

Works like a charm now. Thanks.


Question: What font is used in the screenshot?

Hi! Love the theme! What is the font used in the screenshot by the way?

Hi!
It's Caskaydia Cove Nerd Font SemiLight, the Nerd font version of Cascadia Code.


Move Karabiner config file to ~/Library/Preferences

Hi,
it's a Mac app, no need to use ~/. conventions. The right place should be ~/Library/Preferences

Again, I personally like .karabiner.d. I guess it depends if you spend half your time in linux. But I would not object if it changed.

But it's not a plist any more (which is awesome by the way). Shouldn't Library/Preferences only be plist files?

Common tools that use ~/. conventions, what I could find on my machine.

oh-my-zsh
git
rbenv
bundler
vim
bash
npm
cocoapods
curl(netrc)
docker
jupyter notebooks
mitmproxy

Also most dotfiles repos and similar setups work with ~/. conventions because most tools use it

You should check out this open pull request and @tekezo comment on it
https://github.com/tekezo/Karabiner-Elements/pull/114

I understand that many cross-platform applications follow the ~/.appname convention to store their own data, it's a BSD/Linux tradition. I thought that Karabiner was a Mac application so I don't see why it should go with that convention when the platform has a different one altogether.
Let's just stick to the native conventions, going against the grain of the platform may have unintended effects (iCloud based preferences sync, objections from the App Store, I don't know but why risk anything?)


Add iterator to Collection class

Is your feature request related to a problem? Please describe.
The collection class is a wrapper around an array. However, while the forEach method is implemented, other ways of iterating (for...of, spread operator, etc). are not. For those you always need to first call getArray(), which can lead to a bit of a confusing API (since you get an array that is mutable, but which you should not mutate) etc.
Describe the solution you'd like
I think it would be a nice simple addition to add an iterator on the collection, which could be as simple as:
[Symbol.iterator]() { return this.getArray().values() }
I'm willing to provide a PR if this is acceptable.

I think this is a bit like having a getFeaturesIntersectingExtent method in
https://github.com/openlayers/openlayers/issues/9647#issuecomment-499076061, maybe a nice-to-have but if anyone really needs it it is very easy to add in the application as in https://codesandbox.io/s/layer-clipping-vector-forked-s2owq  If included in the library Array.prototype.values and Symbol.iterator would need to be added to the list of polyfills needed by older browsers.

fair enough


try to avoid multiline panics

I started poking around and found this panic message
        thread 'tests::test' panicked at 'Assert CLI failure:
Expected output to contain
41
but could not find it in
42
', /some/file

This reminds me that my atom IDE currently fails to parse multi-line panics - I just filed this issue yesterday. ^^ https://github.com/AtomBuild/atom-build-cargo/issues/85
When I compare it with the output of assert_eq!(42, 41);
... 'assertion failed: `(left == right)` (left: `42`, right: `41`)' ...

How about something similar to what we already know from std:
... 'CLI assertion failed: `(Output expected to contain `41`)` (observed: `42`)' ...

I only did that because the print_exactly panic is a full, colorful diff anyway… and instead of 42 and 41 the values could well be Lorem ipsum dolor and the full help message…

Ok, interesting. Would it be an option to print to stderr before panicking?
stderr output could be multi-line, and the panic only a one-line summary? That might lead to garbled output, when multiple tests run concurrently - I'm not really sure. I'm just thinking out loud about alternatives. :-)
If you ask me, at least the case above looks nicer in one line.

I will submit a suggestion in a PR. :-)


move tools to a separate directory

The goal is to avoid dependencies between tools.
For example the latest phpstan and the latest phpunit depend on a different version of https://github.com/sebastianbergmann/diff
Also it won't cause as many potential dependency conflicts between RubixML and the end user projects.
I modified composer.json, so scripts should work exactly as before (for example composer anayze runs phpstan).
I made an exception for phpunit, so phpstan can still analyze the test files.
Next step would be to also add https://github.com/rectorphp/rector to the tools directory and update phpunit to v11 (tbd in a different PR)

Is this solving a problem that we have currently? Or a potential one in the future?

@andrewdalpino it solves the current problem: I can't use the latest PHPUnit version alongside the latest PHPStan because they conflict on the required https://github.com/sebastianbergmann/diff library version.
I want to upgrade PHPUnit, because the current version reaches EOL in the February: https://phpunit.de/supported-versions.html
I'm just stuck in dependency hell - I wanted to decouple all of those tooling dependencies from the core codebase. :D

Hey @mcharytoniuk is this still needed?


How to add this to a react project???

react-admin
This is a react project base on gentelella.May it be helpful


[question] volunteering to test for group ironman

Hey there. Not really an issue, more so a discussion topic - but, it does not look like this repo has that option available.
I was wondering if there's any chance of you looking to re-enable the ability for group ironman accounts to use this plug-in and was curious what you needed tested. I don't mind lending a hand if you are interested, since I was hoping to get my clan to more actively use clan chat (as opposed to friend chat/chat channel), but the obvious reason for your plug-in is old habits die hard. Please let me know if there's anything I can do or if you have an agenda for re-introducing gim support. Thanks in advance.

Yeah main issue is I don't have a gim account that is also in a clan. Probably needs an hour or two on one to test and rewrite code to work for them.

Yeah main issue is I don't have a gim account that is also in a clan. Probably needs an hour or two on one to test and rewrite code to work for them.

I have a gim account that is also in a clan, so that works perfectly. I also have programming experience. Let me know if you want to work together to get this tested :) Feel free to contact me on discord "neverlucky#6867"


fixed an unclosed  tag that was triggering invariant element id error

Pre-Submission Checklist

[x] Your pull request targets the staging branch of freeCodeCamp.
[x] Branch starts with either fix/, feature/, or translate/ (e.g. fix/signin-issue)
[x] You have only one commit (if not, squash them into one commit).
[x] All new and existing tests pass the command npm test. Use git commit --amend to amend any fixes.

Type of Change

[x] Small bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would change existing functionality)
[ ] Add new translation (feature adding new translations)

Checklist:

[x] Tested changes locally.
[x] Addressed currently open issue (replace XXXXX with an issue no in next line)

Closes #16581
Description
An unclosed <code> tag in the React challenge "React: Use Array.filter() to Dynamically Filter an Array" was disrupting the DOM (invariant throws the error Unable to find element with ID 646.). As a result, the entire page broke. I closed this tag in the seed file and have successfully tested the page locally.

Thanks @jkao1 👍


fix(widget-select): allow number value type

Summary
Fixes #4577
Allows numbers to be used as the value type in select widgets
Added an example in the dev-test environment

Test plan
yarn start
Open the kitchen sink collection and open the first entry
Scroll down to the select widgets (end of chapter 1)

Obligatory cute koala:

Thanks @Pearce-Ropion


Allow adding of directories to Firebase Storage.

There is no programmatic way to add directories to Firebase.

Worked out how to manage adding directories and reflecting it locally.
See: 807ff91


DigitalOcean: invalid key identifiers for Droplet creation

Bug
Environment

Platform:digital-ocean
OS: OSX
Terraform: v0.11.2
Plugins: provider.digitalocean v0.1.3

Problem
3 error(s) occurred:

* module.digital-ocean-dev.digitalocean_droplet.workers[0]: 1 error(s) occurred:

* digitalocean_droplet.workers.0: Error creating droplet: POST https://api.digitalocean.com/v2/droplets: 422 cd:e8:2a:10:50:ae:eb:6a:91:b9:e8:9f:e0:04:70:7f are invalid key identifiers for Droplet creation.
* module.digital-ocean-dev.digitalocean_droplet.workers[1]: 1 error(s) occurred:

* digitalocean_droplet.workers.1: Error creating droplet: POST https://api.digitalocean.com/v2/droplets: 422 cd:e8:2a:10:50:ae:eb:6a:91:b9:e8:9f:e0:04:70:7f are invalid key identifiers for Droplet creation.
* module.digital-ocean-dev.digitalocean_droplet.controllers: 1 error(s) occurred:

* digitalocean_droplet.controllers: Error creating droplet: POST https://api.digitalocean.com/v2/droplets: 422 cd:e8:2a:10:50:ae:eb:6a:91:b9:e8:9f:e0:04:70:7f are invalid key identifiers for Droplet creation.

When I'm trying to terraform apply fingerprints not recognized. I've tried sshkey ID, few diferent rsa keys
module.digital-ocean-dev.digitalocean_droplet.workers[0]: Creating...
  disk:                 "" => "<computed>"
  image:                "" => "coreos-stable"
  ipv4_address:         "" => "<computed>"
  ipv4_address_private: "" => "<computed>"
  ipv6:                 "" => "true"
  ipv6_address:         "" => "<computed>"
  ipv6_address_private: "" => "<computed>"
  locked:               "" => "<computed>"
  name:                 "" => "dev-worker-0"
  price_hourly:         "" => "<computed>"
  price_monthly:        "" => "<computed>"
  private_networking:   "" => "true"
  region:               "" => "lon1"
  resize_disk:          "" => "true"
  size:                 "" => "s-1vcpu-1gb"
  ssh_keys.#:           "" => "1"
  ssh_keys.0:           "" => "cd:e8:2a:10:50:ae:eb:6a:91:b9:e8:9f:e0:04:70:7f"
  status:               "" => "<computed>"
  tags.#:               "" => "1"
  tags.0:               "" => "dev-worker"

Steps to Reproduce
Fresh install. terraform init, terraform apply

Can use this snippet
FINGERPRINT="$(ssh-keygen -l -E md5 -f ~/.ssh/${DROPLET_NAME}.pub | awk '{print $2}'| sed 's|MD5:||')"

Full example of creating key and getting fingerprint and adding it:
DROPLET_TAG='Cool_Hostname'
DROPLET_NAME="${DROPLET_TAG/_/-}"
ssh-keygen -t rsa -f ~/.ssh/${DROPLET_NAME} -P "" -C "${DROPLET_NAME}"
FINGERPRINT="$(ssh-keygen -l -E md5 -f ~/.ssh/${DROPLET_NAME}.pub | awk '{print $2}'| sed 's|MD5:||')"
doctl compute ssh-key create ${DROPLET_NAME} --public-key "$(cat ~/.ssh/${DROPLET_NAME}.pub)"
doctl compute droplet create --image ubuntu-22-04-x64 --size s-8vcpu-16gb --region nyc1 ${DROPLET_NAME} --tag-name ${DROPLET_TAG} --enable-private-networking --ssh-keys ${FINGERPRINT} --wait ; doctl compute droplet get ${DROPLET_NAME} --format PublicIPv4 ;

Thanks for posting this, saved me lots of time. Only thing that seems wrong with this is the --size. When I used that size it said "This size is currently restricted, please open a ticket to verify your account." Probably the country I am in, so I used 's-1vcpu-1gb' instead and I was able to create and ssh in my droplet just fine 👍.


🔄 synced file(s) with neurobagel/workflows

synced local file(s) with neurobagel/workflows.

Changed files

synced local directory .github/workflows/ with remote directory template_workflows/project_automation/synced local directory .github/workflows/ with remote directory template_workflows/auto_release/created local  .autorc  from remote  template_configs/.autorccreated local  .github/pull_request_template.md  from remote  template_prs/auto_release_repo_pr.md

This PR was created automatically by the repo-file-sync-action workflow run #7536509328

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 99.314%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
579

Relevant Lines:
583

💛  - Coveralls


Limited documentation on the dotnet/mono framework

Issue description
While the dotnet framework is documented in the nixpkgs manual, it is currently not very thorough. The internals of the framework used for building dotnet applications (buildDotnetModule) are mostly undocumented right now.
Documentation for using mono is also completely missing.

It would be great to have some documentation, although I'm guessing there aren't a whole lot of people using dotnet on nixos. Here's a slide deck I found that has some info that might help you out though.
https://www.slideshare.net/sandervanderburg/vanderburg10-dotnetnix

Stumbled on this issue by accident (triage). By now we have in addition to mono a dotnet-sdk package.
I am interested in building and running dotnet-sdk on Linux, which seams to be a different use case then described in the mentioned slides. I have started working on a tool which could help building dotnet-sdk projects with NuGet dependencies. https://github.com/baracoder/dotnet2nix

I was interested in packaging the microsfot python LSP server but I hit the absence of doc too.

still important to me; the kind of documentation I'm looking for would cover what to do with errors such as MSB4242: The SDK resolver "Microsoft.DotNet.MSBuildSdkResolver" failed to run and You must add a reference to assembly netstandard, Version=<IP_ADDRESS>

I added some initial documentation here: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/dotnet.section.md

I added some initial documentation here: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/dotnet.section.md

contributions are welcome.

contributions are welcome.

I'm interested in packaging libraries, but there's little documentation or pre-existing code to reference, and .NET Framework (Mono runtime) isn't "first-class" alongside .NET Core. (Is there a better term for that?) I think .NET language support could use some direction at a high level.

I'm interested in packaging libraries, but there's little documentation or pre-existing code to reference, and .NET Framework (Mono runtime) isn't "first-class" alongside .NET Core. (Is there a better term for that?) I think .NET language support could use some direction at a high level.

Agreed, but in nixpkgs, it largely just takes one person with some domain expertise and will to implement something.

Here is some documentation for the new buildDotnetModule framework, in case that helps: https://nixos.org/manual/nixpkgs/unstable/#packaging-a-dotnet-application.
It's not very thorough just yet, but I'll add to it in the future.

How does this work with fetchNuget functionality?
Are there easy ways to convert already publish Nuget packages (such as global tools)?
Apologise ahead of time, I am not too familiar with Nix itself

How does this work with fetchNuget functionality?

It doesn't use fetchNuget right now, instead we create our own nix-based lockfile containing version information and hashes for nuget dependencies.

Are there easy ways to convert already publish Nuget packages (such as global tools)?

You could set the packNupkg attribute if you can compile it from source, but installing it as a dotnet tool might take a bit more effort. We currently don't have any setup in place for it, although I do plan to get on that in the future. Would you mind sharing what tool you need, and how you would like to use it? I'd be happy to test it while adding the functionality to install global tools to buildDotnetModule.

Apologise ahead of time, I am not too familiar with Nix itself

No need to apologise! Always feel free to ask any questions :blush:

Thank you so much for your reply!
I'd like to use dotnet-ef and I've already made a packaging request about it here. Personally, I have been testing on how to package the simplest CLI tool, dotnetsay.
I am fairly limited in my knowledge of the inner workings and best-practice of .NET landscape, but would it be possible to use makeWrapper and use ${pkgs.dotnet}/bin ${self}/lib/..../tool.dll where self is the fetchNuget already builds all dll. I assume this would make it platform and OS independent. What would be the caveats around this method?

I am fairly limited in my knowledge of the inner workings and best-practice of .NET landscape, but would it be possible to use makeWrapper and use ${pkgs.dotnet}/bin ${self}/lib/..../tool.dll where self is the fetchNuget already builds all dll. I assume this would make it platform and OS independent. What would be the caveats around this method?

That would use the cross platform executable, instead of a native one. Not a huge issue, but it means startup times will be a bit slower. It's also not building from source, but instead using a fetched DLL. Should work fine as a temporary workaround though :+1:


chore(Data/Fin): move Fin.equiv_iff_eq

The new proof is longer but I need this lemma before Fintype
for a future refactor, see #9794

bors merge


Settings experiment virtual cam

Settings experiment virtual cam not showing. Do I need to install a plugin from the web?

In the current version of VCam, the virtual Camera is no longer experimental.
You can install it by following the instructions in the document below:
https://docs.vcamapp.com/v/en/virtual-camera/virtualcamera

I have the 0.10.1. version.

Can you upload the screenshot?
If possible, please update the app. Older versions are no longer supported.

I can't update as I have 12.7.3 version of MacOS.

Thanks for the screenshot!
Sorry, this Experiment tab's feature is only available in the supporter version.
This is a text localization bug and there is not enough explanation about the gray-out in the English version.


Dim/select notifications caused by bots

Description
I receive lots of notifications caused by bots (dependabot in my case), and "real" notifications can get lost in the noise.
So here are two ideas:

It would be neat if those notifications could be dimmed (just like in the PR page).
It would also be nice to have a button which selects them all in order to quickly mark them as read.

Thanks for this extension by the way! ❤
Screenshot
Two real notifications followed by bot noise:

(picture several pages worth of notifications 😞)
Example URLs
https://github.com/notifications

I think something like this was asked before.

It would also be nice to have a button which selects them all in order to quickly mark them as read.

We could add an item in select-notifications.
Alternatively you could also disable Dependabot notifications in your settings.

I think the most common approach here would be to:

Select unread notifications by bots
Mark them as done permanently (aka. unsubscribe)

Can this be turned into a single, very specific button?

Our alternatives:

add a filter to select-notifications, but that's 3 clicks, might as well save the author:dependabot filter, click "Select all" and then "done"
add a permanent "I don't care about dependabot" option that automatically does this on sight, but that would go against https://github.com/refined-github/refined-github/issues/2960#issuecomment-606765382

Can this be turned into a single, very specific button?

Sure, I like that approach. Might as well save a few clicks. 🙂
I'm just not sure about unsubscribe vs mark as read. Someone may comment on a bot PR and the notification would be lost. This never happens in my case but it may be an issue for others.

add a permanent "I don't care about dependabot" option

That feels like a nuclear option indeed, I wouldn't turn that on by default.

This could be improved/generalized by turning it into a slightly-more-general "ignore  notifications" feature.
I ask, because we have an internal GitHub Enterprise bot-user with very similar behaviour — dozens of annoying notifications filling up my GitHub Notifications screen; and I'd love to be able to use RGH to dim those out and help me stay focused.
… similarly, it'd be nice to be able to filter/ignore a certain user in the 'conversation view' of the PRs itself … although that probably belongs in a separate Issue / feature-request.

improved/generalized by turning it into a slightly-more-general "ignore notifications" feature

it'd be nice to be able to filter/ignore a certain user

Can't do either, refer to:

https://github.com/refined-github/refined-github/issues/2672


Add address styles

This should close #44. Adds styles for addresses and mailto/tel links. Things to consider:

I'm currently using the :email: emoji, but it displays incorrectly in some fonts. I prefer it over :e-mail: but the latter seems to have more support.
Are there other protocols besides tel and mailto we should think about?

I'm currently using the ✉ emoji, but it displays incorrectly in some fonts. I prefer it over 📧 but the latter seems to have more support.

I wonder if there's such things as emoji semantics too? Haha.
I think either is fine really. The ✉ symbol is a general envelope, while the 📧 symbol is explicitly an email symbol. It's all aesthetics though, I'm not sure that it matters.
Do you have an example/screenshot of when the ✉ symbol looks bad in one of the fonts in the Water.css stack?

Are there other protocols besides tel and mailto we should think about?

I'm not seeing any others after a quick search.

According to CSS-Tricks there also is sms:, callto: and fax:, but with spotty browser support so we can probably ignore it.
The only other thing we could consider is adding icons for download links, e.g. a document icon for a[download][href$=".pdf"].

I added sms: styles. Any last things or can we merge?

:shipit:


Tooling to determine the retain path(s) of a symbol

In Flutter's hello_world example, run the following:
flutter build apk --target-platform android-arm64 --release --analyze-size --code-size-directory ~/tmp/hw1
snapshot_analysis treemap ~/tmp/hw1/snapshot.arm64-v8a.json ~/tmp/hw1/treemap

Generates a very helpful treemap. However, if a symbol is in there that seems like it should not be, it'd be very helpful to know why. Right now you can kind of figure it out if you're super familiar with Flutter framework/engine and Dart, but even then it can be difficult.
/cc @rmacnak-google @mraleph

I had implemented some prototype CLI for this before but then I lost the prototype :-/ I can look at reimplementing it again

Ryan was saying "I'm about 70% sure Slava has something for this" :D


Readonly fields

Typescript allows to declare fields as readonly. Is there any workaround to add readonly property to interface infer-ed from zod schema?

OK, solved it:
import * as z from "zod";
const MySchema = z.object({
  f1: z.string(),
  f2: z.number()
});
type MyType = Readonly<z.infer<typeof MySchema>>;


Update the color syntax for the doc

closing for now. @clawfire I'd happily take this patch if you want to update the commented code (# -> //) as described above. thanks!


[e2e]: go release main config-ldflags-noassets slsa3

Repo: https://github.com/slsa-framework/example-package/tree/v17.0.76
Run: https://github.com/slsa-framework/example-package/actions/runs/2908647177
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.release.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.release.main.config-ldflags-noassets.slsa3.yml
Trigger: release
Branch: v17.0.76
Date: Tue Aug 23 03:36:10 UTC 2022

Repo: https://github.com/slsa-framework/example-package/tree/v17.0.77
Run: https://github.com/slsa-framework/example-package/actions/runs/2916268587
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.release.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.release.main.config-ldflags-noassets.slsa3.yml
Trigger: release
Branch: v17.0.77
Date: Wed Aug 24 03:39:05 UTC 2022

Repo: https://github.com/slsa-framework/example-package/tree/v17.0.78
Run: https://github.com/slsa-framework/example-package/actions/runs/2923805359
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.release.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.release.main.config-ldflags-noassets.slsa3.yml
Trigger: release
Branch: v17.0.78
Date: Thu Aug 25 03:46:20 UTC 2022

Repo: https://github.com/slsa-framework/example-package/tree/v17.0.79
Run: https://github.com/slsa-framework/example-package/actions/runs/2931117098
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.release.main.config-ldflags-noassets.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.release.main.config-ldflags-noassets.slsa3.yml
Trigger: release
Branch: v17.0.79
Date: Fri Aug 26 03:46:16 UTC 2022
Tests are passing now. Closing this issue.


Communication: Different Types of Reactions

Describe the bug

Is it intentional that one can both upvote and downvote a message?
To Reproduce

Go to any test server (I used TS3)
Click on any course with Communication
React to a message

Expected behavior
I don't know if this is a feature of Artemis for one to both upvote and downvote at the same time but one of two sounds more reasonable for me.
Screenshots
No response
Which version of Artemis are you seeing the problem on?
7.6.1
What browsers are you seeing the problem on?
Microsoft Edge
Additional context
No response
Relevant log output
No response

This is not really a upvote or downvote feature but rather reacting with emojis to a message and as one can react with multiple different emojis, you can "upvote" and "downvote" at the same time.

Ah I see, thank you!


build: fix on windows 10

Connects to #141

Coverage remained the same at 97.826% when pulling 7080d5290cbb7d00f0d1330c090972fe11e8f5f0 on helio-frota:pr into e0dae63ea49b4ef781ff5c60b1519e17f59261f4 on bucharest-gold:master.

🎆  landed 5669222


Not rendering in reading view

Hello, maybe it is my workstation, but the blindfold isn't rendering in reading view, only in live preview like into the screenshot
Reading view :

Live Preview :

Issue confirmed. That is not your workstation. I will fix this soon.

Thank's for this, good luck, and great work as well


Using nohalo/lohalo for downscaling

What are you trying to achieve?
NoHalo downscaler kernel
Hello. Searching issues leads me to believe this used to be supported, but I can't find the exact moment it was dropped or the rationale for it?

HI, "nohalo" is considered an interpolator rather than a kernel in libvips. The vips_resize function internally switched from using vips_affine and interpolators to vips_reduce and kernels from libvips v8.3.0 (Jan 2016) onwards.
Custom interpolators such as "nohalo" are still available via vips_affine, but this is not exposed in sharp - see #2095 for the future possible enhancement to do so.

Thank you. I believe that closes this issue, since the question is "what happened to nohalo."
Have a great day.


Create Binder.java

 Quality gate: Failed
7 Issues

 3 Vulnerabilities
 1 Code issues
 3 Anti-patterns

Details

 Quality gate: Failed
7 Issues

 3 Vulnerabilities
 1 Code issues
 3 Anti-patterns

Details

 Quality gate: Failed
7 Issues

 3 Vulnerabilities
 1 Code issues
 3 Anti-patterns

Details


crossbeam-channel: Remove dependency on smallvec

This doesn't seem to affect benchmarks.

What benchmarks did you try?

 I should mention that I have noticed regressions when using musl, but that has to do with the fact that the malloc implementation in musl is very primitive.
Feel free to close the PR if there are machines on which this does regress the benchmarks.

Interesting. Could you perhaps also say what is your machine (CPU) and rustc version?

$ rustc --version
rustc 1.35.0-nightly (2a65cbeea 2019-03-08)

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       39 bits physical, 48 bits virtual
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               142
Model name:          Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
Stepping:            10
CPU MHz:             770.694
CPU max MHz:         3400.0000
CPU min MHz:         400.0000
BogoMIPS:            3601.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            6144K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp

Here are the benchmarks on my system (specs below). I ran each three times. There's a bit of variance here, just from eye-balling it.

I specifically ran the above on one of my idle machines too. Here is the CPU info:
$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       39 bits physical, 48 bits virtual
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i5-7600 CPU @ 3.50GHz
Stepping:            9
CPU MHz:             800.327
CPU max MHz:         4100.0000
CPU min MHz:         800.0000
BogoMIPS:            7010.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            6144K
NUMA node0 CPU(s):   0-3
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp

Okay, I think we can merge this PR and later optimize the select macro so that it avoids the allocation in Vec (this is possible because we statically know how many cases there are inside an invocation of select!).
Thanks!

bors r+


TODO in PendingSubscriptionSink

https://github.com/ChainSafe/forest/commit/363aac16433c43b3c1abf21417061ed755b753bc#diff-78e932c0563f39b9d22b2cb2cc901ceeac49b68952476be94bb17e40346b9203R127

Introduced in #3978

I think our plan is to get rid of the PendingSubscriptionSink: https://github.com/ChainSafe/forest/issues/4000

This is some TODO comment originating from the jsonrpsee crate.

So, should the comment be removed and closed? Otherwise, let's have a meaningful description on what is to be done.

We can close it.


(v1.3.1)Data change not be saved to MongoDB when using ui-grid!

After I moved my code from v1.2.3 to v1.3.1 , I always can not make my ui-grid work correctly. Here is my code:
angular
    .module('Aries')
    .controller('TPSCtrl', TPSCtrl);
TPSCtrl.$inject = ['$scope', '$reactive'];
function TPSCtrl($scope, $reactive){
    $reactive(this).attach($scope);
    $scope.subscribe('termparams');
    $scope.helpers({
        tps: function()  {
            return TermParams.find({});
        }
    });
    $scope.gridTPS = {
        columnDefs: [
            { name:'name', field: 'name', displayName:'Name', enableCellEdit:false },
            { name:'tag', field: 'tag', width:100, enableCellEdit:false },
            { name:'len', field: 'len',  displayName:'Length', width:100, enableCellEdit:false },
            { name:'val', field: 'val', displayName:'Value', enableCellEdit:true}
        ]
    };
    $scope.gridTPS.data = $scope.tps;
}

When I edit the value in the grid, change cannot be saved to MongoDB.

First thing is that if you are in a regular $scope syntax, you don't need to add $reactive(this).attach($scope); but I'm not sure if that's your problem.
Can you please share a full repo I can play with?

@Urigo ,I create a demo repo, https://github.com/michaelszt/demo
In file "client/term/controllers/termParams.js", the current code can not work correctlly, and the commented code below is my original code based on v1.2.3, which can save data change to MongoDB after edit ui-grid.

+1 same problem

Hi @michaelszt, do you mean the you used the auto-bind feature in 1.2.x? because it does not available in 1.3...

@dotansimha ,thank you for your help, do you know how to use ui-grid in 1.3.1?

@Urigo , do you have any solution for using ui-grid in 1.3.1?

@michaelszt , you need to create a server side method that will update your data in the MongoDB.
Start by using Meteor.methods in the server side, and the call your function using Meteor.call in the client side.
You can use ui-grid's API to know when the user changes the data, I think this feature called afterCellEdit.
Please re-open if you have any further questions.


partial_fit behavior for Facility Location

I am using the library with great results, but I am struggling to understand if the behavior reported below is correct.
I am loading batches of data from the disk and then calling partial_fit on every batch. Let's assume that each batch contains 1000 data points, and I load 10 batches, the data points will be indexed from 0 to 9999.
After each partial fit call, I store the indices returned in the ranking and the associated data points (e.g., [1, 2,.... 100]).
I would expect the list of the indices returned by partial_fit at batch B should be either from the list of the stored indices or from the indices of the batch B but that does not seem to be the case when I use facility location (empirically it is the case for the feature based selection).
For example, if my stored indices are pi = [1,2,5,90] and the current batch contains the points with in bi = [100, 101, ..., 110], I would expect that given ci = selector.ranking to have ci in pi + bi. Is this a wrong assumption?
If my assumption is wrong, how do you retrieve the points associated to the selected indices without a second pass on the data?

The stored indices should be relative to the beginning of the stream. If you selected the 1000th example, regardless of the number of batches, you should have index 1000 stored (or I guess 999 since it's 0-based). See this code here that keeps a running total of the number of examples you've seen: https://github.com/jmschrei/apricot/blob/master/apricot/optimizers.py#L1101
If you just want the examples and not necessarily the indexes you can use the subset attribute (see https://github.com/jmschrei/apricot/blob/master/apricot/optimizers.py#L1122). The way that the sieve algorithm (behind the partial fit method) works is that it makes several guesses as to how good the best example will be and, using math, can turn that into an efficient one-pass algorithm. The one-pass algorithm is then run in parallel for each guess and at the end of each batch the guess closest to the truth has their subset returned.

subset seems to be exactly what I need. Thanks for the reference.
Anyway, what I am observing right now is that when calling partial_fit on a batch of points in the range (i,j) the ranking may contain indices which are smaller than i and which were not present in the previously stored list.

I can't be positive this is what's happening without seeing the example, but I'm almost certain it's related to the procedure I described above. Like I said, the streaming algorithm maintains several parallel subsets that were selected based on different quality estimates. After each partial_fit call, you're given the subset that had the closest estimate given the entire stream so far. From one partial_fit call to the next this can change, and a different subset has the best estimate, meaning the entire returned subset is changed. Unfortunately, it's difficult to get good estimates over the entire stream if your only choices are reject the next example or use it to replace an already selected example within a single subset.

I don't think this is a particularly complicated matter, it would be enough to provide a method retrieving the indices the points in the union over all subsets.

The sieve_selections_ attribute on the selector should contain all the indices. https://github.com/jmschrei/apricot/blob/master/apricot/optimizers.py#L1113

Terrific, thanks!

It works perfectly, thanks a lot for your help and all the work you put in this library.


show language name in it's language

closes https://github.com/aldryn/aldryn-boilerplate-bootstrap3/issues/148

:+1:

LGTM