moved tags related paths into one file so operations prefixes is grouped by files

This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.
PR information

[* ] The title of the PR is clear and informative.
[* ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[* ] Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
[* ] If applicable, the PR references the bug/issue that it fixes.
[* ] Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

[* ] I have read the contribution guidelines.
[* ] My spec meets the review criteria:

[* ] The spec conforms to the Swagger 2.0 specification.
[* ] The spec follows the guidelines described in the Swagger checklist document.
[* ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.

Automation for azure-sdk-for-python

Automation for azure-sdk-for-go

@solankisamir , could you please review this too?
Also, I deleted  apimapisbytags.json, do we need to remove any reference to it in the documentation menu?
Thank you,
Vitaliy.

@dsgouda, there are no errors but warnings only, Which one should we fix? PutRequestResponseScheme? It's not possible, since put method receives existing apiId and tagId to make an association between them, but GET method returns all other properties of the API and TAG. Anyway, the result of GET operation can be used as input for PUT, those properties will be just ignored.


bug(chip-list): Input selectable not working.

Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
idk
Description
When setting selectable to mat-chip-list, you can still select mat-chip. The selectable attr on mat-chip itself does also not work.

Reproduction
See demo.
Expected Behavior
When <mat-chip-list [selectable]="false"><mat-chip></mat-chip></mat-chip-list> then mat-chip should not be selectable.
Actual Behavior
You can still select.
Environment

Angular: v14.2.11
CDK/Material: v14.2.7
Browser(s): Chrome  07.0.5304.107
Operating System; Windows 10

Some background: The annoying thing is, that when you have this in an input and click on the form-field, it always selects the first chip. It's a very bad UX and makes it impossible to scroll through chips. Because it always scrolls back to the first pos. Anyway, selectable should work.

Your gif is showing the focus indication, not the selection indication which is a colored background.

Ok, then feature request for an option to disable focusable chips?

Focus indication is required for accessibility and we generally don't put in options to disable accessibility features. You can disable it in your own app with the following style override:
.mat-mdc-chip-focus-overlay {
  display: none;
}


MoE Expert Parallel Impl

Motivation
This is the implementation of MoE Expert Parallel, seamlessly integrated with TP. The expert parallel size (ep-size) matches the tensor parallel size (tp-size).
It supports the following formats:

[ -] FP16/BF16
[ -] FP8:Dynamic/Static

I used CUDA/CUTLASS to implement these operators. However, integrating CUDA/CUTLASS into SGLang is difficult. Therefore, I rewrote the implementation using Triton. Overall, the Triton implementation performs well, except for Grouped-GEMM, where its performance is significantly lower than the CUTLASS implementation (several times slower on H100).
Modifications

Add MoE EP Layer and kernels.
Add unit test for moe-ep
Add server args: --enable-ep-moe. If want to use moe-ep, just add --enable-ep-moe

Performance
Note: EP is not yet at its optimal performance. The final performance of EP parallelism will better than TP.

 Moving forward, further performance optimization for Grouped-GEMM is required.
I plan to integrate the related optimizations for Grouped-GEMM (CUTLASS) into FlashInfer, enabling SGLang to access high-performance Grouped-GEMM implementations through FlashInfer. Additionally, further optimization of the Triton Grouped-GEMM implementation is necessary.
Update: 11/27
I do a fine-grained performance analysis of MoE-EP on neuralmagic/DeepSeek-Coder-V2-Instruct-FP8
One layer performance results
Prefill phase
The table below shows the time of each module in a specific layer during the Prefill phase, measured in milliseconds (ms).

It can be observed that the MoE-EP kernel has achieved approximately a 3x performance improvement compared to TP. However, since the MoE component constitutes only a small portion (with Attention being the major contributor), the overall model performance improvement in testing is limited.
Base(TP) = 13.8 ms
ThisPR(TP+EP) = 11.2ms
13.8/11.2 = 1.23
26039.52/21004.67 = 1.24
It can be seen that these two numbers are essentially consistent.

Base(TP)-Nsys

ThisPR(TP+EP)-Nsys

Decode phase
The decode phase was also analyzed.

Base(TP) = 4.3ms
ThisPR(TP+EP) = 4.3ms(Approximately)
From the kernel perspective, there is also an improvement in the MoE section during the decode phase. However, due to  MoE expert load balancing, the computation time varies across GPUs (with the worst-case scenario pulling down the lower bound). As a result, the time difference is minimal, which aligns with the E2E test results.
Additionally, even if the load is well-balanced, the MoE component still makes up a small portion of the model (similar to the prefill phase, where Attention dominates). Even with further optimization of MoE, the overall improvement would still be minimal.

Base(TP)-Nsys

ThisPR(TP+EP)-Nsys

In summary, the current MoE-EP kernel shows a 1-3x improvement over MoE-TP. However, since the MoE component makes up a small portion of DeepSeek-V2, the overall improvement in the E2E test is not significant. (Of course, the MoE-EP Triton implementation can still be further improved.)
Additionally, based on the nsys results, the current MLA implementation can still be further improved. I will submit my implementation for it in the future.
Checklist

[ -] Format your code according to the Contributor Guide.
[ -] Add unit tests as outlined in the Contributor Guide.
[ ] Update documentation as needed, including docstrings or example tutorials.

It's amazing! Please join our slack channel https://join.slack.com/t/sgl-fru7574/shared_invite/zt-2rtikx2pv-DUfPrhx2SaNAq~47YtV1XQ

I used CUDA/CUTLASS to implement these operators. However, integrating CUDA/CUTLASS into SGLang is difficult. Therefore, I rewrote the implementation using Triton.

We currently have a kernel library, and I can add you to the team for collaboration.

It's amazing! Please join our slack channel https://join.slack.com/t/sgl-fru7574/shared_invite/zt-2rtikx2pv-DUfPrhx2SaNAq~47YtV1XQ

I used CUDA/CUTLASS to implement these operators. However, integrating CUDA/CUTLASS into SGLang is difficult. Therefore, I rewrote the implementation using Triton.

We currently have a kernel library, and I can add you to the team for collaboration.

I'm in the slack channel

Not sure why one of the unit tests failed. It seems unrelated to this pr.

I have updated the latest performance analysis results.  See Update: 11/27.   @zhyncs @ispobock

@xiaobochen123 Does this rely on dp-attention? Also, from what I understand, ep shouldn't add any extra memory burden, right?

@xiaobochen123 May you highlight how TP is used along with EP within MoE layer here?
Btw, do & how we count in the traffic imbalance criteria into benchmarking? Thanks!

Amazing works, @xiaobochen123 This is super dope!

Does this rely on dp-attention? Also, from what I understand, ep shouldn't add any extra memory burden, right?

@fengyang95 1. Does not rely on DP-attention.   2. Does not add extra memory.

@xiaobochen123 May you highlight how TP is used along with EP within MoE layer here? Btw, do & how we count in the traffic imbalance criteria into benchmarking? Thanks!

@HaiShaw When enables EP, the MoE module is EP, with no TP (EP-Size = TP-Size). The Attention is TP.  Regarding the imbalance issue, you can perform performance analysis using nsys. If the MoE execution time differs across GPUs, you can get it. For more precise analysis, you can collect the top-k information for each layer.

@xiaobochen123 Does this rely on dp-attention? Also, from what I understand, ep shouldn't add any extra memory burden, right?

@fengyang95  1. Does not rely on DP-attention. 2. Does not add extra memory.

@xiaobochen123 May you highlight how TP is used along with EP within MoE layer here? Btw, do & how we count in the traffic imbalance criteria into benchmarking? Thanks!

@HaiShaw When enables EP, the MoE module is EP, with no TP (EP-Size = TP-Size). The Attention is TP. Regarding the imbalance issue, you can perform performance analysis using nsys. If the MoE execution time differs across GPUs, you can get it. For more precise analysis, you can collect the top-k information for each layer.

Can we follow similar/closer naming (variable, func, kernel) conventions from existing fused_moe.py?

@xiaobochen123 , very nice work, and look forward to more contributions from you! Several general comments:

Would you please add some more comments, to new function and/or kernel for example?
Can we follow similar/closer naming (variable, func, kernel) conventions from existing fused_moe.py?
What is the assumption we have from quantization (fp8, etc.) to EP support, e.g. should TP quantized model work seamlessly with EP?
--tp 8 --enable-ep-moe seemed to be confusion, as TP/EP doesn't coexist, possible to introduce --ep <NUM> or similar kind?

similar/closer naming with fused_moe.py, could you tell me more about what need fix.
I think that TP quantized models are seamless to EP. Because when TP quantizes, it calculates a scale according to each expert, which does not affect EP.
I have add —ep-size args。

@xiaobochen123 , very nice work, and look forward to more contributions from you! Several general comments:

Would you please add some more comments, to new function and/or kernel for example?
Can we follow similar/closer naming (variable, func, kernel) conventions from existing fused_moe.py?
What is the assumption we have from quantization (fp8, etc.) to EP support, e.g. should TP quantized model work seamlessly with EP?
--tp 8 --enable-ep-moe seemed to be confusion, as TP/EP doesn't coexist, possible to introduce --ep <NUM> or similar kind?

similar/closer naming with fused_moe.py, could you tell me more about what need fix.
I think that TP quantized models are seamless to EP. Because when TP quantizes, it calculates a scale according to each expert, which does not affect EP.
I have add —ep-size args。
@HaiShaw

so active_scale is a scalar right? weight_scale is a vector for sure, not both though.

@HaiShaw In my implementation, both active_scale and weight_scale are vectors. The values in the vectors can all be the same.

so active_scale is a scalar right? weight_scale is a vector for sure, not both though.

@HaiShaw In my implementation, both active_scale and weight_scale are vectors. The values in the vectors can all be the same.

That is what I see from code, the related question is - then we can't load/reuse the quantized model from TP model (where activation_scale is a scalar not the vector), so do you use a differently quantized model? can we use the same quantized model, s.t. activation_scale doesn't need to be different per expert (or do we know that if we use one scale for activation to different experts, we will have bad accuracy)? Thanks!

so active_scale is a scalar right? weight_scale is a vector for sure, not both though.

@HaiShaw In my implementation, both active_scale and weight_scale are vectors. The values in the vectors can all be the same.

That is what I see from code, the related question is - then we can't load/reuse the quantized model from TP mode (where activation_scale is a scalar not the vector), so do you use a differently quantized model? can we use the same quantized model, s.t. activation_scale doesn't need to be different per expert (or do we know that if we use one scale for activation to different experts, we will have bad accuracy)? Thanks!
If the quantized model for EP is different, then can we have the quantization scripts/tools (or reference to it) included in this PR?

I am using the neuralmagic/DeepSeek-Coder-V2-Instruct-FP8 model, as well as my own DeepSeek-V2 model quantized with AutoFP8. In these two FP8 models, the activation_scale in the weights is a vector, with each expert having its own independent activation_scale. Additionally, having a separate activation_scale for each expert is likely to provide better precision compared to all experts sharing the same activation_scale.    Additionally, when TP loads the activation_scale vector, it likely takes the maximum value among them as the scale

@xiaobochen123 nice update - can you give a try to below model too?
amd/Mixtral-8x7B-Instruct-v0.1-FP8-KV

to merge, thanks.

@xiaobochen123 nice update - can you give a try to below model too? amd/Mixtral-8x7B-Instruct-v0.1-FP8-KV
Accuracy result, you may ignore for now, just check if it runs to completion.

@HaiShaw I tested the model, and it`s ok

@xiaobochen123 nice update - can you give a try to below model too? amd/Mixtral-8x7B-Instruct-v0.1-FP8-KV
Accuracy result, you may ignore for now, just check if it runs to completion.

@xiaobochen123 I tested the model, and it`s ok

@xiaobochen123 May you update branch update, prepare for merge?

@xiaobochen123 May you update branch, prepare for merge?

@HaiShaw Updated

@xiaobochen123 can you fix this CI error:
  File "/actions-runner/_work/sglang/sglang/python/sglang/srt/models/mixtral.py", line 82, in __init__
    MoEImpl = EPMoE if global_server_args_dict["enable_ep_moe"] else FusedMoE
NameError: name 'global_server_args_dict' is not defined

This PR is great, thanks @xiaobochen123!
I am running a benchmark with your code without enabling DP attention, and I consistently see lower performance for EP than TP.
Here are the benchmarking scripts:
model=/ssd/models/deepseek-ai/DeepSeek-Coder-V2-Instruct

# TP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8

# EP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8 \
    --ep-size 8 \
    --enable-ep-moe

The output token throughputs on DGX-A100 are:
EP: 904.29 tok/s
TP: 1027.46 tok/s
Does this align with your expectation given your above benchmarks? Thanks!
Minor comment: although the --ep-size argument exists, it currently requires --tp-size to be set. It might be better to not require --tp-size if EP is used.

This PR is great, thanks @xiaobochen123!
I am running a benchmark with your code without enabling DP attention, and I consistently see lower performance for EP than TP.
Here are the benchmarking scripts:
model=/ssd/models/deepseek-ai/DeepSeek-Coder-V2-Instruct

# TP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8

# EP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8 \
    --ep-size 8 \
    --enable-ep-moe

The output token throughputs on DGX-A100 are: TP: 1027.46 tok/s EP: 904.29 tok/s
Does this align with your expectation given your above benchmarks? Thanks!
Minor comment: although the --ep-size argument exists, it currently requires --tp-size to be set. It might be better to not require --tp-size if EP is used or maybe specify attn vs. ffn in the args?

@Mutinifni Based on my tests, the output is slightly worse than TP.  While ep-size is enabled, tp-size is still required because attention relies on TP.  I added ep-size based on dp-size, and their behaviors should be quite similar.

This PR is great, thanks @xiaobochen123!
I am running a benchmark with your code without enabling DP attention, and I consistently see lower performance for EP than TP.
Here are the benchmarking scripts:
model=/ssd/models/deepseek-ai/DeepSeek-Coder-V2-Instruct

# TP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8

# EP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8 \
    --ep-size 8 \
    --enable-ep-moe

The output token throughputs on DGX-A100 are: TP: 1027.46 tok/s EP: 904.29 tok/s
Does this align with your expectation given your above benchmarks? Thanks!
Minor comment: although the --ep-size argument exists, it currently requires --tp-size to be set. It might be better to not require --tp-size if EP is used or maybe specify attn vs. ffn in the args?

@Mutinifni Based on my tests, the output is slightly worse than TP. While ep-size is enabled, tp-size is still required because attention relies on TP. I added ep-size based on dp-size, and their behaviors should be quite similar.

If set tp-size = 2, ep-size = 2, --enable-ep-moe, is it only ep or hybrid parallel now?

This PR is great, thanks @xiaobochen123!
I am running a benchmark with your code without enabling DP attention, and I consistently see lower performance for EP than TP.
Here are the benchmarking scripts:
model=/ssd/models/deepseek-ai/DeepSeek-Coder-V2-Instruct

# TP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8

# EP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8 \
    --ep-size 8 \
    --enable-ep-moe

The output token throughputs on DGX-A100 are: TP: 1027.46 tok/s EP: 904.29 tok/s
Does this align with your expectation given your above benchmarks? Thanks!
Minor comment: although the --ep-size argument exists, it currently requires --tp-size to be set. It might be better to not require --tp-size if EP is used or maybe specify attn vs. ffn in the args?

@Mutinifni Based on my tests, the output is slightly worse than TP. While ep-size is enabled, tp-size is still required because attention relies on TP. I added ep-size based on dp-size, and their behaviors should be quite similar.

If set tp-size = 2, ep-size = 2, --enable-ep-moe, is it only ep or hybrid parallel now?

@sitabulaixizawaluduo It`s hybrid parallel. MoE-EP, Attention-TP.

This PR is great, thanks @xiaobochen123!
I am running a benchmark with your code without enabling DP attention, and I consistently see lower performance for EP than TP.
Here are the benchmarking scripts:
model=/ssd/models/deepseek-ai/DeepSeek-Coder-V2-Instruct

# TP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8

# EP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8 \
    --ep-size 8 \
    --enable-ep-moe

The output token throughputs on DGX-A100 are: TP: 1027.46 tok/s EP: 904.29 tok/s
Does this align with your expectation given your above benchmarks? Thanks!
Minor comment: although the --ep-size argument exists, it currently requires --tp-size to be set. It might be better to not require --tp-size if EP is used or maybe specify attn vs. ffn in the args?

@Mutinifni Based on my tests, the output is slightly worse than TP. While ep-size is enabled, tp-size is still required because attention relies on TP. I added ep-size based on dp-size, and their behaviors should be quite similar.

If set tp-size = 2, ep-size = 2, --enable-ep-moe, is it only ep or hybrid parallel now?

@sitabulaixizawaluduo It`s hybrid parallel. MoE-EP, Attention-TP.

Thanks for reply,  I still have a question, is the calculation speed of Only EP necessarily faster than that of TP + EP in MLP calculation?

This PR is great, thanks @xiaobochen123!
I am running a benchmark with your code without enabling DP attention, and I consistently see lower performance for EP than TP.
Here are the benchmarking scripts:
model=/ssd/models/deepseek-ai/DeepSeek-Coder-V2-Instruct

# TP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8

# EP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8 \
    --ep-size 8 \
    --enable-ep-moe

The output token throughputs on DGX-A100 are: TP: 1027.46 tok/s EP: 904.29 tok/s
Does this align with your expectation given your above benchmarks? Thanks!
Minor comment: although the --ep-size argument exists, it currently requires --tp-size to be set. It might be better to not require --tp-size if EP is used or maybe specify attn vs. ffn in the args?

@Mutinifni Based on my tests, the output is slightly worse than TP. While ep-size is enabled, tp-size is still required because attention relies on TP. I added ep-size based on dp-size, and their behaviors should be quite similar.

If set tp-size = 2, ep-size = 2, --enable-ep-moe, is it only ep or hybrid parallel now?

@sitabulaixizawaluduo It`s hybrid parallel. MoE-EP, Attention-TP.

Thanks for reply, I still have a question, is the calculation speed of Only EP necessarily faster than that of TP + EP in MLP calculation?

@sitabulaixizawaluduo Sorry, I might not have fully understood your point. Why would TP + EP be used in an MLP? From my understanding, an MLP uses either TP or EP.

This PR is great, thanks @xiaobochen123!
I am running a benchmark with your code without enabling DP attention, and I consistently see lower performance for EP than TP.
Here are the benchmarking scripts:
model=/ssd/models/deepseek-ai/DeepSeek-Coder-V2-Instruct

# TP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8

# EP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8 \
    --ep-size 8 \
    --enable-ep-moe

The output token throughputs on DGX-A100 are: TP: 1027.46 tok/s EP: 904.29 tok/s
Does this align with your expectation given your above benchmarks? Thanks!
Minor comment: although the --ep-size argument exists, it currently requires --tp-size to be set. It might be better to not require --tp-size if EP is used or maybe specify attn vs. ffn in the args?

@Mutinifni Based on my tests, the output is slightly worse than TP. While ep-size is enabled, tp-size is still required because attention relies on TP. I added ep-size based on dp-size, and their behaviors should be quite similar.

If set tp-size = 2, ep-size = 2, --enable-ep-moe, is it only ep or hybrid parallel now?

@sitabulaixizawaluduo It`s hybrid parallel. MoE-EP, Attention-TP.

Thanks for reply, I still have a question, is the calculation speed of Only EP necessarily faster than that of TP + EP in MLP calculation?

@sitabulaixizawaluduo Sorry, I might not have fully understood your point. Why would TP + EP be used in an MLP? From my understanding, an MLP uses either TP or EP.

For example, a model with 8 experts performs four-card calculations, tp_size = 2, ep_size = 2, in the MLP layer, GPU0 and GPU1, using 4 experts, this 4 experts do parallel computing with tp

This PR is great, thanks @xiaobochen123!
I am running a benchmark with your code without enabling DP attention, and I consistently see lower performance for EP than TP.
Here are the benchmarking scripts:
model=/ssd/models/deepseek-ai/DeepSeek-Coder-V2-Instruct

# TP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8

# EP
python -m sglang.bench_offline_throughput \
    --model-path $model \
    --num-prompts 1000 \
    --trust-remote-code \
    --tp-size 8 \
    --ep-size 8 \
    --enable-ep-moe

The output token throughputs on DGX-A100 are: TP: 1027.46 tok/s EP: 904.29 tok/s
Does this align with your expectation given your above benchmarks? Thanks!
Minor comment: although the --ep-size argument exists, it currently requires --tp-size to be set. It might be better to not require --tp-size if EP is used or maybe specify attn vs. ffn in the args?

@Mutinifni Based on my tests, the output is slightly worse than TP. While ep-size is enabled, tp-size is still required because attention relies on TP. I added ep-size based on dp-size, and their behaviors should be quite similar.

If set tp-size = 2, ep-size = 2, --enable-ep-moe, is it only ep or hybrid parallel now?

@sitabulaixizawaluduo It`s hybrid parallel. MoE-EP, Attention-TP.

Thanks for reply, I still have a question, is the calculation speed of Only EP necessarily faster than that of TP + EP in MLP calculation?

@sitabulaixizawaluduo Sorry, I might not have fully understood your point. Why would TP + EP be used in an MLP? From my understanding, an MLP uses either TP or EP.

For example, a model with 8 experts performs four-card calculations, tp_size = 2, ep_size = 2, in the MLP layer, GPU0 and GPU1, using 4 experts, this 4 experts do parallel computing with tp

@sitabulaixizawaluduo TP requires one communication, and EP adds another. Could this result in degraded performance?

@sitabulaixizawaluduo From my understanding, the overall computation should be the same for TP=4, EP=4, and TP=2/EP=2. It’s unlikely that TP=2/EP=2 would have less computation. In my current implementation, MoE-EP uses AllReduce, and the communication cost is the same as TP.

Thanks! I will recalculate the amount of forward communication and computation for this part. By the way, when I tested the response time of a single request( input length 1000 - 2000) on the L40 (PCIE), I found that the TP speed is faster than that of the EP. When TP is deployed: --tp-size = 2, when EP is deployed: --tp-size = 2 --enable-ep-moe, what is the reason for the performance gap between the two?

@sitabulaixizawaluduo Sorry, I don’t have L40, so I’m unable to perform specific tests or analyze the performance directly. However, there might be a few possible reasons: the current MoE-EP implementation in Triton do not have been fully tuned or configured with the best settings; additionally, Expert Parallel might face load balancing issues when concurrency is low.

@sitabulaixizawaluduo Sorry, I don’t have L40, so I’m unable to perform specific tests or analyze the performance directly. However, there might be a few possible reasons: the current MoE-EP implementation in Triton do not have been fully tuned or configured with the best settings; additionally, Expert Parallel might face load balancing issues when concurrency is low.

Thanks!


The last link doesn't get active on the documentation page

Describe the Bug
On the documentation page, the last link on the right side isn't active even after clicking the link. Looks like it only gets active if the scrollbar reaches the top of that section.

gets
Which package is affected (leave empty if unsure)
No response
Link to the code that reproduces this issue
https://react.email/docs/introduction
To Reproduce
Click the "authors" link on the right side and notice it doesn't get active.
Expected Behavior
Should be active
What's your node version? (if relevant)
No response

Is this Mintlify's default behavior or is this our fault?

seems it's the default behaviour,, nvm, closing this issue 👍


Unit test

This is adding unit tests for for lazy subscription to server streams.
I also added debug logging (VLOG) for the async signals queuing and executing. It helps with debugging issues when the queued signals are not delivered.
clang-formatting got in the way...

since this is not adding or changing functionality, I am going to merge the change right away.


Grabbing window by upper part of application bar doesn't trigger fancy zone use

Steps to reproduce

Have configuered fancy zones on working monitor
Open application with modern UI eg. Slack or Visual studio 2019
Hold "shift" and click grab window by upper part of application bar
Move mouse

Expected behavior
Fancy zones should appear on monitor and user should be able to place window into them
Actual behavior
Fancy zones doesn't appear and are unusable

Fancy zones is grabbing window moveSize event. Which doesn't differentiate between move and resize events. To determine which one of this event occured fancy zones is using padding method:
const auto padding_x = 8;
const auto padding_y = 6;
windowRect.top += padding_y;
windowRect.left += padding_x;
windowRect.right -= padding_x;
windowRect.bottom -= padding_y;

if (PtInRect(&windowRect, ptScreen) == FALSE)
{
    return;
}

Problem is that different windows have different paddings.
Probably instead of looking to paddings fancy zones can check cursor type. If it's normal arrow then it's move event. If it's diffent cursor like resize then it's resize event

The same for me with Visual Studio 2019 starting from version 16.4.4 or 16.4.5.

The same for me with Visual Studio 2019 starting from version 16.4.4 or 16.4.5.

Running as administrator (could be found in General Settings) fixed same issue for me with Visual Studio 2019 16.5 and IIS Manager

Hi @ukirill
what you describe (snapping applications that require PT to run as admin) is a different issue from the one tracked here.
Here the issue is with the cursor position, if it's too close to the top edge of the title-bar, FZ may fail to recognize that the window is dragged and think it's resized.

Estimating the cost to medium because of the testing needed for this one, ideally we should also have WinAppDriver tests for it.

I have 0.17 but it still doesn't work for the following applications:
Visual Studio 2019, 16.5.2
Windows Terminal,  0.11.1191.0
Task Manager
Running in VM on 1909 18363.778

@UsmanMohammad
for Task Manager you need to run PowerToys with administrator privilages.
For the other apps I can't repro the problem, they work as expected.
Can you please make sure it still doesn't work for you?

@UsmanMohammad
for Task Manager you need to run PowerToys with administrator privileges.
For the other apps I can't repro the problem, they work as expected.
Can you please make sure it still doesn't work for you?

Hi @enricogior I just saw power toys give a notification mentioning this, and it works now.

OK thanks, I'll close this issue as resolved.

@enricogior for me it still doesn't work. Main window of Visual Studio works OK, but any tool window (eg. error list, output...) doesn't fire fancy zone.

@PiterGoCompany
this issue was for handling the main application window.
For the the child windows https://github.com/microsoft/PowerToys/issues/1445


MagicalRecord integration

Hi. Did you try to combine your project with MagicalRecord?
I see, that it closely integrated with DATAStack and it seems, like quite bit different way to manage with  Core Data, than Magical Record implementation.
MR use simple method:
    + (void)setupCoreDataStack;

for initialisation of DataStack)
Is there any way to integrate Sync with MagicalRecord?

Hi @skywinder, interesting thought, at the moment MagicalRecord and DATAStack clash and they couldn't live together.
Could be interesting to replace DATAStack with MagicalRecord thought.
We'll make a pros and cons list and think about it.
Personally I think MagicalRecord it's a pretty massive dependency that I would rather not have.

@skywinder Any reason why you would prefer MagicalRecord over DATAStack?

According:

NShipster reference
According GithubStars
A lot of references to it over the internet.

I tend believe that this most common "Core Data wrapper".
And a lot of apps already use it in their apps
As a particular case: I would like to use, Sync in my app too, but have no idea, how to integrate it with my project, that use MagicalRecord already.
I will let you know, if I found some solution for it.

@skywinder Interesting points :point_up:

Hi Petr! At the moment we will keep only supporting DATAStack it's pretty lightweight and it fits our needs.
We will reconsider it if we get more requests.
DATAStack provides a simple way to create a Core Data stack, if there's anything you miss from MagicalRecord let us know and we could collaborate into make it more awesome. :grin: