Port most vLLM kernels to ROCm

This ports most vLLM kernels to ROCm (with the exception of the quantization_ops which is not critical to run some of the models).
If you have a working ROCm installation, you can compile this with python setup.py develop or python setup.py install (on my local machine, pip install -e . is NOT working, not sure if that is generic or specific to my setup).

It does build, but I don't think I have enough VRAM to run any unquantized model. Also, note that it require ROCm 5.7, amd_hip_bf16.h didn't exist before.
Anyway, doesn't it have a hard dependency on xFormers? Just trying to call vllm.entrypoints.openai.api_server throw ModuleNotFoundError: No module named 'xformers'. All the xFormers part need to have a different path if it's not installed.

The first step here will be to get one of the smaller models like OPT-125M working, which doesn't need AWQ. For dealing with xformers my plan is to work with https://github.com/pcmoritz/vllm-public/tree/flash-attn for the time being :)

@pcmoritz Curious if you would mind looking into porting the AWQ quantization kernels to ROCm too? Would be a benefit to everyone running quantized models.

@casper-hansen It depends on how the rest of the port goes and how the performance numbers for the rest looks like. I hope we can get some help from AMD to port them since it is likely more involved :)

@WoosukKwon Can you have a look at the PR? All the vLLM layers are working now on AMD hardware except the ones that depend on xformers (with the following patch that I didn't want to merge). Make sure to have a closer look at and test the refactoring in setup.py since I haven't tested it on nvidia hardware :)
(base) anyscale@phx2-pc2-017:/tmp/vllm-public$ git diff
diff --git a/tests/kernels/test_activation.py b/tests/kernels/test_activation.py
index 0b3ad0a..44aa944 100644
--- a/tests/kernels/test_activation.py
+++ b/tests/kernels/test_activation.py
@@ -72,4 +72,4 @@ def test_gelu_fast(
     out = torch.empty(num_tokens, d, dtype=dtype, device="cuda")
     activation_ops.gelu_fast(out, x)
     ref_out = get_activation("gelu_fast")(x)
-    assert torch.allclose(out, ref_out, atol=1e-5, rtol=1e-5)
+    assert torch.allclose(out, ref_out, atol=1e-3, rtol=1e-3)
diff --git a/tests/kernels/test_attention.py b/tests/kernels/test_attention.py
index 31d78dd..82ecc09 100644
--- a/tests/kernels/test_attention.py
+++ b/tests/kernels/test_attention.py
@@ -12,7 +12,7 @@ from vllm.utils import get_max_shared_memory_bytes
 FLOAT32_BYTES = torch.finfo(torch.float).bits // 8
 # This will change depending on the compute capability.
 # - 512 as a buffer
-MAX_SEQ_LEN = get_max_shared_memory_bytes() // FLOAT32_BYTES - 512
+MAX_SEQ_LEN = 8192
 NUM_BLOCKS = 128  # Arbitrary values for testing
 PARTITION_SIZE = 512

diff --git a/tests/kernels/test_pos_encoding.py b/tests/kernels/test_pos_encoding.py
index d660417..0f66a9e 100644
--- a/tests/kernels/test_pos_encoding.py
+++ b/tests/kernels/test_pos_encoding.py
@@ -170,5 +170,5 @@ def test_rotary_embedding(
     ref_key = ref_key.view(num_tokens, num_heads * head_size)

     # Compare the results.
-    assert torch.allclose(out_query, ref_query, atol=1e-5, rtol=1e-5)
-    assert torch.allclose(out_key, ref_key, atol=1e-5, rtol=1e-5)
+    assert torch.allclose(out_query, ref_query, atol=1e-3, rtol=1e-3)
+    assert torch.allclose(out_key, ref_key, atol=1e-3, rtol=1e-3)
(base) anyscale@phx2-pc2-017:/tmp/vllm-public$ ls tests/kernels/
conftest.py  __pycache__  test_activation.py  test_attention.py  test_cache.py  test_layernorm.py  test_pos_encoding.py
(base) anyscale@phx2-pc2-017:/tmp/vllm-public$ python -m pytest -s tests/kernels/test_activation.py
WARNING 10-16 23:45:51 ray_utils.py:35] Failed to import Ray with ModuleNotFoundError("No module named 'msgpack'"). For distributed inference, please install Ray with `pip install ray pandas pyarrow`.
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.11.4, pytest-7.4.2, pluggy-1.0.0
rootdir: /tmp/vllm-public
collected 108 items

tests/kernels/test_activation.py ............................................................................................................

=========================================================================================================== 108 passed in 0.50s ============================================================================================================
(base) anyscale@phx2-pc2-017:/tmp/vllm-public$ python -m pytest -s tests/kernels/test_cache.py
WARNING 10-16 23:46:02 ray_utils.py:35] Failed to import Ray with ModuleNotFoundError("No module named 'msgpack'"). For distributed inference, please install Ray with `pip install ray pandas pyarrow`.
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.11.4, pytest-7.4.2, pluggy-1.0.0
rootdir: /tmp/vllm-public
collected 270 items

tests/kernels/test_cache.py ..............................................................................................................................................................................................................................................................................

=========================================================================================================== 270 passed in 6.21s ============================================================================================================
(base) anyscale@phx2-pc2-017:/tmp/vllm-public$ python -m pytest -s tests/kernels/test_layernorm.py
WARNING 10-16 23:46:15 ray_utils.py:35] Failed to import Ray with ModuleNotFoundError("No module named 'msgpack'"). For distributed inference, please install Ray with `pip install ray pandas pyarrow`.
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.11.4, pytest-7.4.2, pluggy-1.0.0
rootdir: /tmp/vllm-public
collected 45 items

tests/kernels/test_layernorm.py .............................................

============================================================================================================ 45 passed in 0.44s ============================================================================================================
(base) anyscale@phx2-pc2-017:/tmp/vllm-public$ python -m pytest -s tests/kernels/test_pos_encoding.py
WARNING 10-16 23:46:21 ray_utils.py:35] Failed to import Ray with ModuleNotFoundError("No module named 'msgpack'"). For distributed inference, please install Ray with `pip install ray pandas pyarrow`.
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.11.4, pytest-7.4.2, pluggy-1.0.0
rootdir: /tmp/vllm-public
collected 864 items

tests/kernels/test_pos_encoding.py ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

=========================================================================================================== 864 passed in 3.77s ============================================================================================================
(base) anyscale@phx2-pc2-017:/tmp/vllm-public$ python -m pytest -s tests/kernels/test_attention.py
WARNING 10-16 23:46:37 ray_utils.py:35] Failed to import Ray with ModuleNotFoundError("No module named 'msgpack'"). For distributed inference, please install Ray with `pip install ray pandas pyarrow`.
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.11.4, pytest-7.4.2, pluggy-1.0.0
rootdir: /tmp/vllm-public
collecting ... Deterministic: False
Performance Mode: True
collected 324 items

tests/kernels/test_attention.py ................................................................................................................................................................................................................................................................................................FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

(the failures are the ones from xformers modules not being available)

Hi @pcmoritz, huge thanks for the great work! I will definitely take a look. As for the FlashAttention and xformers, we will probably be able to collaborate with the AI teams at AMD.

@pcmoritz @casper-hansen Speaking of the AWQ kernels, I believe they are for temporary use. We plan to implement much faster kernels (probably using Triton) in the near future.

Thanks for the great work @pcmoritz ! @WoosukKwon !
FYI - As to FlashAttention, MosaicML posted a blog on LLM training with AMD's ML250 via ROCmSoftwarePlatform's relevant effort on FlashAttention. It might be worth having a test with it
From their blog

Start with a ROCm 5.4  Docker image: [rocm/dev-ubuntu-20.04:5.4.3-complete](https://hub.docker.com/r/rocm/dev-ubuntu-20.04/tags)
Install PyTorch for ROCm 5.4: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.4.2
Install ROCm-based FlashAttention following the instructions here.

@pcmoritz @jingqiao @WoosukKwon I am the leader of flash-attention in AMD.
Please reach out me if you need a newer flash attention with newer ROCm.

@pcmoritz Thanks again for the great work! I have a quick question: How did you port the CUDA kernels to HIP? Did you use the AMD's hipify script or just do it manually?

I did it manually (the hipify script is actually called under the hood by the compiler to translate the file, which mostly replaces the cuda* APIs with hip* APIs). In my first iteration, I called hipify on the files but then you get a bunch of copied files and that would be hard to maintain. Later I figured out that hipify is called automatically under the hood, and this PR just contains the stuff that hipify can't convert :)

@iAmir97 converted the script to AMD assembler instructions in https://github.com/pcmoritz/vllm-public/pull/1/files, let me try if I can cherrypick the commits into this PR and if the tests still pass, then maybe that's the better solution since the performance might be better :)

The tests are passing, so I changed the code.
I'll be at skycamp tomorrow @WoosukKwon, if you want and have time, we can chat / hack some more on this :)

When I tried flash_attn branch, I got NameError: name 'BlockDiagonalCausalMask' is not defined.

Does asm work as expect?

@fsx950223 It is passing the unit tests for the layers, so seems to be working for me. On the BlockDiagonalCausalMask error -- there are some more changes that need to be made to get flash attention working, see https://github.com/vllm-project/vllm/pull/1314. It is not working fully for me yet but I'm also planning to spend some more time on it -- let me know if you'd like to help, it should be possible to get it working :)

@sabreshao I'm planning to work more on flash attention integration, currently I'm working with the latest master in https://github.com/ROCmSoftwarePlatform/flash-attention, let me know if you think using a different code base will make it more likely to succeed :) Currently the focus is on correctness and not speed yet :)

@fsx950223 It is passing the unit tests for the layers, so seems to be working for me. On the BlockDiagonalCausalMask error -- there are some more changes that need to be made to get flash attention working, see #1314. It is not working fully for me yet but I'm also planning to spend some more time on it -- let me know if you'd like to help, it should be possible to get it working :)

Could you share the development environment with me (maybe a docker file)? I met some issues on my local environment. And what's the type of GPU you are using MI250 or MI300?

@pcmoritz @WoosukKwon @sabreshao
Below is a workable solution to enable Flash Attention for vLLM per my testing. Pls just treat it as a reference, and I am sure you can make further optimizations.
To be specific, it is to follow the instruction at https://github.com/ROCmSoftwarePlatform/flash-attention except that, instead of using ROCm 5.4, we can use ROCm5.7 (already supported by pytorch nightly), the same version @pcmoritz used above.
To be specific, one can replace the following code in vLLM
        out = xops.memory_efficient_attention_forward(
            query.unsqueeze(0),
            key.unsqueeze(0),
            value.unsqueeze(0),
            attn_bias=input_metadata.attn_bias,
            p=0.0,
            scale=self.scale,
        )

with the following code (similar to the code here)
        from einops import repeat
        from flash_attn.bert_padding import unpad_input, pad_input
        from flash_attn.flash_attn_interface import flash_attn_unpadded_qkvpacked_func

        # Use Flash Attention (with ROCm support)
        batch_size = len(input_metadata.prompt_lens)
        seq_len = input_metadata.max_prompt_len

        lengths = torch.tensor(input_metadata.prompt_lens, device=query.device).view(-1,1)
        attention_mask_bool = repeat(
            torch.arange(seq_len, device=query.device),
            's -> b s', b=batch_size
        ) < lengths

        qkv = torch.stack([query, key, value], dim=1).view(
            batch_size, seq_len, 3, self.num_heads, self.head_size
        )
        qkv_unpad, indices, cu_seqlens, max_seqlen_in_batch = unpad_input(qkv, attention_mask_bool)

        out_unpad = flash_attn_unpadded_qkvpacked_func(
            qkv_unpad, cu_seqlens, max_seqlen_in_batch, dropout_p=0.0, causal=True
        )
        out = pad_input(out_unpad, indices, batch_size, seq_len).view(
                  -1, self.num_heads, self.head_size
        )

Hi @pcmoritz, I got the following error when building vLLM:
In file included from csrc/attention/attention_kernels.hip:23:
    In file included from csrc/attention/attention_dtypes_hip.h:7:
    In file included from csrc/attention/dtype_bfloat16_hip.cuh:29:
    /opt/rocm-5.6.0/include/hip/hip_bf16.h:30:10: fatal error: 'hip/amd_detail/amd_hip_bf16.h' file not found
    #include <hip/amd_detail/amd_hip_bf16.h>
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1 error generated when compiling for gfx1030.
    error: command '/opt/rocm-5.6.0/bin/hipcc' failed with exit code 1

Do you happen to know how to resolve this?
I'm using PyTorch 2.1.0 with ROCm 5.6.0. My local ROCm version is also 5.6.0.

This header is only available with ROCm 5.7 unfortunately (somebody else
ran into the same problem)
On Sat, Oct 21, 2023, 7:47 PM Woosuk Kwon @.***> wrote:

Hi @pcmoritz https://github.com/pcmoritz, I got the following error
when building vLLM:
In file included from csrc/attention/attention_kernels.hip:23:
In file included from csrc/attention/attention_dtypes_hip.h:7:
In file included from csrc/attention/dtype_bfloat16_hip.cuh:29:
/opt/rocm-5.6.0/include/hip/hip_bf16.h:30:10: fatal error: 'hip/amd_detail/amd_hip_bf16.h' file not found
#include <hip/amd_detail/amd_hip_bf16.h>
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated when compiling for gfx1030.
error: command '/opt/rocm-5.6.0/bin/hipcc' failed with exit code 1
Do you happen to know how to resolve this?
I'm using PyTorch 2.1.0 with ROCm 5.6.0. My local ROCm version is also
5.6.0.
—
Reply to this email directly, view it on GitHub
https://github.com/vllm-project/vllm/pull/1313#issuecomment-1773976938,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA3VJEMJEDN6CQ6VBYSPT3YASCLBAVCNFSM6AAAAAA52XIR6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZTHE3TMOJTHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

@pcmoritz I see. Then, which version of PyTorch did you use?

The nightly one that comes with ROCm 5.7 support
On Sat, Oct 21, 2023, 8:03 PM Woosuk Kwon @.***> wrote:

@pcmoritz https://github.com/pcmoritz I see. Then, which version of
PyTorch did you use?
—
Reply to this email directly, view it on GitHub
https://github.com/vllm-project/vllm/pull/1313#issuecomment-1773978992,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA3VJC2EKW7FDYTM3IA3ELYASEH3AVCNFSM6AAAAAA52XIR6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZTHE3TQOJZGI
.
You are receiving this because you were mentioned.Message ID:
@.***>

I optimized the performance on ROCM, with the patch, the performance improved about 10%.
diff --git a/csrc/attention/attention_kernels.cu b/csrc/attention/attention_kernels.cu
index babd15b..4937f55 100644
--- a/csrc/attention/attention_kernels.cu
+++ b/csrc/attention/attention_kernels.cu
@@ -23,7 +23,11 @@
#include
-#define WARP_SIZE 32
+#ifndef USE_ROCM

#define DEFAULT_NUM_THREADS 128
+#else

#define DEFAULT_NUM_THREADS 256
+#endif
#define MAX(a, b) ((a) > (b) ? (a) : (b))
#define MIN(a, b) ((a) < (b) ? (a) : (b))
#define DIVIDE_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
@@ -39,7 +43,7 @@ inline device float block_sum(float* red_smem, float sum) {
// Compute the sum per warp.
#pragma unroll

for (int mask = WARP_SIZE / 2; mask >= 1; mask /= 2) {

for (int mask = WARP_SIZE / 2; mask >= 1; mask >>= 1) {
sum += VLLM_SHFL_XOR_SYNC(sum, mask);
}

@@ -58,7 +62,7 @@ inline device float block_sum(float* red_smem, float sum) {
// Parallel reduction inside the warp.
#pragma unroll

for (int mask = NUM_WARPS / 2; mask >= 1; mask /= 2) {

for (int mask = NUM_WARPS / 2; mask >= 1; mask >>= 1) {
sum += VLLM_SHFL_XOR_SYNC(sum, mask);
}

@@ -114,8 +118,6 @@ device void paged_attention_kernel(
const int num_tokens = end_token_idx - start_token_idx;
constexpr int THREAD_GROUP_SIZE = MAX(WARP_SIZE / BLOCK_SIZE, 1);

constexpr int NUM_THREAD_GROUPS = NUM_THREADS / THREAD_GROUP_SIZE; // Note: This assumes THREAD_GROUP_SIZE divides NUM_THREADS
assert(NUM_THREADS % THREAD_GROUP_SIZE == 0);
constexpr int NUM_TOKENS_PER_THREAD_GROUP = DIVIDE_ROUND_UP(BLOCK_SIZE, WARP_SIZE);
constexpr int NUM_WARPS = NUM_THREADS / WARP_SIZE;
const int thread_idx = threadIdx.x;
@@ -149,11 +151,11 @@ device void paged_attention_kernel(
// th vectors of the query, and so on.
// NOTE(woosuk): Because q is split from a qkv tensor, it may not be contiguous.
const scalar_t* q_ptr = q + seq_idx * q_stride + head_idx * HEAD_SIZE;
shared Q_vec q_vecs[THREAD_GROUP_SIZE][NUM_VECS_PER_THREAD];

Q_vec q_vecs[NUM_VECS_PER_THREAD];
#pragma unroll

for (int i = thread_group_idx; i < NUM_VECS_PER_THREAD; i += NUM_THREAD_GROUPS) {

for (int i = 0; i < NUM_VECS_PER_THREAD; i++) {
const int vec_idx = thread_group_offset + i * THREAD_GROUP_SIZE;

q_vecs[thread_group_offset][i] = reinterpret_cast<const Q_vec>(q_ptr + vec_idx * VEC_SIZE);

q_vecs[i] = reinterpret_cast<const Q_vec>(q_ptr + vec_idx * VEC_SIZE);
}
__syncthreads(); // TODO(naed90): possible speedup if this is replaced with a memory wall right before we use q_vecs

@@ -200,7 +202,7 @@ device void paged_attention_kernel(
   // Compute dot product.
   // This includes a reduction across the threads in the same thread group.

 float qk = scale * Qk_dot<scalar_t, THREAD_GROUP_SIZE>::dot(q_vecs[thread_group_offset], k_vecs);

 float qk = scale * Qk_dot<scalar_t, THREAD_GROUP_SIZE>::dot(q_vecs, k_vecs);
 // Add the ALiBi bias if slopes are given.
 qk += (alibi_slope != 0) ? alibi_slope * (token_idx - context_len + 1) : 0;

@@ -231,7 +233,7 @@ device void paged_attention_kernel(
// Get the max qk value for the sequence.
qk_max = lane < NUM_WARPS ? red_smem[lane] : -FLT_MAX;
#pragma unroll

for (int mask = NUM_WARPS / 2; mask >= 1; mask /= 2) {

for (int mask = NUM_WARPS / 2; mask >= 1; mask >>= 1) {
qk_max = fmaxf(qk_max, VLLM_SHFL_XOR_SYNC(qk_max, mask));
}
// Broadcast the max qk value to all threads.
@@ -319,7 +321,7 @@ device void paged_attention_kernel(
for (int i = 0; i < NUM_ROWS_PER_THREAD; i++) {
float acc = accs[i];
#pragma unroll

for (int mask = NUM_V_VECS_PER_ROW / 2; mask >= 1; mask /= 2) {

for (int mask = NUM_V_VECS_PER_ROW / 2; mask >= 1; mask >>= 1) {
acc += VLLM_SHFL_XOR_SYNC(acc, mask);
}
accs[i] = acc;
@@ -332,8 +334,8 @@ device void paged_attention_kernel(
// Perform reduction across warps.
float* out_smem = reinterpret_cast<float*>(shared_mem);
#pragma unroll

for (int i = NUM_WARPS; i > 1; i /= 2) {
int mid = i / 2;

for (int i = NUM_WARPS; i > 1; i >>= 1) {
int mid = i >> 1;
// Upper warps write to shared memory.
if (warp_idx >= mid && warp_idx < i) {
float* dst = &out_smem[(warp_idx - mid) * HEAD_SIZE];
@@ -485,7 +487,7 @@ global void paged_attention_v2_reduce_kernel(
// Get the global max logit.
// Reduce within the warp.
#pragma unroll

for (int mask = WARP_SIZE / 2; mask >= 1; mask /= 2) {

for (int mask = WARP_SIZE / 2; mask >= 1; mask >>= 1) {
max_logit = fmaxf(max_logit, VLLM_SHFL_XOR_SYNC(max_logit, mask));
}
if (lane == 0) {
@@ -495,7 +497,7 @@ global void paged_attention_v2_reduce_kernel(
// Reduce across warps.
max_logit = lane < NUM_WARPS ? red_smem[lane] : -FLT_MAX;
#pragma unroll

for (int mask = NUM_WARPS / 2; mask >= 1; mask /= 2) {

for (int mask = NUM_WARPS / 2; mask >= 1; mask >>= 1) {
max_logit = fmaxf(max_logit, VLLM_SHFL_XOR_SYNC(max_logit, mask));
}
// Broadcast the max value to all threads.
@@ -533,9 +535,9 @@ global void paged_attention_v2_reduce_kernel(
} // namespace vllm

#define LAUNCH_PAGED_ATTENTION_V1(HEAD_SIZE)                                                  \

cudaFuncSetAttribute(                                                                       \

hipFuncSetAttribute(
(void*)vllm::paged_attention_v1_kernel<T, HEAD_SIZE, BLOCK_SIZE, NUM_THREADS>,            \

cudaFuncAttributeMaxDynamicSharedMemorySize, shared_mem_size);                            \

hipFuncAttributeMaxDynamicSharedMemorySize, shared_mem_size);
vllm::paged_attention_v1_kernel<T, HEAD_SIZE, BLOCK_SIZE, NUM_THREADS>
<<<grid, block, shared_mem_size, stream>>>(
out_ptr,
@@ -556,7 +558,7 @@ global void paged_attention_v2_reduce_kernel(
template<
typename T,
int BLOCK_SIZE,

int NUM_THREADS = 128>

int NUM_THREADS = DEFAULT_NUM_THREADS>
void paged_attention_v1_launcher(
torch::Tensor& out,
torch::Tensor& query,
@@ -715,7 +717,7 @@ void paged_attention_v1(
template<
typename T,
int BLOCK_SIZE,

int NUM_THREADS = 128,

int NUM_THREADS = DEFAULT_NUM_THREADS,
int PARTITION_SIZE = 512>
void paged_attention_v2_launcher(
torch::Tensor& out,
diff --git a/csrc/attention/attention_utils.cuh b/csrc/attention/attention_utils.cuh
index ff64c4b..1987ad4 100644
--- a/csrc/attention/attention_utils.cuh
+++ b/csrc/attention/attention_utils.cuh
@@ -39,7 +39,7 @@ inline device float qk_dot_(const Vec (&q)[N], const Vec (&k)[N]) {
// Finalize the reduction across lanes.
float qk = sum(qk_vec);
#pragma unroll

for (int mask = THREAD_GROUP_SIZE / 2; mask >= 1; mask /= 2) {

for (int mask = THREAD_GROUP_SIZE / 2; mask >= 1; mask >>= 1) {
qk += VLLM_SHFL_XOR_SYNC(qk, mask);
}
return qk;
diff --git a/csrc/cuda_compat.h b/csrc/cuda_compat.h
index 8991462..f56cbd0 100644
--- a/csrc/cuda_compat.h
+++ b/csrc/cuda_compat.h
@@ -17,3 +17,10 @@
#else
#define VLLM_SHFL_SYNC(var, src_lane) __shfl(var, src_lane)
#endif

+#ifndef USE_ROCM

#define WARP_SIZE 32
+#else
#define WARP_SIZE 64
+#endif
diff --git a/csrc/reduction_utils.cuh b/csrc/reduction_utils.cuh
index b95ccef..563b776 100644
--- a/csrc/reduction_utils.cuh
+++ b/csrc/reduction_utils.cuh
@@ -19,12 +19,13 @@

#include "cuda_compat.h"

namespace vllm {
template
inline device T warpReduceSum(T val) {
#pragma unroll

for (int mask = 16; mask > 0; mask >>= 1)

for (int mask = WARP_SIZE / 2; mask > 0; mask >>= 1)
val += VLLM_SHFL_XOR_SYNC(val, mask);
return val;
}
@@ -32,9 +33,15 @@ inline device T warpReduceSum(T val) {
/* Calculate the sum of all elements in a block */
template
inline device T blockReduceSum(T val) {

static shared T shared[32];

static shared T shared[WARP_SIZE];

+#ifndef USE_ROCM
int lane = threadIdx.x & 0x1f;
int wid = threadIdx.x >> 5;
+#else

int lane = threadIdx.x & 0x3f;

int wid = threadIdx.x >> 6;
+#endif
val = warpReduceSum(val);

@@ -45,7 +52,7 @@ inline device T blockReduceSum(T val) {
// Modify from blockDim.x << 5 to blockDim.x / 32. to prevent
// blockDim.x is not divided by 32

val = (threadIdx.x < (blockDim.x / 32.f)) ? shared[lane] : (T)(0.0f);

val = (threadIdx.x < (blockDim.x / float(WARP_SIZE))) ? shared[lane] : (T)(0.0f);
val = warpReduceSum(val);
return val;
}

@pcmoritz Thank you for the port. Are you aware of a way to compile against RoCm 5.6?

need to support 5.6 because thats what pytorch supports now

@ehartford Yes it would be nice. So far I have been using nightly and it works quite nicely!

Ok I figured out how to install both versions of rocm so that I can use 5.7 with vllm and 5.6 with pytorch

what's preventing this from being merged?

(vllm) eric@quixi1:~/vllm-public$ python setup.py install
Traceback (most recent call last):
  File "/home/eric/vllm-public/setup.py", line 28, in <module>
    raise RuntimeError(
RuntimeError: Cannot find CUDA_HOME. CUDA must be available to build the package.

@WoosukKwon Do you have any idea when the changing of AWQ kernels will happen? planning to port them to rocm.

@ehartford Are you on the right branch? In this branch, this should be handled (https://github.com/vllm-project/vllm/pull/1313/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R27) -- your error seems to indicate that the raise RuntimeError line is line 28, which is not the case in this branch :)

Closed as we merged #1836 which is a superset of this PR. @pcmoritz Thanks for the amazing work!


[GOBBLIN-1323] wrap extractColumnListInQuery in RestApiExtractor

Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[ ] My PR addresses the following Gobblin JIRA issues and references them in the PR title.
https://issues.apache.org/jira/browse/GOBBLIN-1323

Description

[ ] Here are some details about my PR, including screenshots (if applicable):
Wrap Utils.getColumnListFromQuery(inputQuery) so that child class can override it to support customized query.

Tests

[ ] My PR adds the following unit tests OR does not need testing for this extremely good reason:
I do not include a test because there is no code content change, only structure changes.

Commits

[ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report

Merging #3156 (dda7995) into master (f81a2a2) will decrease coverage by 36.70%.
The diff coverage is 0.00%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f81a2a2...dda7995. Read the comment docs.


[Linter] Rule to use loadJsonContent instead json & loadTextContent

Usage of json(loadTextContent('jsonFile.json')) should be discouraged by linter and we suggest converting code to use loadJsonContent function with a quick fix.
Originally posted by @alex-frankel in https://github.com/Azure/bicep/issues/5846#issuecomment-1034150255
To be done once #6563 is merged.

@alex-frankel shouldn't we milestone it to the next release after the loadJsonContent will be released?

Are you going to be the one picking it up? If so, we can change the milestone. Otherwise, I am not sure who will be free to pick it up.

LoadJSONContent is missing from the docs i believe, logged an issue here.
Issue azure docs


Added the EPEL repositories in the FTS installation process with multiline

Summary

Related issue :

The jscpd check can be ignored as the duplication seem to make sense there.

And this was fixed by a commit in another PR.

@andrea-manzi can you check this one again? We'd like to introduce some big changes to the structure of the docs and it would be good not to have any pending PR

@andrea-manzi can you check this one again? We'd like to introduce some big changes to the structure of the docs and it would be good not to have any pending PR

Sorry in fact it's on @crsndr to address the comments by Andrea: basically to remove mention of EPEL-8 as it's not supported.


CORE-15815: Reduce the number of open connections

Add a cache of key stores in the TrustStoresMap so that two identical sets of certificates will have the same key store, and the same key in the connection manager, which will reduce the number of connections in the connection manager.

@yift-r3 thinking about this a bit more, a better way of resolving this might actually be to adjust DestinationInfo, so that it takes the actual certificates (e.g. as PEM strings). And then these are converted to keystores only at the point of use, e.g. when establishing the connections.
It also seems the same issue can happen with clientCertificatesKeyStore in the case of mutual TLS. So, we need to change this too to fix the issue properly.


How do we load previously generated paths?

Firstly, thanks for writing this component.
I'd like to control the UI side of things myself and just make use of SketchCanvas but I'm struggling to understand how I can apply previous paths.
If I store an array of paths (sketch) as described in the docs and have the following in my app:
componentDidMount() {
  sketch.map((movement) => {
    console.log('drawing', movement.path)
    this.refs.canvas.addPath(movement)
  })
}

I would expect the canvas to display them but it remains blank. It could be that I have to somehow invalidate the render so it redraws, but I'm unsure if what I'm trying to accomplish is supported.

I have fixed this bug! Thanks!

how sir


Get lists of input devices

Is there any way to get the list of the devices? I've been reading the code but I cannot find any way to do it.
Thanks for this awesome library! It works wonderful!
Edit: Could this be related? https://github.com/fluent-ffmpeg/node-fluent-ffmpeg/issues/351

I don't think this is possible with ffmpeg. You'll have to look for other libraries to do that, or inspect /sys or /proc if you target unixes.


RocketMQ Dashbaord 是否支持RocketMQ 5.3.1版本

The issue tracker is ONLY used for bug report and feature request.
Any question or RocketMQ proposal please use our mailing lists.
BUG REPORT

Please describe the issue you observed:

What did you do (The steps to reproduce)?

What did you expect to see?

What did you see instead?

Please tell us about your environment:

Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):

FEATURE REQUEST

Please describe the feature you are requesting.

Provide any additional detail on your proposed use case for this feature.

Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?

If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:

sub-task1-issue-number: sub-task1 description here,
sub-task2-issue-number: sub-task2 description here,
...

主要是acl api相关的

cc 哥哥们
@jimjag @bsnyder @raboof

同问。rocketmq5.x官方文档写了这个doashboard，但是感觉这边完全没有对5的版本做更新啊

主要是acl api相关的

放弃吧，dashboard估计短期内没人维护了


Fix incomplete project properties

I found that in a certain case, I wasn't getting all of the preprocessor
definitions that were present:
In a property sheet:
<ItemDefinitionGroup>
  <ClCompile>
     <PreprocessorDefinitions>NOMINMAX;%(PreprocessorDefinitions)</PreprocessorDefinitions>
  </ClCompile>
<ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)'=='Debug'">
  <ClCompile>
    <PreprocessorDefinitions>_DEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
  </ClCompile>
</ItemDefinitionGroup>

In a project including the property sheet:
<ItemDefinitionGroup Condition="'$(Configuration)'=='Debug'">
  <ClCompile>
    <PreprocessorDefinitions>%(PreprocessorDefinitions)</PreprocessorDefinitions>
  </ClCompile>
</ItemDefinitionGroup>

In this case, I was missing NOMINMAX. I'm not sure why, but the
functionality that was being used to get these values just left it out.
The tool wrappers can each cast their wrapped objects to an
IVCRulePropertyStorage, which does return the correct information. It also
handles property sheets and everything else, so it should simplify the
code a lot.
Fixes #12

Wow, great work at cutting down the clumsiness of the way we used to fetch include directories and preprocessor defines!
Just one request before merging: In your commit you mentioned an example case that isn't working correctly in the current release of this extension. It would be great if you could add a test case for the situation you described so that we can make sure that this will never break again. Can you please do that?

Sure, I can add it to the existing property sheet case. It can pull in a configuration-specific definition and a generic one. I only wish I knew why the old version didn't work... maybe it just ignores %(Additional...) item metadata, the documentation doesn't say either way :(

You can review the test changes in isolation in this commit.

Thanks for adding to the test case. I'll merge this now.


Feature: Support for theme packs

What feature or improvement do you think would benefit Files?
We are going to support for displaying a background image. It would make the distribution of themes for Files easier if the theme could be changed by double-clicking on a theme pack file, which contains a background image and other theme settings.
Requirements

Theme pack files have .filestheme extension.
A theme pack file is actually a zip archive, which contains an image file for the background and a json file which stores theme settings.
If the user opens a theme pack file, Files applies the background image and other theme settings contained within it.

Files Version
<IP_ADDRESS>
Windows Version
10.0.22631.3447
Comments
No response

Thank you for your feedback

Wouldn't it make more sense to use CSS? How are the build-in "themes" currently implemented?

CSS is for web browsers and Files isn't a web site or based on one. Currently there values in settings that get applied across Files code base in XAML

We have zip extractor inside of the codebase so making .filestheme as zip is a good idea.
We should register in our new database and show them like Windows Settings app does.

For ease of use I would not put it in any type of archive, this way it's easy to edit without having to unzip and zip constantly. Another solution would be to have it work like how Minecraft does texturepacks, having both folders and zip files work.

You wouldn't have to un-zip and re-zip to edit and test. It stored as a zip when finalized so that all the files are available and easy to move around. Windows themes work the same just under a different extension.

No reason to edit by your own it probably makes worsen because once it break you have to creat new one with correct names and configurations. So as @Josh65-2201 said, being archive is good enough.

@hishitetsu I like this idea, couple of thoughts

as there are sometimes changes to the theming options, we'll need some sort of versioning system, or at least proper error handling to only import valid values.
it would be cool if this works with background images, we can probably copy the images to the app data folder.
we might need to add a "reset" option to the appearance settings UI.

Would you recommend reading the values and saving them to Files settings, or is there a use case to keep a reference to the theme pack and read them directly each time Files is launched? Curious to hear your thoughts on the best approach.

IMO, I would imagine something like Windows Settings; you can save the current theme including background and accent color as a theme set and Themes section enables you to reapply another theme or export/import. As an edge case, since someone would want to try to edit an exported theme file and import with modified, Files better copy and show it on Themes section (just recent 6 theme packs). Also it'd be nice if you can delete a theme and clear all themes.


GA plan for Microsoft.Azure.Functions.Worker.ApplicationInsights

There are several outstanding work items needed before the Microsoft.Azure.Functions.Worker.ApplicationInsights nuget package can be declared generally available (GA).
We'll be resolving any related App Insights issues here and making sure there is a central place to bring up issues that we can add to this list. Once the package is GA, we'll close this and handle new issues as we would any other package.
To fix the issue with App Insights logs missing from Console logging in Core Tools:

[ ] Allow worker packages to specify capabilities (#1183)
[ ] Host release that includes App Insights PR (https://github.com/Azure/azure-functions-host/pull/8965) must be in production
[ ] #1264

OpenTelemetry support (on worker only):

[ ] #1358
[ ] #1359

Documentation:

[ ] Explain the split between worker and host logging. Especially how configuration is managed for each (https://github.com/Azure/azure-functions-dotnet-worker/pull/944#issue-1282987627)

Might I also suggest that the Net7Worker sample be updated to include proper Application Insights configuration when available? I have recently been on the rollercoaster (#1182) that is AI configuration for .NET 7 Isolated Workers and at this point I'm very much interested in some official documented solutions.

Is there already any progress or news regarding the General Availability of this package ?
The latest release of this (preview) package is already 5 months old.  Is this project still alive ?

Functions with HttpTrigger don't work with Microsoft.Azure.Functions.Worker 1.8.0 and Microsoft.Azure.Functions.Worker.ApplicationInsights 1.0.0-preview3 packages. The problem is solved by updating Microsoft.Azure.Functions.Worker to 1.10.0 version

Functions with HttpTrigger don't work with Microsoft.Azure.Functions.Worker 1.8.0 and Microsoft.Azure.Functions.Worker.ApplicationInsights 1.0.0-preview3 packages. The problem is solved by updating Microsoft.Azure.Functions.Worker to 1.10.0 version

How does custom telemetry then work, as there is no more TelemetryClient registered in the dependency container?

@Shumigaj any update on this?

My functions doesn't work at all with Microsoft.Azure.Functions.Worker 1.8.0 and Microsoft.Azure.Functions.Worker.ApplicationInsights 1.0.0-preview3 packages. I couldn't make any requests to it on my local machine until I just updated Microsoft.Azure.Functions.Worker to 1.10.0 version, after that update everything works as expected. I wanted to mention this side effect with the version because it seems that there is no limitation for the version of the Microsoft.Azure.Functions.Worker package in the Microsoft.Azure.Functions.Worker.ApplicationInsights package.

My functions doesn't work at all with Microsoft.Azure.Functions.Worker 1.8.0 and Microsoft.Azure.Functions.Worker.ApplicationInsights 1.0.0-preview3 packages. I couldn't make any requests to it on my local machine until I just updated Microsoft.Azure.Functions.Worker to 1.10.0 version, after that update everything works as expected. I wanted to mention this side effect with the version because it seems that there is no limitation for the version of the Microsoft.Azure.Functions.Worker package in the Microsoft.Azure.Functions.Worker.ApplicationInsights package.

Yes, ok, but how does custom telemetry work with this? As it is the preview package that registers the TelemetryClient.

Could you please provide some feedback on how to do custom telemetry in isolated Azure Functions, without using the Microsoft.Azure.Functions.Worker.ApplicationInsights package?
As no TelemetryClient or TelemetryConfiguration is registered in the application services.
@Shumigaj
FYI @fgheysels @pim-simons

It is possible, just use Microsoft.ApplicationInsights and Microsoft.ApplicationInsights.WorkerService packages. Dependencies can be registered by AddApplicationInsightsTelemetryWorkerService method. But, there are drawbacks of this approach (e.g. operation id won't be populated)

Any news about this ?
Will the previewXX packages prevents us from upgrading our Function to .NET 8 in the future ?

But, there are drawbacks of this approach (e.g. operation id won't be populated)

Then it is rather pointless to use this approach when there's no service-to-service/transactional correlation available; feels like an incomplete workaround.
The preview package is still the best solution, here.

I see the package is available as stable in Nuget. https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.ApplicationInsights/1.0.0
Is this thread still open ?

We wanted to wait for the Azure Update. But yes, now closeable.

There's a note on This page

When custom application logs are sent directly, the host no longer be emits them, and host.json no longer controls their behavior

To what extent is this true?
Without additional values in host.json I get duplicate entries for worker process log messages i.e.
One with customDimensions.Category = 'Host.Function.Console'
And the other with customDimensions.CategoryName = Fully qualified name of my function


Tooltips for Annotorious labels

How do I add tooltips to  annotorious labels created in headless mode with disableEditor: true set ?
Is there a demo somewhere ?

See the hover tooltip plugin here:
https://github.com/recogito/recogito-client-plugins/blob/main/plugins/annotorious-hover-tooltip/src/index.js
This plugin is, unfortunately, undocumented. But the code shows how it works. I.e. you can track mouse position and use the built-in onMouseEnterAnnotation and
onMouseLeaveAnnotation events to show/hide the tooltip.

PS: for general support questions, please use the Gitter chat here rather than the issue tracker:
https://gitter.im/recogito/annotorious


pull request for e2e test at 12/10/2016 02:56:02

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


sql error on registration from japan

Error

Error message

PDOException: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'field_address_country' at row 1: INSERT INTO {field_data_field_address} (entity_type, entity_id, revision_id, bundle, delta, language, field_address_country, field_address_administrative_area, field_address_sub_administrative_area, field_address_locality, field_address_dependent_locality, field_address_postal_code, field_address_thoroughfare, field_address_premise, field_address_sub_premise, field_address_organisation_name, field_address_name_line, field_address_first_name, field_address_last_name, field_address_data) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8, :db_insert_placeholder_9, :db_insert_placeholder_10, :db_insert_placeholder_11, :db_insert_placeholder_12, :db_insert_placeholder_13, :db_insert_placeholder_14, :db_insert_placeholder_15, :db_insert_placeholder_16, :db_insert_placeholder_17, :db_insert_placeholder_18, :db_insert_placeholder_19); Array ( [:db_insert_placeholder_0] => user [:db_insert_placeholder_1] => 1705473 [:db_insert_placeholder_2] => 1705473 [:db_insert_placeholder_3] => user [:db_insert_placeholder_4] => 0 [:db_insert_placeholder_5] => und [:db_insert_placeholder_6] => JPN [:db_insert_placeholder_7] => [:db_insert_placeholder_8] => [:db_insert_placeholder_9] => [:db_insert_placeholder_10] => [:db_insert_placeholder_11] => [:db_insert_placeholder_12] => [:db_insert_placeholder_13] => [:db_insert_placeholder_14] => [:db_insert_placeholder_15] => [:db_insert_placeholder_16] => [:db_insert_placeholder_17] => [:db_insert_placeholder_18] => [:db_insert_placeholder_19] => ) in field_sql_storage_field_storage_write() (line 514 of /var/www/dev.dosomething.org/html/modules/field/modules/field_sql_storage/field_sql_storage.module).
The website encountered an unexpected error. Please try again later.

We should be receiving only the two-letter codes from Fastly.
Obviously, the site shouldn't totally die if we pass in bad data for the country.

might be my bad, I explicitly set the header to JPN
but yeah, the db column has a limit of 2 characters (should we increase it...)

yeah, this is me sending the wrong header.


Next Manual, 2.8.0

Here's where issues for the 2.8.0 manual go.

Missed this for 2.7.0, from @jonathan-g:

[ ] page 451, section "Warmup Times and Estimating the Mass Matrix": minor LaTeX problem due to using non-ASCII character instead of ASCII single-quote for apostrophe in source: "... standardizing the data doesnâ ̆ Z ́ t always help. Therefore, Stan ..."

[ ] Add Dustin Tran as developer

[ ] thank @dvukcevic for stan-dev/stan#1548

From Andrew:
A funny thing:  exponential distribution is parameterized in terms of inverse-scale, but double_exponential is parameterized in terms of its scale.  For historical reasons I understand this but maybe

[ ] we should flag it in the manual.

From Mike Lawrence on stan-users:

In the pre-optimized code for the multivariate regression example, spanning pgs 66-67 in version 2.7.0 of the manual, there are two sampling statements for y; is this an error? If not, can someone explain what the two different statements are achieving?

There's a vectorized and non-vectorized version both in the same model block.

[ ] we should remove one of the sampling statements

[ ] fix sign error in simple harmonic oscillator --- diff eq notation doesn't match code

[ ] change "strangely" to "seemingly" in the unrelated regressions description

https://github.com/JuliaPalacios said in https://github.com/stan-dev/stan/issues/1577

section{Variational Inference} (the first Variational Inference section 1.6), first paragraph, there is this sentence:
"Variational inference can be much for estimating posterior means and marginal variances than MCMC. See Chapter 59 for more information."

[ ] fill in the much what?

In the section on the multinomial distribution, multinomial_rng returns an int[] rather than a vector.

[ ] Thank Daniel Chen in the acknowledgements for supplying Python install instructions (that's code and doc!)

From Guido Biele on stan-users:
I think there is a small error in the example for ragged arrays on page 135 of the stan-reference.
the relevant section in the model block of the example reads:
  int pos;
  pos <- 1;
  for (k in 1:K)
    segment(y, pos, s[k]) ~ normal(mu[k], sigma);

which means that the stating index of the slicing operation is always 1.
instead it should probably be something like
 int pos;
  pos <- 1;
  for (k in 1:K){
    segment(y, pos, s[k]) ~ normal(mu[k], sigma);
    pos <- pos + s[k];
  }

(in figure 2 it should also be z = [†1:3 2:4 0:9 ...  and not y = [†1:3 2:4 0:9 ..." , or better change the figure caption because y` is also used in the example model ...)

[ ] fix the example
[ ] fix the figure
[ ] thank Guido in the acknowledgements

[ ] Thank Takahiro Kubo for install instructions for PyStan (I guess that's doc)

[ ] replace post_mulitply with pre_multiply in the Cholesky example

[ ] compare the mixture at model level vs. mixture at observation level

This is confusing a lot of users who try to vectorize and wind up mixing at the model level rather than observation level.

[ ] remove any mention of mc-stan.org emails

From Andy on stan-users:

[...] under "Time-Series Models"->"Autoregressive Models" of the manual (Ch 6.1) <lower=0> is not used for specifying sigma (e.g. pg 77)

[ ] add lower=0 constraint for sigma in time series section

[x] update grant funding and add corporate funding and foundation funding

There is a typo in the section '48.6. Multivariate Student-t Distribution' in the PDF formula
It is: \Gamma \chi ((...
it should be: \Gamma ((...


Add Flag to Disable Implicit Optional Conversions

:warning: Controversial :warning:
Adds the flag -disable-implicit-optional-conversion to the frontend.  When enabled, the compiler will reject:

Implicit wrapping of value types into optionals
Implicit unwrapping of IUO values

I was thinking about going deeper, but I want feedback on this part first.  If I dropped the optional part and just went full-on -disable-implicit-conversion, I think you could get some interesting results out of the language.
There would definitely be performance gains to be had by disabling all instances of constraint generation that does casting because there are so many cases that are being handled right now.  In addition, though you would lose a lot of "convenience", there are times where it pays to be type-safe.

This is a change that would need discussion on swift-evolution before moving forward. It'd probably be valuable to the discussion to see what kinds of things this breaks.

Right, especially if I were to go forward with the latter.  I'll draw something up and we'll revisit this after the community has their say.
Thanks!

@jckarter +1, seeing what this change does, e.g., to the standard library and test code is a good first step.

As others have pointed out, this is better for swift-evolution.  That said, the discussion can be much more productive with data!


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target_0d515806-c1d0-4731-b8de-3465d0fcb309/pull/1 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-target_0d515806-c1d0-4731-b8de-3465d0fcb309.git

Docs Build status updates of commit 9c254e3:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error: RepositoryUnauthorizedWithSourceCommit] Git data access to 'https://github.com/opstest2/docs-archive-test-source' (branch: Release_Archive_main_2022-02-16-09-49-04) is not authorized.. It looks like this might be the first time user: v-alji submitted a pull request to this repo. Before user can run builds in a pull request, he/she can enable 'v-alji' GitHub account in Docs by visiting Docs Portal (https://ops.microsoft.com). After enabling his/her account (v-alji), close and then reopen this pull request to start another build.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Feat: add right sidebar nav desktop

Description

This solves #85

Add Desktop right sidebar for Trending quests etc
Implementation

Only available in Desktop
Create a RightSidebar screen, some implementation are in the layout already
Add TABS for this Right bar: Trending, Quests.
Add some data as examples using Nostr hooks, Launchpad hooks etc
Add Right sidebar to the Router.ts and navigator for Main, Degen etc
Add this in the Router.ts component maybe (some impl have been try to add two DrawerNavigator)

Screenshots
Demo
https://github.com/user-attachments/assets/17d367c6-6920-498f-9f39-70536b6997d9
Desktop

Mobile

@MSghais 👋 Hey this is ready for review!


Fix for CVE-2024-28180 (Improper Handling of Highly Compressed Data ) introduced through gopkg.in/square/go-jose.v2

What version of Go are you using (go version)?

$ go version v1.22

Does this issue reproduce with the latest release?
Yes
What version of Go-Guardian are you using ?

Go-Guardian Version:  github.com/shaj13/go-guardian/v2 v2.11.5

What did you do?
Github security scan (dependabot) and checkmarx has reported a CVE in this library that is introduced through gopkg.in/square/go-jose.v2 v2.5.1.
CVE information can be found here : https://github.com/advisories/GHSA-c5q2-7r4c-mv6g
The version of go-jose library need to updated. The path with the fix is available in github.com/go-jose/go-jose/v4 4.0.1 onwards.

What did you expect to see?
No Vulnerabilities.
What did you see instead?
Saw CVE-2024-28180 https://github.com/advisories/GHSA-c5q2-7r4c-mv6g

@shaj13 do you have plans on addressing this issue? or if I open a PR with a fix for this issue, would it get accepted?
Would really appreciated your inputs on this.

@shaj13 do you have plans on addressing this issue? or if I open a PR with a fix for this issue, would it get accepted? Would really appreciated your inputs on this.

Yes, I am aware of the issue and currently, there are no plans for a fix. However, if you open a PR with a fix, it would be welcomed and reviewed. Just for context, what strategy are you using? I assume it's JWT. Is it in a production environment?

We are using JWT strategy it in one of our open sourced components for JWT based authentication and authorization.
I will open a PR for this in that case, would go version upgrade along with it also be welcomed?

@hmgowda  Yes, but that would be a significant change, as it would require a refactor and updates to the CI/CD pipeline as well.

@shaj13  I have a PR for this issue https://github.com/shaj13/go-guardian/pull/132
Would appreciate your review and see if this can be merged into master.

Thanks @hmgowda, Fixed included in v2.11.6.
PR #132


Fix: Delete modal does not disappear immediately

Details

Fixed Issues

$ https://github.com/Expensify/App/issues/43496
PROPOSAL: https://github.com/Expensify/App/issues/43496#issuecomment-2162194698
Tests

precondition: user is a member of a collect workspace

Go to OD as a workspace admin
Add some text reports fields
As a member of the workspace, open the ND app
Open the workspace chat
Submit some expense
As admin delete any report field in OD
Open the expense report in ND app as a member
Select the deleted report field and deleted
The delete modal disappears immediately before the user is returned to the report

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/170341821/fe72a566-00ec-427c-b304-75f4b31fd913

Android: mWeb Chrome

https://github.com/Expensify/App/assets/170341821/af4abb99-4991-40e6-9d42-3e51a8fda2fc

iOS: Native

https://github.com/Expensify/App/assets/170341821/e46e8b7d-bdb9-4b0b-a931-e441f0cfd1d5

iOS: mWeb Safari

https://github.com/Expensify/App/assets/170341821/33be70f9-2603-44ce-8773-62f126f29050

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/170341821/0acccd02-b650-4c21-9020-f546c2539985

MacOS: Desktop

https://github.com/Expensify/App/assets/170341821/176460f4-3aea-4234-81de-e8fb6a46dbf8

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native
https://github.com/Expensify/App/assets/35371050/fc3bcaf4-065a-4f24-9a81-d1e5c3046f04

Android: mWeb Chrome
https://github.com/Expensify/App/assets/35371050/af27b229-c6d3-4668-b1be-43b15c1422be

iOS: Native
https://github.com/Expensify/App/assets/35371050/01f5a2ff-aafd-46de-8a4c-4dcb3cc9ce35

iOS: mWeb Safari
https://github.com/Expensify/App/assets/35371050/3883a1df-7924-4fef-8492-7c632d171008

MacOS: Chrome / Safari
https://github.com/Expensify/App/assets/35371050/9412c43e-8302-4023-8228-70065964241c

MacOS: Desktop
https://github.com/Expensify/App/assets/35371050/c7a0bce6-9388-4442-b85b-944c81888e36

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/bondydaa in version: 1.4.86-0 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/bondydaa in version: 1.4.86-0 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/bondydaa in version: 1.4.86-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/AndrewGable in version: 9.0.0-9 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


docs: Add note to LDAP docs about Zulip not tracking user email changes.

Doc note for https://chat.zulip.org/#narrow/stream/3-backend/topic/LDAP.20documentation.20.2324104
Not sure if this really belongs all the way at the top, but I think it might make sense? And down below there's just a ton of instructions and different, often independent sections, so it's hard to find a good spot for this note there without risk that many users will never even see it there.

I think it probably makes sense to put this next to the "Syncing names and avatars" type sections. This detail isn't important when you're just trying to get something working -- but instead when you're trying to figure out how to maintain your userbase.
(It'll also be where we'll want to document a future feature to make it possible to automated this issue)

I think it probably makes sense to put this next to the "Syncing names and avatars" type sections. This detail isn't important when you're just trying to get something working -- but instead when you're trying to figure out how to maintain your userbase.
(It'll also be where we'll want to document a future feature to make it possible to automated this issue)

Hmm yeah that actually feels like it belongs pretty well, moved it to the top of the Synchronizing data section

Cool, I did an edit pass to spell out the options for how to do manual adjustments clearly -- this should have all the information a user needs. Tagging to merge once CI passes.


banner_url Method

Implement banner_url methods for User and record::PartialUser structs, to get the user's banner image URLs.

These methods were already implemented:
https://docs.rs/tetr_ch/0.6.0/tetr_ch/index.html?search=banner_url


sed: can't read conftest.cpp: No such file or directory

line 9595
/home/cabox/workspace/coachshake/ppl-1.0/configure

line 2322
conftest.cpp missing

commented out
in trial

fixed, configure file
forgot edits global.inlines.hh and compiler.hh
that fixed problems
mostly static_cast edits

conftest.c makes self


[security] Update 3rd party dependencies to get rid of all currently known CVE issues

Hijacking this issue as placeholder for security upgrade.
original text:

The smime-module has dependency on utils-mail-smime and it has dependency on bcjmail-jdk15to18 along with further transitive dependencies from Bouncy Castle.
From your this comment @bbottema I come to know that you are in process of updating 3rd party dependencies, hence consider a suggestion of using bcjmail-jdk18on instead of bcjmail-jdk15to18 as simple-java-mail is compatible from JDK8+.
Also, bc*-jdk15to18 JARs are designed to be compatible with JDK versions 1.5 through 1.8, where on other hand bc*-jdk18on are designed to be compatible with JDK 1.8 and later versions. So it makes sense, isn't it? even the https://bouncycastle.org/latest_releases.html also says the same.
Please ignore if you have already considered this :)

Indeed, I will look into it 👍

Thanks. Any tentative plan for new version release.

Today probably

Updates:

Dependencies

Spring 5.3.27 -> 5.3.34
Spring Boot 2.5.15 -> 2.7.18
commons-io 2.7 -> 2.11.0
utils-mail-smime 2.3.1 -> 2.3.3

org.bouncycastle:bcjmail-jdk15to18 1.75 -> org.bouncycastle:bcjmail-jdk18on 1.78.1

ical4j 2.2.4 -> ical4j-vcard 2.0.0-beta2

Other:

Junit 4 -> Junit 5 (including Mockito, AssertJ and got rid of Powermock)
maven-surefire-plugin 2.19.1 -> 3.2.5

v8.9.0 was released to Maven Central!


feat: Save currency user preference

The type of this PR is: Feature

This PR resolves CX-1947
Description
This PR adds a new model to save user preferences and adds the currency for the artwork form to it.
PR Checklist (tick all before merging)

[ ] I have included screenshots or videos to illustrate my changes, or I have not changed anything that impacts the UI.
[ ] I have tested my changes on iOS and Android.
[ ] I have added tests/stories for my changes, or my changes don't require testing/stories, or I have included a link to a separate Jira ticket covering the tests.
[ ] I have added a feature flag, or my changes don't require a feature flag. (How do I add one?)
[ ] I have documented any follow-up work that this PR will require, or it does not require any.
[ ] I have added an app state migration, or my changes do not require one. (What are migrations?)
[ ] I have added a changelog entry below or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

Warnings

:warning:

❌ An error occurred while validating your changelog, please make sure you provided a valid changelog.

  Generated by :no_entry_sign: dangerJS against 84d82933cfc8ec43aced3801c6d97fbdbf5de0a8


file-upload: acceptedFiles removed after invalid file is uploaded

🐛 Bug report

previous acceptedFiles automatically removed/reset to empty array after uploading invalid file to  file-upload
only happened in maxFiles: 1
multiple files working as expected
is it like this by design/on purpose?

💥 Steps to reproduce

Upload any valid files
Valid file is set to acceptedFiles
Upload file again that should be invalid
Invalid file is set to rejectedFiles
acceptedFiles automatically reset to empty array

💻 Link to reproduction
CodeSandbox reproduction: https://codesandbox.io/p/sandbox/2nhrqh
🧐 Expected behavior

I expected it would keep the acceptedFIles since invalid file is rejected

🌍 System information

Software
Version(s)

ark-ui/react Version
v4.0.0

Zag Version

Browser

Operating System

📝 Additional information
https://github.com/chakra-ui/zag/blob/571852d3d279fed8d5d3c454e858938dbbb97c7a/packages/machines/file-upload/src/file-upload.machine.ts#L123
im pretty sure it is this line of code that get executed for my usecase

I just pushed a fix for this. We'll release an update shortly.
If the issue persists after upgrading, I'll re-open it.


Backport of 2675 in stable-3.1

Backport of #2675 in stable-3.1
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1578086

jenkins test luminous-ansible2.4-ooo_collocation

failure is unrelated.


Implement async/await listeners for @laconia/batch

Context
See https://github.com/laconiajs/laconia/issues/838
Consideration
(Draft)

https://www.npmjs.com/package/async-eventemitter - Can't use because it doesn't support multiple args (will break backward compatibility)
https://www.npmjs.com/package/await-event-emitter
Emittery - https://www.npmjs.com/package/emittery

Just checking:
The bug: we are not awaiting all items callback to complete
This PR: we will await item callback to complete and then call another or finish
But could be: we will not wait unless we're finishing, then we have to wait all pending
I'm not sure about this last one, that's the question,

Good: I faced cases where PR behavior is desirable
Bad: Could reduce parallelism of batch process
Ugly: Await all could add unnecessary(?) complexity and memory leaks

@hugosenari Hello! I'm glad you're still here!
Thanks for your thoughts.

Bad: Could reduce parallelism of batch process

When we're hitting an API endpoint via laconia/batch, we have the following scenarios:

Scenario
Slow endpoint?
Rate limited endpoint?

1
Yes
Yes

2
Yes
No

3
No
Yes

4
No
No

Scenario 1: Parallelism is undesirable as if it's already slow, that's putting more pressure to the endpoint
Scenario 2: I can see the benefit of parallelism here, but why wouldn't dev just do Promise.all instead of using laconia/batch?
Scenario 3: Not affected by async/await
Scenario 4: Not affected by async/await

But could be: we will not wait unless we're finishing, then we have to wait all pending

The way I think about this is what's 'intuitive' in devs head. At the moment, I think most devs think that if they put async as the listener, laconia should wait for it i.e. when they write the following code, the most intuitive thing is that laconia will wait for this code. Do you agree with the intuitiveness?
.on('item', async (_, item) =>  wait(1500))

Ugly: Await all could add unnecessary(?) complexity and memory leaks

I'm not sure what complexity and memory leaks you meant by this, could you elaborate more?

@hugosenari Hello! I'm glad you're still here!

I never stop believing in the right tool for the job, sadly people use hammer to screwup things, so we have to use other tools.
Let me make something clear: I never used laconia/batch :sheep:
My comments are pure theory.

why wouldn't dev just do Promise.all instead of using laconia/batch?

Because Laconia author is smarter and made a pretty cool interface that would reduce my code complexity also calling my function again before timeouts.

The bug: we are not awaiting all items callback to complete

A different perspective: we are already waiting for all item callback / listener to complete when they're synchronous.

I'm trying to sum the bug we already have before your PR, to make sure we have a common ground =)

When we're hitting an API endpoint via laconia/batch, we have the following scenarios:

But 'slow endpoint' could be 'slow IO' or 'slow CPU', if slow IO, endpoint may handle more requests.
And  pressure are handled by 'bottleneck' correct?

The way I think about this is what's 'intuitive' in devs head. At the moment, I think most devs think that if they put async as the listener, laconia should wait for it i.e. when they write the following code, the most intuitive thing is that laconia will wait for this code. Do you agree with the intuitiveness?

From intuition, I'm not the best guy to ask this question. ;-)
It should wait for it, but there are three ways to wait for it:
let tasks = [1, 2, 3].map(Promise.resolve);

// await synchronous
for (let somePromise of tasks) {
   console.log(await somePromise);
}

// await all
for (let someValue of await Promise.all(tasks)) {
  console.log(someValue)
}

// await race
while (tasks.length) {
   const somePromise = Promises.race(works);
   console.log(await somePromise);
   tasks = tasks.filter(p => p != somePromise);
}

I'm not sure what others devs would expect, to me best is race, takes the first completed finish theirs job and goes to the next.
The Better Boarding Method Airlines Won't Use  https://www.youtube.com/watch?v=oAHbLRjF0vo

I'm not sure what complexity and memory leaks you meant by this, could you elaborate more?

Your solution is simple, I'm reading this books

@hugosenari Thanks for your thoughts! In this scenario, not supporting async is causing quite an unpredictable behaviour (see the original ticket) - so I think we need to fix that regardless.

I've released this under 1.13.2-alpha0 to test the solution against https://github.com/mpxr/laconia-batch-test. The issue is now fixed.
With @laconia/batch version 1.10.2:
START RequestId: c2b40fc0-7a8b-4a1a-8dc7-ae74a4c90206 Version: $LATEST
2022-07-30T05:54:10.031Z c2b40fc0-7a8b-4a1a-8dc7-ae74a4c90206 INFO START
2022-07-30T05:54:10.616Z c2b40fc0-7a8b-4a1a-8dc7-ae74a4c90206 INFO JOKE { name: 'VS Code', '#1': "What is bread's favorite number? Leaven.", '#2': 'Why are pirates called pirates?  I have a hunch, it might be me.', '#2': 'I’ll tell you something about German sausages, they’re the wurst' }
2022-07-30T05:54:14.043Z c2b40fc0-7a8b-4a1a-8dc7-ae74a4c90206 INFO END
END RequestId: c2b40fc0-7a8b-4a1a-8dc7-ae74a4c90206
REPORT RequestId: c2b40fc0-7a8b-4a1a-8dc7-ae74a4c90206 Duration: 4018.30 ms Billed Duration: 4019 ms Memory Size: 1024 MB Max Memory Used: 87 MB Init Duration: 691.50 ms
2022-07-30T06:00:50.210Z c2b40fc0-7a8b-4a1a-8dc7-ae74a4c90206 ERROR Unhandled Promise Rejection {"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error:

With @laconia/batch version 1.13.2-alpha0:
START RequestId: 9076f0af-0aeb-4db3-a2eb-0564557ff4ab Version: $LATEST
2022-07-30T06:11:02.617Z 9076f0af-0aeb-4db3-a2eb-0564557ff4ab INFO START
2022-07-30T06:11:03.019Z 9076f0af-0aeb-4db3-a2eb-0564557ff4ab INFO JOKE { name: 'VS Code', '#1': 'Where do you take someone who has been injured in a Peek-a-boo accident? To the I.C.U.', '#2': 'Don’t interrupt someone working intently on a puzzle. Chances are, you’ll hear some crosswords.' }
2022-07-30T06:11:05.102Z 9076f0af-0aeb-4db3-a2eb-0564557ff4ab INFO JOKE { name: 'Fleet', '#1': 'How do you make a hankie dance? Put a little boogie in it.', '#2': "I was fired from the keyboard factory yesterday. I wasn't putting in enough shifts." }
2022-07-30T06:11:06.900Z 9076f0af-0aeb-4db3-a2eb-0564557ff4ab INFO JOKE { name: 'Eclipse', '#1': 'My cat was just sick on the carpet, I don’t think it’s feline well.', '#2': "I was going to learn how to juggle, but I didn't have the balls." }
2022-07-30T06:11:06.900Z 9076f0af-0aeb-4db3-a2eb-0564557ff4ab INFO END
END RequestId: 9076f0af-0aeb-4db3-a2eb-0564557ff4ab
REPORT RequestId: 9076f0af-0aeb-4db3-a2eb-0564557ff4ab Duration: 4288.69 ms Billed Duration: 4289 ms Memory Size: 1024 MB Max Memory Used: 87 MB Init Duration: 699.89 ms


Add a label check action

Checks if one of the lerna-changelog tags is present

You can see a failed check here: https://github.com/deephaven/web-client-ui/runs/4081157392?check_suite_focus=true
I like how it tells you how to correct the problem:

Error: Label check failed: required 1 of bug, enhancement, internal, documentation, but found 0.

We should consider adding that to check-doc-labels.sh in deephaven-core as well, as the current error "No documentation requirements found" and "Conflicting documentation requirements" is vague/requires prior knowledge of which labels satisfy that condition.

Can you do a gitbot comment with the error, so you don't have to dig? Could have something like "One of the following labels are required: internal, enhancement, etc.. see [link to guideline section on labels] for more details." or something.
 - name: 'Comment PR'
        uses: actions/github-script@v3
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            const { issue: { number: issue_number }, repo: { owner, repo }  } = context;
            github.issues.createComment({ issue_number, owner, repo, body: `comment goes here, markdown works` }

Does this error out if you have more than one of a label? I'm not against simplifying the deephaven-core side label check w/ better action if it's applicable and easy to understand.

Yeaaa it probably shouldn't take a minute. I used this action because it does have the one_of or any_of or all_of operations, so it would fail if there are more than one of a label. But could implement similar to deephaven-core to make it simpler

@devinrsmith I wrote a label-check-one-of.sh script to handle this, now it takes ~4s instead of ~1m. The script checks all LABEL_ prefixed environment variables and ensures exactly 1 is true; in all other cases it lists off how many were true and the list of labels it can only be one of.
This can be used in deephaven-core as well, the error message there doesn't tell you what labels you need.
Note that this method doesn't work with labels that have spaces (since env vars can't have spaces)... you could just sub in underscores and I'm sure people would figure it out


Unify Dockerfiles

Description of the Tech Debt
All of the Dockerfiles that we use to deploy the front end are (nearly) identical. Without too much difficulty, we should be able to have a single Dockerfile for the entire monorepo, with minor config changes handled by the Nx targets (Nx already handles the build for Next.js).
Development checklist

[ ] Collate dockerfile in a single location
[ ] Point all Nx container targets to single location, with any relevant properties passed.

Tech Debt Triage
The purpose of our technical debt triage process is to analyze technical debt to determine risk level of the technical debt and the value in tackling that technical debt.
Risk Value Scoring:

Level
Value

High
3

Medium
2

Low
1

Technical Debt - Risk Types
Level
Value

Business Area Risk - Risk of business area visibility / damage to user experience

Developer Fault Risk - How likely will this tech debt cause a future error related to coding on top of it

System Fault Risk - Risk of system errors or application downtime

Time Scale Risk - Compound risk effect if left alone. How much more difficult to fix or dangerous will this become over time?

Time Sink Risk - How much will this tech debt slow the development process down

TOTAL SCORE:

Handled as a single commit in https://github.com/bcgov/cas-registration/pull/1869!


problem using generator-angular2-library to import non-Angular dependencies (FullCalendar.io)

I have used generator-angular2-library to create a component to wrap and use FullCalendar http://fullcalender.io for Angular 4 projects, but found I cannot include and package the FullCalendar libraries properly within the generator-angular2-library generated component. The end result is that using in final application (like the one created from angular-cli), the component within the app cannot load reference to FullCalendar.io.
I have created 2 demos, one is the component itself created by generator-angular2-library, fullcalendar-ag4 https://github.com/kktam/fullcalendar-ag4.  This is published in npm https://www.npmjs.com/package/fullcalendar-ag4.
The demo applicaton that consumes fullcalendar-ag4, is fullcalendar-app-ag4, https://github.com/kktam/fullcalendar-app-ag4.
Aftering installing and runnning
npm i
ng serve

Then the following error occurs during ng serve.
$ ng serve
** NG Live Development Server is listening on localhost:4200, open your browser on http://localhost:4200 **
Hash: fdfb486dd1a1fc9bf051
Time: 33871ms
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 160 kB {4} [initial] [rendered]
chunk    {1} main.bundle.js, main.bundle.js.map (main) 5.41 kB {3} [initial] [rendered]
chunk    {2} styles.bundle.js, styles.bundle.js.map (styles) 10.5 kB {4} [initial] [rendered]
chunk    {3} vendor.bundle.js, vendor.bundle.js.map (vendor) 2.18 MB [initial] [rendered]
chunk    {4} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

ERROR in ./~/fullcalendar-ag4/index.js
Module not found: Error: Can't resolve 'fullcalendar/index' in '/Users/user/Documents/Work/angular2/fullcalendar-app-ag4/node_modules/fullcalendar-ag4'
 @ ./~/fullcalendar-ag4/index.js 5:0-28
 @ ./src/app/app.module.ts
 @ ./src/main.ts
 @ multi webpack-dev-server/client?http://localhost:4200 ./src/main.ts

I have checked another example https://github.com/lbertenasco/ap-ng2-fullcalendar that uses hand crafted bundling method, and it fullcalendar is packaged correctly.  Your help is most appreciated.

Hi @kktam  have a look here https://github.com/angular/angular/issues/16084
can you try to set annotateForClosureCompiler to false?

I does not seemed to be related (I do not have this flag in my tsconfig.json at first). I put it on and try enable and disable and does not make a difference.

I attempt to try to see if modifying the gulp step and bundle jquery and fullcalendar directly into umd build would help.   I need some help to work on the gulp task.  I have part of the work in gulp branch https://github.com/kktam/fullcalendar-ag4/tree/gulp. Help is most welcome.

@kktam — Do you have fullcalendar listed as peer dependency in your library's package.json?

By check here https://github.com/kktam/fullcalendar-app-ag4/blob/master/package.json I believed I missed that. I will add that and retest.

Question. I added FullCalendar in the generated library package here https://github.com/kktam/fullcalendar-ag4/blob/fullcalendar-ag4/package.json.
The dependencies are
 "dependencies": {
    "@types/fullcalendar": "^2.7.44",
    "@types/jquery": "3.2.5",
    "fullcalendar": "^3.4.0",
    "jquery": "^3.2.1"
}

Will this already work?

I'm having this issue as well. I have in my module import {BsDropdownModule} from 'ngx-bootstrap'; but in my index.js i have import { BsDropdownModule } from 'ngx-bootstrap/index';

@BenevidesLecontes have the same problem with openlayers library. Banging my head for three hours, still, can't find the solution.

@adjioev we did it with simple gulp task, but it is temporary until this generator fix it.

gulp.task('rollup:fix:replace', function () {
  gulp.src(['dist/index.js', 'dist/wiseit-ui.umd.js'])
    .pipe(replace('ng-selectize/index', 'ng-selectize'))
    .pipe(replace('ngx-ngSelectize_index', 'ngSelectize'))
    .pipe(replace('ngx-bootstrap/index', 'ngx-bootstrap'))
    .pipe(replace('ngxBootstrap_index', 'ngxBootstrap'))
    .pipe(replace('ng2-daterangepicker/index', 'ng2-daterangepicker'))
    .pipe(replace('ng2-daterangepicker/index', 'ng2-daterangepicker'))
    .pipe(replace('ng2Daterangepicker_index', 'ng2Daterangepicker'))
    .pipe(gulp.dest('dist'));

});

gulp.task('rollup:fix', function () {
  runSequence(
    'rollup:fix:replace',
    function() {
      gulp.src("dist/.replace/*")
        .pipe(gulp.dest('dist'));
    }
  );
});

@BenevidesLecontes, are you trying to include closure library?
If yes try to edit /src/tsconfig.ts5.json and set "annotateForClosureCompiler" to false.

@adjioev can you try to fork my test library and see if that work?

@BenevidesLecontes I am a bit new to gulp. Can you fork both my project https://github.com/kktam/fullcalendar-ag4 and https://github.com/kktam/fullcalendar-app-ag4  with a commit to show me how to apply your changes?  many thanks.
In the meantime, I will try to patch my project using your approach.

@kktam Hi, i did a pull request with some changes and now it's working fine. I tested with your app.

@BenevidesLecontes Thanks for the pull request. I am testing it on my demo app, but I think I am missing something on app code and webpack build failed with a different problem now.
Can you show me how you setup your demo app (if it is ok to ask if you can fork my demo app as well).
Demo app
https://github.com/kktam/fullcalendar-app-ag4
Error
npm run start

><EMAIL_ADDRESS>start /Users/xyz/Documents/Work/angular2/fullcalendar-app-ag4
> ng serve

** NG Live Development Server is listening on localhost:4200, open your browser on http://localhost:4200 **
Hash: 5fc714906c60e021f348
Time: 16735ms
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 160 kB {4} [initial] [rendered]
chunk    {1} main.bundle.js, main.bundle.js.map (main) 8.22 kB {3} [initial] [rendered]
chunk    {2} styles.bundle.js, styles.bundle.js.map (styles) 10.5 kB {4} [initial] [rendered]
chunk    {3} vendor.bundle.js, vendor.bundle.js.map (vendor) 3.34 MB [initial] [rendered]
chunk    {4} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

ERROR in /Users/xyz/Documents/Work/angular2/fullcalendar-app-ag4/src/app/app.module.ts (5,32): Cannot find module 'fullcalendar-ag4'.

ERROR in Error encountered resolving symbol values statically. Could not resolve fullcalendar-ag4 relative to /Users/xyz/Documents/Work/angular2/fullcalendar-app-ag4/src/app/app.module.ts., resolving symbol AppModule in /Users/xyz/Documents/Work/angular2/fullcalendar-app-ag4/src/app/app.module.ts, resolving symbol AppModule in /Users/xyz/Documents/Work/angular2/fullcalendar-app-ag4/src/app/app.module.ts

ERROR in ./src/app/app.module.ts
Module not found: Error: Can't resolve 'fullcalendar-ag4' in '/Users/nelsontam/Documents/Work/angular2/fullcalendar-app-ag4/src/app'
 @ ./src/app/app.module.ts 10:0-50
 @ ./src/main.ts
 @ multi webpack-dev-server/client?http://localhost:4200 ./src/main.ts
webpack: Failed to compile.

@kktam you doing wrong look at https://gist.github.com/BenevidesLecontes/b54cbfa8e83cb17fdcd99f8b8f9ebca4

@BenevidesLecontes — Thank you for your help, much appreciated! 👍
@kktam — Can you let us know if you got it working? Thanks!

@BenevidesLecontes  first problem solved. Now I got the same result as in your screenshot. Thanks very much!
@BenevidesLecontes second question. FullCalendar comes with its own set of .css. It appeared the css never got built into the webpack, so that when the final host app import npm library it does not contain the non angular library css as well. I am not sure if there is a solution for this, or just the developer must always look and import all underlying dependencies css in the index.html. (a bit undesirable).
@BenevidesLecontes  another quick question. Since you fixed my issue with annotateForClosureCompiler. So under what situation do we need to apply your patch with gulp tasks ('rollup:fix')?

@jvandemo  @BenevidesLecontes  based on the readme.md help for importing css using scss...
...
use a tilde to import a file from the nearest parent node_modules directory:
@import<EMAIL_ADDRESS>I have created a scss import file, but it does not work right now. Is the scss feature intended for this type of purpose (to import dependent library css into npm project so users don't need to add them separately in each application setup).
I have checked in the test changes here:
https://github.com/kktam/fullcalendar-ag4/commit/dedc2d87fcf852e7cbd7a694d9eecf293d9ec353

@kktam i'll try to make the css import work for you and you don't need to apply the patch in the gulp file. I think you may close this issue since the main problem is fixed.

@BenevidesLecontes Thanks very much.

@BenevidesLecontes — Thank you, really appreciate your help!
@kktam — Thank you for your follow-up 👍

@jvandemo @BenevidesLecontes  Thanks very much for all your help. It turns out amongst many users errors,  I did not synchronize peer dependencies in src/package.json and match exactly with /package.json and caused problems in the umd file. It would be a nice feature if the generator library can transform the package.json contents and render src/package.json automatically. Cheers

@jvandemo You may consider using project https://github.com/kktam/fullcalendar-ag4 and https://github.com/kktam/fullcalendar-app-ag4 to demo the use case on how to build npm components using generator-angular2-component referencing non-angular peer dependencies.

@kktam, @BenevidesLecontes — Would you be interested in writing out a guide on how to do that?
I would be happy to add the guide to the documentation and WIKI page so others can benefit from your learnings.
Thanks in advance!


Adding the ability to set default storageclasses

Adding the ability to select default storage classes as it was missed as part of the migration.
rancher/dashboard#2411

Storage went to v1 and deprecated beta in k8s 1.6.  I didn't easily find exactly what version it was removed in but it's certainly before any current version dashboard supports, so you don't need to set the beta annotation.

Storage went to v1 and deprecated beta in k8s 1.6. I didn't easily find exactly what version it was removed in but it's certainly before any current version dashboard supports, so you don't need to set the beta annotation.

@vincent99 ember ends up setting and seemingly reading both which ends up with incorrect behavior when they're set on dashboard but read on ember. Is it harmful to keep this in? I'd rather leave it in until ember is deprecated, I can create an issue or leave a todo.

Leaving it in isn't going to break anything, it's just extra data and code that isn't needed...  But Ember sets both and reads either so I don't see how Vue only reading/writing the non-beta one would be a problem either.