%134 = arith.addi %133, %132 : vector<8xi32>
      %135 = vector.extract %8[63] : vector<64x8xi32>
      %136 = arith.addi %135, %134 : vector<8xi32>
      scf.yield %136 : vector<8xi32>
    }
    vector.transfer_write %5, %3[%arg0] {in_bounds = [true]} : vector<8xi32>, memref<32xi32, affine_map<(d0)[s0] -> (d0 + s0)>>
  }

InnerParallel may offer higher accuracy on floating-point reductions since floating-point elements are reduced in sequential order (assuming that sequential order means higher accuracy, which is something arguable). However, transposing the data is very expensive!
InnerReduction
This strategy keeps the reduction dimension in the innermost dimension. Vectorization is applied on the reduction dimension and a scalar accumulator is used along the reduced dimension. This requires performing horizontal vector reductions per iteration of the innermost loop to add the partial reduction of each iteration to the scalar accumulator. The following snippet shows the resulting IR after applying the InnerReduction strategy on the running example:
  scf.for %arg0 = %c0 to %c32 step %c8 {
    %5 = scf.for %arg1 = %c0 to %c512 step %c64 iter_args(%arg2 = %cst) -> (vector<8xi32>) {
      %6 = vector.transfer_read %4[%arg0, %arg1], %c0_i8 {in_bounds = [true, true]} : memref<32x512xi8, affine_map<(d0, d1)[s0] -> (d0 * 512 + s0 + d1)>>, vector<8x64xi8>
      %7 = arith.extsi %6 : vector<8x64xi8> to vector<8x64xi32>
      %8 = vector.extract %7[0] : vector<8x64xi32>
      %9 = vector.extract %arg2[0] : vector<8xi32>
      %10 = vector.reduction <add>, %8, %9 : vector<64xi32> into i32
      %11 = vector.insertelement %10, %cst[%c0 : index] : vector<8xi32>
      %12 = vector.extract %7[1] : vector<8x64xi32>
      %13 = vector.extract %arg2[1] : vector<8xi32>
      %14 = vector.reduction <add>, %12, %13 : vector<64xi32> into i32
      %15 = vector.insertelement %14, %11[%c1 : index] : vector<8xi32>
      %16 = vector.extract %7[2] : vector<8x64xi32>
      %17 = vector.extract %arg2[2] : vector<8xi32>
      %18 = vector.reduction <add>, %16, %17 : vector<64xi32> into i32
      %19 = vector.insertelement %18, %15[%c2 : index] : vector<8xi32>
      %20 = vector.extract %7[3] : vector<8x64xi32>
      %21 = vector.extract %arg2[3] : vector<8xi32>
      %22 = vector.reduction <add>, %20, %21 : vector<64xi32> into i32
      %23 = vector.insertelement %22, %19[%c3 : index] : vector<8xi32>
      %24 = vector.extract %7[4] : vector<8x64xi32>
      %25 = vector.extract %arg2[4] : vector<8xi32>
      %26 = vector.reduction <add>, %24, %25 : vector<64xi32> into i32
      %27 = vector.insertelement %26, %23[%c4 : index] : vector<8xi32>
      %28 = vector.extract %7[5] : vector<8x64xi32>
      %29 = vector.extract %arg2[5] : vector<8xi32>
      %30 = vector.reduction <add>, %28, %29 : vector<64xi32> into i32
      %31 = vector.insertelement %30, %27[%c5 : index] : vector<8xi32>
      %32 = vector.extract %7[6] : vector<8x64xi32>
      %33 = vector.extract %arg2[6] : vector<8xi32>
      %34 = vector.reduction <add>, %32, %33 : vector<64xi32> into i32
      %35 = vector.insertelement %34, %31[%c6 : index] : vector<8xi32>
      %36 = vector.extract %7[7] : vector<8x64xi32>
      %37 = vector.extract %arg2[7] : vector<8xi32>
      %38 = vector.reduction <add>, %36, %37 : vector<64xi32> into i32
      %39 = vector.insertelement %38, %35[%c7 : index] : vector<8xi32>
      scf.yield %39 : vector<8xi32>
    }
    vector.transfer_write %5, %3[%arg0] {in_bounds = [true]} : vector<8xi32>, memref<32xi32, affine_map<(d0)[s0] -> (d0 + s0)>>
  }

InnerReduction may be more performant than InnerParallel since data transposition is not needed. However, horizontal reductions are also somewhat expensive and they are used in every iteration of the reduction loop. This approach does not reduce the elements in sequential order so the final result may be “less accurate”.
Proposal
We want to introduce a new reduction strategy that avoids the overhead of transposing data but also minimizes the number of horizontal reductions by:

Using a vector accumulator across the reduction dimension instead of a scalar accumulator.
Computing a single horizontal vector reduction at the end of the reduction dimension, when all the elements have been reduced to the single vector accumulator.

The following snippet shows the potential resulting IR after applying the new strategy on the running example (please, mind some mistakes, I modified the IR by hand):
func.func @main_dispatch_5_dispatch_0() {
  %cst = arith.constant dense<0> : vector<8xi32>
  %cst8x16 = arith.constant dense<0> : vector<8x16xi32>
  %c0_i8 = arith.constant 0 : i8
  %c512 = arith.constant 512 : index
  %c32 = arith.constant 32 : index
  %c64 = arith.constant 64 : index
  %c1 = arith.constant 1 : index
  %c8 = arith.constant 8 : index
  %c384 = arith.constant 384 : index
  %c0 = arith.constant 0 : index
  %c0_i32 = arith.constant 0 : i32
  %0 = hal.interface.binding.subspan set(0) binding(0) type(storage_buffer) offset(%c0) alignment(64) : !flow.dispatch.tensor<readonly:384x512xi8>
  %tmp = hal.interface.binding.subspan set(0) binding(1) type(storage_buffer) offset(%c0) alignment(64) : !flow.dispatch.tensor<readwrite:8x16xi32>
  %1 = hal.interface.binding.subspan set(0) binding(1) type(storage_buffer) offset(%c0) alignment(64) : !flow.dispatch.tensor<writeonly:384xi32>
  %workgroup_id_x = hal.interface.workgroup.id[0] : index
  %workgroup_count_x = hal.interface.workgroup.count[0] : index
  %2 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_id_x]
  %3 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_count_x]
  scf.for %arg0 = %2 to %c384 step %3 {
    %4 = flow.dispatch.tensor.load %1, offsets = [%arg0], sizes = [32], strides = [1] : !flow.dispatch.tensor<writeonly:384xi32> -> tensor<32xi32>
    %5 = flow.dispatch.tensor.load %0, offsets = [%arg0, 0], sizes = [32, 512], strides = [1, 1] : !flow.dispatch.tensor<readonly:384x512xi8> -> tensor<32x512xi8>
    %6 = scf.for %arg1 = %c0 to %c32 step %c8 iter_args(%arg2 = %4) -> (tensor<32xi32>) {
      // Create a new temporary that will hold the vector partial results that
      // will later be horizontally reduced.
      %tmp_ld = flow.dispatch.tensor.load %tmp, offsets = [%c0, %c0], sizes = [8, 16], strides = [1, 1] : !flow.dispatch.tensor<readwrite:8x16xi32> -> tensor<8x16xi32>
      %tmp_init = vector.transfer_write %cst8x16, %tmp_ld[%c0, %c0] {in_bounds = [true, true]} : vector<8x16xi32>, tensor<8x16xi32>
      %9 = scf.for %arg3 = %c0 to %c512 step %c64 iter_args(%arg4 = %tmp_init) -> (tensor<8x16xi32>) {
        %11 = vector.transfer_read %5[%arg1, %arg3], %c0_i8 {in_bounds = [true, true]} : tensor<32x512xi8>, vector<8x64xi8>
        %12 = vector.transfer_read %arg4[%c0, %c0], %c0_i32 {in_bounds = [true, true]} : tensor<8x16xi32>, vector<8x16xi32>
        %13 = arith.extsi %11 : vector<8x64xi8> to vector<8x64xi32>
        // Extend 'vector.multi_reduction' so that the output type can be a
        // vector. The output reduction dimension must be multiple of the input
        // reduction dimension. This should be lowered to non-horizontal vector additions.
        %14 = vector.multi_reduction <add>, %13, %12 [1] : vector<8x64xi32> to vector<8x16xi32>
        %15 = vector.transfer_write %14, %arg4[%c0, %c0] {in_bounds = [true, true]} : vector<8x16xi32>, tensor<8x16xi32>
        scf.yield %15 : tensor<8x16xi32>
      }
      // Perform a single horizontal reduction once the full reduction dimension
      // has been reduced to a single vector.
      %vred = vector.transfer_read %9[%c0, %c0], %c0_i32 {in_bounds = [true, true]} : tensor<8x16xi32>, vector<8x16xi32>
      %hred = vector.multi_reduction <add>, %vred, %cst [1] : vector<8x16xi32> to vector<8xi32>
      %10 = vector.transfer_write %hred, %arg2[%arg1] {in_bounds = [true]} : vector<8xi32>, tensor<32xi32>
      scf.yield %10 : tensor<32xi32>
    }
    flow.dispatch.tensor.store %6, %1, offsets = [%arg0], sizes = [32], strides = [%c1] : tensor<32xi32> -> !flow.dispatch.tensor<writeonly:384xi32>
  }
  return
}

This approach should be more performant than InnerReduction and InnerParallel. "Accuracy" should be on pair with the "InnerReduction" approach since the order in which the elements are reduced is, to some extent, similar. Extra accuracy might be achieved if we enforced horizontal reductions to be computed in sequential order, at the expenses of some performance.
Potential names for this new reduction approach: VectorAccum, (Other suggestions?)
Implementation
Some steps that could help with the implementation:

Extend vector.multireduction operation to perform a reduction of a dimension up to a given vector size. This may require some discussion in MLIR core.
Extend Linalg vectorizer to generate a vector accumulator and a final horizontal reduction for 1-D reductions when the new reduction strategy is used.
Extend Linalg vectorizer support to n-D reductions for the new strategy.
Extend Code Generation Strategy to select this new reduction strategy.
Deprecate InnerReduction strategy?

@MaheshRavishankar, @hanhanW, @nicolasvasilache

Once we agree on the direction for IREE, perhaps I should send part of this to Discourse...

@ThomasRaoux

@vmurali

The transformation suggested seems very similar to doing a split-reduction/reduction tiling transformation?
After tiling the parallel dim you get:
%3 = linalg.generic {indexing_maps = [#map3, #map4], iterator_types = ["parallel", "reduction"]}
  ins(%0 : tensor<384x32xi8>) outs(%2 : tensor<32xi32>) {
  ^bb0(%arg2: i8, %arg3: i32):
    %4 = arith.extsi %arg2 : i8 to i32
    %5 = arith.addi %4, %arg3 : i32
    linalg.yield %5 : i32
  } -> tensor<32xi32>

which then the split reduction would transform it to:
%3 = linalg.generic {indexing_maps = [(d0, d2, d1) -> (d0, d2, d1), (d0, d1, d2) -> (d0, d1)], iterator_types = ["parallel", "parallel", "reduction"]}
  ins(%0 : tensor<16x24x32xi8>) outs(%init_to_zero : tensor<16x32xi32>) {
  ^bb0(%arg2: i8, %arg3: i32):
    %4 = arith.extsi %arg2 : i8 to i32
    %5 = arith.addi %4, %arg3 : i32
    linalg.yield %5 : i32
  } -> tensor<16x32xi32>
%r = linalg.generic {indexing_maps = [...], iterator_types = ["parallel", "reduction"]}
  ins(%3 : tensor<16x32xi32>) outs(%2 : tensor<32xi32>) {
  ^bb0(%arg2: i32, %arg3: i32):
    %5 = arith.addi %arg2, %arg3 : i32
    linalg.yield %5 : i32
  } -> tensor<32xi32>

Now if you tile the reduction dim in the first linalg then vectorize you get the IR you want.
This linalg reduction already exists here:
https://github.com/llvm/llvm-project/blob/29f852a1516bcd3928dad74835965f238de34409/mlir/include/mlir/Dialect/Linalg/Transforms/Transforms.h#L1413
It is a bit limited right now but there are discussions about how to generalize it. This is a transformation we use on GPU in IREE, if we could make the same general transformation work here that's would be great.
Note that in addition to what we already do on the GPU side, we do want to do the same thing as what you describe in some cases. I was thinking about re-using the split reduction linalg transformation for that but I haven't given it that much thoughts.

Can't this happen at the Vector dialect level. Is there a reason to do this in Linalg itself?

Can't this happen at the Vector dialect level. Is there a reason to do this in Linalg itself?

One advantage here is that it allows us to tile the reduction dimension of the first op after splitting which would be hard to do at the vector level.
The other advantage of doing it at linalg is that we get to re-use the existing transformation but that's very little code so I don't think it is a significant reason.

Also not having to change linalg vectorizer is a big advantage in my option. It might be possible to do something at the vector level but not the exact IR described above.

To me just looking at the form of the loop nest generated, it seems like a single loop nest. Having two generic ops seems like having two loop nests adjacent to each other which is not how the generate code looks like.
I do see how using split-k can help here. And this in the backend, so it is much easier to do in IREE. My only concern with split-k is that you need to generalize the linalg.matmul to linalg.generic and then recognize that this is a "+=" during split-k. This is a lowering and a lifting stitched one after the other, that is better avoided, but it will work. Doing this at the vector level is much easier since you have a vector.reduction <add>, so the identity element, etc. are easy to figure out.

To me just looking at the form of the loop nest generated, it seems like a single loop nest. Having two generic ops seems like having two loop nests adjacent to each other which is not how the generate code looks like.

One of the loop is vectorized so this seems to make sense to me.

I do see how using split-k can help here. And this in the backend, so it is much easier to do in IREE. My only concern with split-k is that you need to generalize the linalg.matmul to linalg.generic and then recognize that this is a "+=" during split-k.

I'm not sure how this is related to this case. There is no linalg.matmul in what Diego is trying to solve.

This is a lowering and a lifting stitched one after the other, that is better avoided, but it will work. Doing this at the vector level is much easier since you have a vector.reduction <add>, so the identity element, etc. are easy to figure out.

What part is a lowering and what part is a lifting? I'm failing to see how the IR we want can be generated directly for vector. Could you give a bit more details there?

I'm not sure how this is related to this case. There is no linalg.matmul in what Diego is trying to solve.

Sorry, had diverged in my mind from the original example, so lost track of the original example. Please ignore.

This is a lowering and a lifting stitched one after the other, that is better avoided, but it will work. Doing this at the vector level is much easier since you have a vector.reduction <add>, so the identity element, etc. are easy to figure out.

What part is a lowering and what part is a lifting? I'm failing to see how the IR we want can be generated directly from vector. Could you give a bit more details there?

Since this is not a linalg.matmul there is no lowering (IMO linalg.matmul -> linalg.generic is a lowering, but that is not relevant here). The lifting part is where the neutral element of split-k is found here. I dont know if this kind of thing is avoided when going through vectorization (I would hope it does since vectorization goes op-by-op within the linalg.generic.) That will be limiting cause it wont work for any arbitrary DAG.

This is a lowering and a lifting stitched one after the other, that is better avoided, but it will work. Doing this at the vector level is much easier since you have a vector.reduction <add>, so the identity element, etc. are easy to figure out.

What part is a lowering and what part is a lifting? I'm failing to see how the IR we want can be generated directly from vector. Could you give a bit more details there?

Since this is not a linalg.matmul there is no lowering (IMO linalg.matmul -> linalg.generic is a lowering, but that is not relevant here). The lifting part is where the neutral element of split-k is found here. I dont know if this kind of thing is avoided when going through vectorization (I would hope it does since vectorization goes op-by-op within the linalg.generic.) That will be limiting cause it wont work for any arbitrary DAG.

Finding the neutral element is needed no matter what I believe. In the IR written by Diego it is there as well. To avoid that we would need to either not have a loop around the reduction or peel out the first iteration of the loop. I don't think starting from vector remove this problem.

Finding the neutral element is needed no matter what I believe. In the IR written by Diego it is there as well.

Not sure. It depends on how vectorization proceeds. I would expect that the original IR if you just vectorize it would become
%0 = arith.extsi ... : vector<384x512xi8> to vector<384x512xi32>
%1 = vector.multi_reduction <add>, %0, %arg4 [1] : vector<384x512xi32> to vector<512xi32>

(I am just simplifying here. You can tile the non-reduction dimensions to something that is more convenient, the rest of the discussion refers to how to handle the reduction dimension)
If you start from here, and just implement equivalent of split-k on vector.multi_reduction <add> you know the identity element is 0 : f32. In the split-k transformation  (at least as implemented today), the identity element finder will have to walk through the arith.extsi and get the identity element to be 0 : i8.  I think it is better to do the extsi first, then the rest of the computation is just a reduction in i32. This is inherently more stable than trying to find the identity element for the arith.extsi -> vector.multi_reduction<add> chain (or something more complex). The difference then comes down to what's the type of accumulator preferred, i.e. i8 or i32...
Anyway, there is a choice to be made. @dcaballe my thinking here is that doing this post vectorization layers better, but it depends on how things are sequenced overall. Both approaches can be made to work. The split-K approach can also work, but I am apprehensive about trying to discover the identity element to use based on the analysis of the body of a linalg.generic, as opposed to just anchoring on vector.multi_reduction <add> where you can maybe avoid this. Its your call about which direction to take.

Finding the neutral element is needed no matter what I believe. In the IR written by Diego it is there as well.

Not sure. It depends on how vectorization proceeds. I would expect that the original IR if you just vectorize it would become
%0 = arith.extsi ... : vector<384x512xi8> to vector<384x512xi32>
%1 = vector.multi_reduction <add>, %0, %arg4 [1] : vector<384x512xi32> to vector<512xi32>

(I am just simplifying here. You can tile the non-reduction dimensions to something that is more convenient, the rest of the discussion refers to how to handle the reduction dimension)
If you start from here, and just implement equivalent of split-k on vector.multi_reduction <add> you know the identity element is 0 : f32. In the split-k transformation (at least as implemented today), the identity element finder will have to walk through the arith.extsi and get the identity element to be 0 : i8.

Not really, the split-K transformation would look for the neutral element associated to the the reduction so it would be independent of the extsi as well and it would be 0 : i32

I think it is better to do the extsi first, then the rest of the computation is just a reduction in i32. This is inherently more stable than trying to find the identity element for the arith.extsi -> vector.multi_reduction<add> chain (or something more complex). The difference then comes down to what's the type of accumulator preferred, i.e. i8 or i32...

Yes I think there is no choice here really, the reduction is in i32 so it will have to be that way no matter what.

Anyway, there is a choice to be made. @dcaballe my thinking here is that doing this post vectorization layers better, but it depends on how things are sequenced overall. Both approaches can be made to work. The split-K approach can also work, but I am apprehensive about trying to discover the identity element to use based on the analysis of the body of a linalg.generic, as opposed to just anchoring on vector.multi_reduction <add> where you can maybe avoid this. Its your call about which direction to take.

Based on the IR written above doing it on linalg seems like a more straight forward way. If we unroll everything like in the IR you have above that's a different story but that means we cannot go back and have a reduction loop which I think will be a problem if we want big tiles. The linalg solution would work right now with no new transformation. It would be interesting to see what can be done on vector but I suspect we would want to look more into what the steps in the IR would look like.
I think if we end up having to do changes in the vectorization this is the wrong direction. I'm happy to discuss it if needed.

Not sure. It depends on how vectorization proceeds. I would expect that the original IR if you just vectorize it would become
%0 = arith.extsi ... : vector<384x512xi8> to vector<384x512xi32>
%1 = vector.multi_reduction <add>, %0, %arg4 [1] : vector<384x512xi32> to vector<512xi32>

To clarify the problem I'm talking about with this IR, it is that the while reduction along the dim 384 is fully unrolled. If this value if very large that's problem as the kernel will become very large. I'm not sure how you handle this with vectorizing first.

Awesome! Sorry, I’ve been disconnected from the *-K work. Happy to reuse whatever is there, if it makes sense for this case. Please, give me some time to digest that information. Some initial comments based on the discussion above:

Also not having to change linalg vectorizer is a big advantage in my option. It might be possible to do something at the vector level but not the exact IR described above.

Honestly, I think this is a disadvantage. The Linalg vectorizer is not auto-contained and we currently rely on some pre-processing transformations that are supposed to leave the IR in a state that the vectorizer can digest. Unfortunately, these transformations are applied in isolation and do not compose well. I’m not even sure how the split-k transformation would compose with peeling and masking, for example, where the horizontal reduction should happen depending on whether the peeled loop is vectorized (with masks) or not. The Linalg vectorizer was implemented some time ago and I think it hasn’t evolved with Linalg. Loops have been around Linalg (literally) for quite some time and I think the Linalg vectorizer should take them into consideration. That or implementing a vectorizer where Linalg is just a sub-component of it. This would be a lot of work but we should find a way to incrementally move towards that direction because the level of complexity around the pre-processing transformations is getting a bit out of control, IMO.
I also have some ideas to explore on the reduction field that would make the transformation even more complicated.

To me just looking at the form of the loop nest generated, it seems like a single loop nest. Having two generic ops seems like having two loop nests adjacent to each other which is not how the generate code looks like.

One of the loop is vectorized so this seems to make sense to me.
IIUC, both loops would be vectorized but the latter would have a single vector iteration so the loop would go away. That would work but I think it’s a very indirect way of representing what we want.

In general, this proposal or split-k are not new. They both are basically privatizing a reduction in a parallel execution context. However, the kind of parallelism matters because the requirements are not the same. For thread-level or SIMT-level parallelism I would expect some synchronization to be needed between the two generic ops (?) or perhaps each generic op lands in a separate kernel. Also the resulting partial reductions should be accessible by the threads computing the final reduction (second generic op) so even some memory movement is needed if the partial reductions were computed on a local memory. All these constraints are not present in the SIMD-level scenario. Taking into account the the Linalg vectorizer focuses on SIMD (or SIMD-like, e.g., warps in the future), perhaps making some assumptions and starting from a more specialized and simple representation is worth it.
Again, I’m open to everything. I need to look a bit more into what split-k does and aim for something that aligns with the long term direction.
Thanks for the feedback!
(Btw, I accidentally mixed two examples in my IR above. I’ll fix it tomorrow with a clearer mind :))

Also not having to change linalg vectorizer is a big advantage in my option. It might be possible to do something at the vector level but not the exact IR described above.

Honestly, I think this is a disadvantage. The Linalg vectorizer is not auto-contained and we currently rely on some pre-processing transformations that are supposed to leave the IR in a state that the vectorizer can digest. Unfortunately, these transformations are applied in isolation and do not compose well. I’m not even sure how the split-k transformation would compose with peeling and masking, for example, where the horizontal reduction should happen depending on whether the peeled loop is vectorized (with masks) or not. The Linalg vectorizer was implemented some time ago and I think it hasn’t evolved with Linalg. Loops have been around Linalg (literally) for quite some time and I think the Linalg vectorizer should take them into consideration. That or implementing a vectorizer where Linalg is just a sub-component of it. This would be a lot of work but we should find a way to incrementally move towards that direction because the level of complexity around the pre-processing transformations is getting a bit out of control, IMO.
I also have some ideas to explore on the reduction field that would make the transformation even more complicated.

Interesting, I would be curious to learn about your ideas there. My concern was that the vectorization would become more complicated or opinionated than it is now. The split-k transformation is pretty high level so I don't see why it wouldn't compose.

In general, this proposal or split-k are not new. They both are basically privatizing a reduction in a parallel execution context. However, the kind of parallelism matters because the requirements are not the same. For thread-level or SIMT-level parallelism I would expect some synchronization to be needed between the two generic ops (?) or perhaps each generic op lands in a separate kernel. Also the resulting partial reductions should be accessible by the threads computing the final reduction (second generic op) so even some memory movement is needed if the partial reductions were computed on a local memory. All these constraints are not present in the SIMD-level scenario. Taking into account the the Linalg vectorizer focuses on SIMD (or SIMD-like, e.g., warps in the future), perhaps making some assumptions and starting from a more specialized and simple representation is worth it.

The transformation is happening at the tensor level so there is no concept of synchronization or distribution or memory representation. This all comes later during vectorization and distribution. So the transformation itself should be agonistic to all that. I'm wondering why we would need to special those transformation (including vectorization).
Sorry that was a long thread. Maybe doing a quick brainstorming/white boarding at some point would be more efficient. That would also help me stay in touch with the plans for vectorization. Whatever solution is picked I think we will want to use it for the GPU as well.

We had a discussion offline. The conclusion is that we will discuss this further next Tuesday once @nicolasvasilache is back. As part of this work, we may have to investigate how the split-k approach will compose with peeling + masking transformations. Therefore, let's keep any implementation on hold for now until we reach a conclusion.

Let's keep this issue open to address the missing unrolling so that we don't have to open a new one just for that.

I was planning to file a bug in MLIR

That sounds good, actually. We don't have to fix everything.
How difficult would be to go with vector unrolling approach? Did you report the issue with alloca and bufferization to @matthias-springer? That could be an initial step to solve the unrolling problem.

I thought the issue with alloca was gone. Could you file an issue in IREE and attach a repro if it's still a case?

Alloca gets optimized away in llvm's sroa pass. It remains in our generated
code
On Fri, Oct 14, 2022, 11:14 Han-Chung Wang @.***> wrote:

I thought the issue with alloca was gone. Could you file an issue in IREE
and attach a repro if it's still a case?
—
Reply to this email directly, view it on GitHub
https://github.com/iree-org/iree/issues/9981#issuecomment-1279317789,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABZURMWNUIAMDSRNEPVYPDWDGPHBANCNFSM55JHBD6Q
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

Generating an alloca shouldn't be a problem if it's not causing other issues, again, as an initial step. LLVM vectorizer does that as well. It could be that LLVM is also able to schedule the instructions in the right way so maybe this could be a final solution. It's worth giving it a try. If you could provide the implementation and can take a look at the generated assembly and RISC-V performance.

We don't need an alloca op if all the ops are vectorized. I don't expect an alloca op in the approach, do I miss something? Can we fix it at MLIR level?

An alloca is generated by bufferization but I could be doing the right thing. Proving an example so that @matthias-springer would be very helpful. Worse case, we leave it there if it's not causing any problems. Better case, we could optimize the alloca using some simple mem2reg algorithm. I think we don't have any production quality one in MLIR but there are a bunch of them in downstream implementations out there that perhaps we could reuse, if absolutely needed.

Yes, I agree with you. We should raise the issue and take a look at it. If it's intended, that's fine. The next question would be if the allocation sizes bounded by constants or register size. We don't want the allocation size depends on input operand sizes. Because that can lead to a stack overflow.
My point is that all the operations should be vectorized, and the types between load/store should all be vector types. In this case, we don't have to rely on mem2reg algorithm. The vectors would be mapped to registers directly. That's why I'm surprised that there is an alloca op. It's also worth to put a small repro somewhere. Either in this bug, or a separate bug.

The allocation size solely depends on the (single dimension) reduction
tiling size.
On Fri, Oct 14, 2022, 12:13 Han-Chung Wang @.***> wrote:

Yes, I agree with you. We should raise the issue and take a look at it. If
it's intended, that's fine. The next question would be if the allocation
sizes bounded by constants or register size. We don't want the allocation
size depends on input operand sizes. Because that can lead to a stack
overflow.
My point is that all the operations should be vectorized, and the types
between load/store should all be vector types. In this case, we don't have
to rely on mem2reg algorithm. The vectors would be mapped to registers
directly. That's why I'm surprised that there is an alloca op. It's also
worth to put a small repro somewhere. Either in this bug, or a separate bug.
—
Reply to this email directly, view it on GitHub
https://github.com/iree-org/iree/issues/9981#issuecomment-1279367058,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABZURLWBDISEHRFGDZB5M3WDGWGJANCNFSM55JHBD6Q
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

@MaheshRavishankar had explained why alloca was being generated earlier. From what I understand, bufferization can reuse input buffers only if there's a chain of use-defs from inputs to outputs. If we create two levels of reduction, the first level reduction requires creation of a single dimensional tensor of length equal to reduction tiling size, which is not part of the inputs or outputs. But from what I observe, bufferization pass eliminates the alloca in case there are no other uses of the final reduced tensor in the dispatch, but retains the alloca when there are other uses of the reduced tensor.
Here are the examples @matthias-springer :
Creates alloca:
#map0 = affine_map<(d1,d2,d3) -> (d1,d2,d3)>
#map1 = affine_map<(d1,d2,d3) -> (d1,d2)>
#map2 = affine_map<(d1,d2) -> (d1,d2)>

func.func @split_reduction_pass1(%input: tensor<1024x512x256xi32>) -> tensor<1024x512xi32> {
  %zeros = arith.constant dense<0> : tensor<1024x512xi32>
  %reduced = linalg.generic {indexing_maps = [#map0, #map1], iterator_types = ["parallel", "parallel", "reduction"]} ins(%input : tensor<1024x512x256xi32>) outs(%zeros : tensor<1024x512xi32>) {
  ^bb0(%arg1: i32, %arg2: i32):
    %add = arith.addi %arg1, %arg2 : i32
    linalg.yield %add : i32
  } -> tensor<1024x512xi32>
  %cst_1 = arith.constant 1 : i32
  %init = tensor.empty() : tensor<1024x512xi32>
  %result = linalg.generic {indexing_maps = [#map2, #map2], iterator_types = ["parallel", "parallel"]} ins(%reduced : tensor<1024x512xi32>) outs(%init : tensor<1024x512xi32>) {
  ^bb1(%arg1: i32, %arg2: i32):
    %add = arith.addi %arg1, %cst_1 : i32
    linalg.yield %add : i32
  } -> tensor<1024x512xi32>
  return %result : tensor<1024x512xi32>
}

Passes: https://gist.github.com/ef7d9292e30020026ac85f522e0212e4
Doesn't create alloca:
#map0 = affine_map<(d1,d2,d3) -> (d1,d2,d3)>
#map1 = affine_map<(d1,d2,d3) -> (d1,d2)>
#map2 = affine_map<(d1,d2) -> (d1,d2)>

func.func @split_reduction_pass1(%input: tensor<1024x512x256xi32>) -> tensor<1024x512xi32> {
  %zeros = arith.constant dense<0> : tensor<1024x512xi32>
  %reduced = linalg.generic {indexing_maps = [#map0, #map1], iterator_types = ["parallel", "parallel", "reduction"]} ins(%input : tensor<1024x512x256xi32>) outs(%zeros : tensor<1024x512xi32>) {
  ^bb0(%arg1: i32, %arg2: i32):
    %add = arith.addi %arg1, %arg2 : i32
    linalg.yield %add : i32
  } -> tensor<1024x512xi32>
  return %reduced : tensor<1024x512xi32>
}

Passes: https://gist.github.com/bdc02d7f4bdd0dfb32ed65f76672784c

I created a smaller example showcasing bufferization issue:
iree-opt --pass-pipeline='iree-codegen-iree-comprehensive-bufferize' --split-input-file test.mlir
func.func @test() {
  %cst = arith.constant dense<0> : vector<4xi32>
  %c0 = arith.constant 0 : index
  %c0_i32 = arith.constant 0 : i32
  %load = hal.interface.binding.subspan set(0) binding(1) type(storage_buffer) offset(%c0) alignment(64) : !flow.dispatch.tensor<writeonly:512xi32>
  %tile = flow.dispatch.tensor.load %load, offsets = [0], sizes = [32], strides = [1] : !flow.dispatch.tensor<writeonly:512xi32> -> tensor<32xi32>
  %extracted_slice = tensor.extract_slice %tile[%c0] [4] [1] : tensor<32xi32> to tensor<4xi32>
  %dummy_write = vector.transfer_write %cst, %extracted_slice[%c0] {in_bounds = [true]} : vector<4xi32>, tensor<4xi32>
  %next_init = arith.constant dense<0> : tensor<4xi32>
  %read = vector.transfer_read %next_init[%c0], %c0_i32 {in_bounds = [true]} : tensor<4xi32>, vector<4xi32>
  %res = vector.transfer_write %read, %tile[%c0] {in_bounds = [true]} : vector<4xi32>, tensor<32xi32>
  //%res = tensor.insert_slice %next_init into %tile[%c0] [4] [1] : tensor<4xi32> into tensor<32xi32>
  flow.dispatch.tensor.store %res, %load, offsets = [0], sizes = [32], strides = [1] : tensor<32xi32> -> !flow.dispatch.tensor<writeonly:512xi32>
  return
}

If the alternative %res is uncommented, then it doesn't create an alloca.

Regarding the bufferization issue: The bufferization is currently not aware of which parts of a buffer are accessed. Even if only a small part of the buffer is read/written, we assume for analysis purposes that the entire buffer is read/written. Conceptually, to produce %res, we need access to the original buffer contents of %tile (without the modifications made by %dummy_write). That's why a copy is inserted.
func.func @test() {
  %cst = arith.constant dense<0> : vector<4xi32>
  %c0 = arith.constant 0 : index
  %c0_i32 = arith.constant 0 : i32
  %load = hal.interface.binding.subspan set(0) binding(1) type(storage_buffer) offset(%c0) alignment(64) : !flow.dispatch.tensor<writeonly:512xi32>
  %tile = flow.dispatch.tensor.load %load, offsets = [0], sizes = [32], strides = [1] : !flow.dispatch.tensor<writeonly:512xi32> -> tensor<32xi32>
  %extracted_slice = tensor.extract_slice %tile[%c0] [4] [1] : tensor<32xi32> to tensor<4xi32>

  %dummy_write = vector.transfer_write %cst, %extracted_slice[%c0] {in_bounds = [true]} : vector<4xi32>, tensor<4xi32>
  //                                         ^ READ FROM AND WRITE TO buffer(%tile)

  %next_init = arith.constant dense<0> : tensor<4xi32>
  %read = vector.transfer_read %next_init[%c0], %c0_i32 {in_bounds = [true]} : tensor<4xi32>, vector<4xi32>

  %res = vector.transfer_write %read, %tile[%c0] {in_bounds = [true]} : vector<4xi32>, tensor<32xi32>
  //                                  ^ READ FROM AND WRITE TO buffer(%tile)

  flow.dispatch.tensor.store %res, %load, offsets = [0], sizes = [32], strides = [1] : tensor<32xi32> -> !flow.dispatch.tensor<writeonly:512xi32>
  return
}

Note: The example is a bit odd because %dummy_write is not used anywhere. If you add a tensor.extract %dummy_write or similar, there should be an alloc even in the tensor.insert_slice variant.
We have a slightly better analysis for tensor.insert_slice (to avoid copies when tiling). Because of this we can avoid a copy.
Is this an important test case? How often do we see this pattern? Is this a typical piece of IR after vectorization? I could probably make the analysis around vector.transfer_write a bit smarter.

Side note:
The original test case has a %init = tensor.empty() : tensor<1024x512xi32>. This will always bufferize to an allocation. But looking at the dump, this is not very useful because by the time the IR hit bufferization, the tensor.empty() has already disappeared (and the IR is completely different now).

This is done https://github.com/openxla/iree/pull/10537


[Bug] broker jvm crash

Search before asking

[X] I searched in the issues and found nothing similar.

Version
2.11.1
Minimal reproduce step

It happened several times randomly and is not easy to reproduce. There are multiple brokers.
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f05ac3c07f2, pid=1, tid=652

hs_err_pid1.log
replay_pid1.log
docker_broker-2023-11-11.log
What did you expect to see?
broker crash
What did you see instead?
no crash
Anything else?
No response
Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

Could you provide the dump file ?

Thanks for reporting. Do you have a chance to upgrade to 2.11.2 ? 2.11.1 runs on 17.0.6 which is rather old. Some JVM crashes were fixed in 17.0.7 .
Btw. Probably not related, but I'd recommend setting /sys/kernel/mm/transparent_hugepage/enabled to madvise when running Java on Linux. (-XX:+UseTransparentHugePages can be used with madvise to opt-in to use THP for the JVM when needed). Alternatively -XX:+AlwaysPreTouch jvm startup option could be used to mitigate some issues that THP causes.

Thanks for reporting. Do you have a chance to upgrade to 2.11.2 ? 2.11.1 runs on 17.0.6 which is rather old. Some JVM crashes were fixed in 17.0.7 .
Btw. Probably not related, but I'd recommend setting /sys/kernel/mm/transparent_hugepage/enabled to madvise when running Java on Linux. (-XX:+UseTransparentHugePages can be used with madvise to opt-in to use THP for the JVM when needed). Alternatively -XX:+AlwaysPreTouch jvm startup option could be used to mitigate some issues that THP causes.

Thank you for your reply. After upgrading to 2.11.2, it has not appeared again for a month.


Update functions

[x] refactored wallet and connection  params, pulling them off of the client.provider object
[x] reassigned program var in each function to await client.useProgram()
[x] updated logic in hubs.js to match logic of current working nina-internal-sdk and accommodate magic wallets
[x] updated logic in releaseCreate and releaseInitViaHub in release.js to match logic of current working nina-internal-sdk and accommodate magic wallets
[x] created functions for exchange - exchangeInit(), exchangeAccept() and exchangeCancel()
[x] updated logic to invoke program.methods instead of program.transaction / program.rpc
[x] updated docs to represent function implementation with examples
[x] return objects from each function to the frontend

this pass of updates to the repo focused on creating consistent return values. returning objects instead of strings
ex. return { release: newReleaseData }

merged clean-up branch into update-functions


bug (dont hide image upload instructions in editor on mobile

Closes #202
Describe your changes
On small width, remove left margin before upload instructions to ensure that it is shown.
Before:

After:

Environment(s) tested
Windows 10 Chrome 103, dev tools

Fixed in c47728aaf514d89f3c92d29137b451f58c2c1d9b


Improve sandbox manager API

This PR contains an (in my opinion) improvement of the sandbox API.
The sandbox manager's withSandbox method now requires an array of nodes from the original tree and the corresponding sandbox nodes are now positional arguments of the callback function. E.g.:
// before
sandboxManager.withSandbox(a, (sa) => { ... })
// after
sandboxManager.withSandbox([a], (sa) => { ... })

// before
sandboxManager.withSandbox([a, b], ([sa, sb]) => { ... })
// after
sandboxManager.withSandbox([a, b], (sa, sb) => { ... })

Note that this PR contains a breaking change in the signature of the withSandbox method.
@xaviergonz What do you think? I hope a minor breaking change in this isolated feature is okay.

I don't see anything wrong with the change (the sandbox was your idea after all). I'm curios though, why do you feel the new syntax is better?

I think the new API is more consistent and intuitive because nodes are always passed to sandbox.withSandbox as an array of nodes and the callback function has a corresponding number of arguments. In case of a single node, the array has one element and the callback function has one argument. Before, a single node could be passed in two ways: (1) wrapped as an element and (2) without an array. This was ambiguous, I just thought the new ways is nicer. :slightly_smiling_face:

Did you consider this alternative?
sandboxManager.withSandbox(a, (sa) => { ... })
sandboxManager.withSandbox(a, b, (sa, sb) => { ... })
I think with newer versions of typescript it should be possible to type that correctly

No, I hadn't thought about that. Do you think it's better?

If the main use case is a single node then I'd think so

I've been using multiple nodes quite often actually. But it probably depends on the use case.

Then up to whichever you think the main use case is :)

I think your suggestion may have an advantage: It is clearer that the array is not meant to be a tweaked array but just a collection of nodes. I'll think about it until tomorrow and probably try to get the types working for your suggestion, at least to see whether it's possible.

This is the best I could come up with (using variadic tuple types and labeled tuple elements introduced in Typescript 4.0):
   withSandbox<T extends readonly [object, ...object[]], R = void>(
-    nodes: T,
-    fn: WithSandboxCallback<T, R>
+    ...args: [...nodes: T, fn: WithSandboxCallback<T, R>]
   ): R {
+    const nodes = (args.slice(0, -1) as unknown) as T
+    const fn = (args[args.length - 1] as unknown) as WithSandboxCallback<T, R>
     for (let i = 0; i < nodes.length; i++) {
       assertTweakedObject(nodes[i], `nodes[${i}]`)
     }

Unfortunately, there seems to be no way of destructuring a tuple where the rest argument is at the beginning:
const [...nodes, fn] = args // ERROR: A rest element must be last in a destructuring pattern. ts(2462)

And I couldn't find a way to split nodes and fn in a way that correctly preserves the types. The type casts are a bit ugly, but they are not visible to the user, so it may be okay to do it this way.
Or can you think of a better solution?

Until TS 4.2 arrives (https://github.com/microsoft/TypeScript/pull/41544) I can only think of overloads:
// overloads
function f<N0 extends object>(node0: N0, callback: (node0: N0) => void): void
function f<N0 extends object, N1 extends object>(node0: N0, node1: N1, callback: (node0: N0, node1: N1) => void): void
// and so on until 10 args or so

// base case, type not exported
function f(...args: any[]) {
  const nodes = args.slice(0, args.length - 1);
  const callback = args[args.length - 1];
}

f({x: 10}, (n) => {})
f({x: 10}, {y: 20}, (n, n2) => {})

so which one would you prefer, overloads, then proper support once TS4.2 arrives or just use arrays?

I think I prefer your suggested API. I also noticed it is similar to, e.g., reselect's createSelector signature. But I suggest to wait until Typescript 4.2 has been released instead of temporarily using overloads.

Merged then, thanks! :D


Spelling mistake: Derjiling to Derjeeling in WB state.

correct pronunciation is Darjeeling . It will be great if we correct Darjiling to Darjeeling, thanks :)

https://github.com/covid19india/api/issues/51#issuecomment-607166974


lib/src/models/image.cpp change results in fixing of not-retried "too many requests" for sankaku

I'm not doing a pull request cause it's so damn hacky but:
void Image::parseDetails()
{
	m_loadingDetails = false;

	// Aborted or connection error
	const int statusCode = m_loadDetails->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();

	if (m_loadDetails->error()) {
		if (m_loadDetails->error() != QNetworkReply::OperationCanceledError) {
            if (statusCode == 429) {
                log(QStringLiteral("Request limit reached (429). New try."));
                loadDetails(true);
                return;
            }
            log(QStringLiteral("Loading error for '%1': %2").arg(m_pageUrl.toString(), m_loadDetails->errorString()), Logger::Error);
		}
		m_loadDetails->deleteLater();
		m_loadDetails = nullptr;
		return;
	}

	// Check redirection
	QUrl redir = m_loadDetails->attribute(QNetworkRequest::RedirectionTargetAttribute).toUrl();
	if (!redir.isEmpty()) {
		m_pageUrl = redir;
		loadDetails();
		return;
	}

	if (statusCode == 429) {
		log(QStringLiteral("Details limit reached (429). New try."));
		loadDetails(true);
		return;
	}

Fixes the retrying of images that were otherwise stuck, not using the "retry error" timeout, when sankakuchannel reported a 429 too many requests error.
original code, as per commit 016731941d29526d7f8ee3a5afd05163bb3fec36
void Image::parseDetails()
{
	m_loadingDetails = false;

	// Aborted or connection error
	if (m_loadDetails->error()) {
		if (m_loadDetails->error() != QNetworkReply::OperationCanceledError) {
			log(QStringLiteral("Loading error for '%1': %2").arg(m_pageUrl.toString(), m_loadDetails->errorString()), Logger::Error);
		}
		m_loadDetails->deleteLater();
		m_loadDetails = nullptr;
		return;
	}

	// Check redirection
	QUrl redir = m_loadDetails->attribute(QNetworkRequest::RedirectionTargetAttribute).toUrl();
	if (!redir.isEmpty()) {
		m_pageUrl = redir;
		loadDetails();
		return;
	}

	const int statusCode = m_loadDetails->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();
	if (statusCode == 429) {
		log(QStringLiteral("Details limit reached (429). New try."));
		loadDetails(true);
		return;
	}

In short, move the declaration of "const int statusCode = m_loadDetails->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();" to the top, and use it to double check for "aborted or connection error" at the beginning.
I don't know the codebase well enough to know how robust this is, hence no pull request, and only a code example. It does, however, fix the "stuck download" issue I had (but did not report, lol) when sankaku threw a 429. The file would just be stuck forever until I manually skipped it.
Now it's a fire and forget, even if it takes a while due to the somewhat high retry timeout I set.

I feel like simply moving the error check after the redirection and 429 code checks does the same thing without duplicating the code.
Like so:
// Check redirection
QUrl redir = m_loadDetails->attribute(QNetworkRequest::RedirectionTargetAttribute).toUrl();
if (!redir.isEmpty()) {
    m_pageUrl = redir;
    loadDetails();
    return;
}

const int statusCode = m_loadDetails->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();
if (statusCode == 429) {
    log(QStringLiteral("Details limit reached (429). New try."));
    loadDetails(true);
    return;
}

// Aborted or connection error
if (m_loadDetails->error()) {
    if (m_loadDetails->error() != QNetworkReply::OperationCanceledError) {
        log(QStringLiteral("Loading error for '%1': %2").arg(m_pageUrl.toString(), m_loadDetails->errorString()), Logger::Error);
    }
    m_loadDetails->deleteLater();
    m_loadDetails = nullptr;
    return;
}

So that's what I'll merge, unless there's an issue with this method? 🤔

It works, thanks~


Add text role implementations

https://docutils.sourceforge.io/docs/ref/rst/roles.html

I am wondering if we can somehow use the work I put into https://github.com/doctrine/rst-parser/pull/257 to solve this

I think we can use parts of it, I replaced the full SpanParser to be able to split the parser into a lexing and parsing part.
The lexer will break the text into smaller elements, called Tokens. The parser will consume those tokens to turn them into what I now named SpanToken But a better name would be InlineNodes as that refers to the inline markup that is supported by RST.
I would like to introduce more of these kinds of SpanToken, the text roles you created could be just that! We could extend the rendering of the span to transform it completely into what we have now for nodes. So we can use the same mechanism as we have for other node types.


Set brightness

builtins.PlugEnergyMonitoringHandler' object has no attribute 'set_brightness'
I am getting this error in here, what could be the reason?

import asyncio
import os

from tapo import ApiClient

async def main():
    client = ApiClient("xd", "xd")
    device = await client.p110("<IP_ADDRESS>")

    while True:
        user_input = input("Enter 'o' to turn the device on or 'c' to turn it off (or 'q' to quit): ")

        if user_input == 'o':
            await device.on()
            device_usage = await device.get_device_usage()
            print("Işık Açıldı, bilgiler:")
            print(f"Device usage: {device_usage.to_dict()}")
        elif user_input == 'c':
            await device.set_brightness('30')
            print("Işık kapatıldı.")

        elif user_input == 'q':
            break
        else:
            print("Invalid input. Please enter 'o' or 'c'.")

if __name__ == "__main__":
    asyncio.run(main())

i also wonder if there is a way to look for device state?

set_brightness is not found because you are configuring a P110 device.
Change
device = await client.p110("<IP_ADDRESS>")

to
device = await client.l510("<IP_ADDRESS>")

The lightbulb example contains information about how to get the device info.

omg thank you so much!! how can i miss that, and i also wonder if there is a way to look for device state? whether is on or off?

Thank you so much!

Hi again, thank you so much,  i think i kinda figured it out but i am getting this error now:
Serde(Error("missing field type", line: 1, column: 742)) in here   device_info = await device.get_device_info()
in this code:```
import asyncio
import os
import PySimpleGUI as sg
from tapo import ApiClient
async def main():
client = ApiClient("xd", "xd")
device = await client.l510("<IP_ADDRESS>")
device_info = await device.get_device_info()
layout = [
[sg.Text("Işık Kontrolcüsü")],
[
sg.Button("Aç", key="ON"),
sg.Button("Kapat", key="OFF"),
sg.Button("Çıkış"),
],
[sg.Multiline("", size=(40, 10), key="CIHAZ_BILGILERI")],
[sg.Text("Cihaz Durumu: "), sg.Text("OFF", key="STATUS")],
[sg.Text("0-100: "), sg.InputText(key="BRIGHTNESS")],
[sg.Button("Parlaklık Ayarla", key="SET_BRIGHTNESS")],
]
window = sg.Window("Işık", layout)
while True:
event, values = window.read()
# Handle button events
if event == "ON":
  if device_info.device_on == False:
    await device.on()
    window["CIHAZ_BILGILERI"].update("Device is off. Turning it on...")
    device_info = await device.get_device_info()
  else:
    window["CIHAZ_BILGILERI"].update("Device is already on.")
elif event == "OFF":
  if device_info.device_on == True:
    await device.off()
    window["CIHAZ_BILGILERI"].update("Device is on. Turning it off...")
    device_info = await device.get_device_info()
  else:
    window["CIHAZ_BILGILERI"].update("Device is already off.")
elif event == "SET_BRIGHTNESS":
  brightness = int(values["BRIGHTNESS"])
  await device.set_brightness(brightness)
  device_usage = await device.get_device_usage()
  window["CIHAZ_BILGILERI"].update(
    f"Parlaklık ayarlandı, kullanım bilgileri:\n{device_usage.to_dict()}"
  )
elif event == "Çıkış" or event == sg.WIN_CLOSED:
  break

window.close()
if name == "main":
asyncio.run(main())

What's the device that you're using?
Can you use get_device_info_json instead of get_device_info and paste here the output?

get_device_info_json  this here worked! thanks again the device is l510, I think I should read the source code more :D

Can you post the output of get_device_info_json please? I'd like to understand what went from with get_device_info for you.

yes sure, this is my output for get device info json
{'avatar': 'bulb', 'brightness': 100, 'color_temp_range': [2700, 2700], 'default_states': {'brightness': {'type': 'last_states', 'value': 100}, 're_power_type': 'always_on'}, 'device_id': '80235B0BA056116528E8D5FF817F3EF2210F8F62', 'device_on': True, 'fw_id': '00000000000000000000000000000000', 'fw_ver': '1.1.0 Build 230721 Rel.224802', 'has_set_location_info': True, 'hw_id': 'FDE1C68674D1535B12A042682B192E4E', 'hw_ver': '2.0', 'ip': '<IP_ADDRESS>', 'lang': 'tr_TR', 'latitude': 397856, 'longitude': 305033, 'mac': '48-22-54-F5-92-23', 'model': 'L510', 'nickname': 'TWVydCBvZGEg', 'oem_id': '0BE00E1E2C6CAE2A732FCA44CC9A5D59', 'overheated': False, 'region': 'Europe/Istanbul', 'rssi': -56, 'signal_level': 2, 'specs': '', 'ssid': 'bW1kMjYyNi0x', 'time_diff': 180, 'type': 'SMART.TAPOBULB'}

Oh wow, the value of default_states is totally different than what I was expecting.
It must be a L510 thing because I couldn't replicate it with L530.

Wow this is weird then, what are the usual values?

same issue w/ a newer hardware version (some fields have been redacted)
Object` {"avatar": String("bulb"), "brightness": Number(20), "color_temp_range": Array [Number(2700), Number(2700)], "default_states": Object {"brightness": Object {"type": String("last_states"), "value": Number(20)}, "re_power_type": String("always_on")}, "device_id": String(".........."), "device_on": Bool(true), "fw_id": String("00000000000000000000000000000000"), "fw_ver": String("1.1.2 Build 231120 Rel.201048"), "has_set_location_info": Bool(false), "hw_id": String(".........."), "hw_ver": String("3.0"), "ip": String("<IP_ADDRESS>"), "lang": String(""), "latitude": Number(-1879048193), "longitude": Number(-1879048193), "mac": String(".........."), "model": String("L510"), "music_rhythm_enable": Bool(false), "music_rhythm_mode": String("single_lamp"), "nickname": String("QnVsYiBEZXY="), "oem_id": String(".........."), "overheated": Bool(false), "region": String("Europe/Berlin"), "rssi": Number(-50), "signal_level": Number(2), "specs": String(""), "ssid": String(".........."), "time_diff": Number(60), "type": String("SMART.TAPOBULB")}

Thank you for confirming a second instance.
It must be a change in the L510/L610 API. I was not able to replicate it with L530.
Unfortunately, I do not own a L510 or L610 device.
To apply a fix, I need to know more about all the possible values that default_states could contain.

the json structure is as following (L510, L610):
"default_states": Object {
    "brightness": Object {
        "type": String(),
        "value": Number(),
    },
    "re_power_type": String(),
},

type: custom, last_states
value: 1..100 (including 100)
re_power_type: always_on, last_states

Thank you for the help.
The fix has been released in Tapo Rust v0.7.7 and Tapo Python v0.1.4.


Error when using git-town ship

[master] git merge --squash somebranch
fatal: You cannot combine --squash with --no-ff.
2018/01/20 23:54:18 Error squash merging: exit status 128

This happens every time, no matter what the project is. What am I doing wrong?

Confirmed this happens when a user has the merge.ff option set to false. I think this is a bug in git as that should not be added when squash merging

Found a discussion of the bug in the git mailing list<EMAIL_ADDRESS>

Also confirmed this is still happening on git 2.17.0 even though the discussion is quite old

That same email thread suggests providing --ff to the squash command in order to override the Git configuration, which shouldn't apply in this case anyways. So, this squash command should work in all cases:
git merge --squash --ff <feature branch name>


Improved concurrency performance

https://github.com/polkadot-js/api/issues/4707

I think we can also do some caching for apiAt for the latest 10 blocks, since most queries ask for these blocks. The implementation should be pretty similar to how we cache tx hashes

so basically if many request comes in, they can share the same instance of apiAt and won't need to make many new connections. But still, this is a workaround we can do, it will definitely be nice to improve concurrency performance from the api itself

I think we can also do some caching for apiAt for the latest 10 blocks in RPC adaptor, since most queries ask for these blocks. The implementation should be pretty similar to how we cache tx hashes

Yes, the api has fixed some of the problems. But it still creates a new api object for each blockhash, which can cause a lot of memory consumption and service crash when processing many requests at the same time. I will create a block registry based cache.


KAFKA-2807: Move ThroughputThrottler back to tools jar to fix upgrade tests.

LGTM. Merging to trunk only.


V-optimal hist error

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Before submitting (checklist)

[ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[ ] Did you read the contribution guide?
[ ] Did you check the code style? make lint (poetry install -E style).
[ ] Did you make sure to update the docs? We use Numpy format for all the methods and classes.
[ ] Did you write any new necessary tests?
[ ] Did you check that your code passes the unit tests pytest tests/ ?
[ ] Did you add your new functionality to the docs?
[ ] Did you update the CHANGELOG?

Type of Change

[ ] Examples / docs / tutorials / contributors update
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Improvement (non-breaking change which improves an existing feature)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Proposed Changes

Related Issue

PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Closing issues
Put closes #XXXX in your comment to auto-close the issue that your PR fixes (if such).

Codecov Report

Merging #79 (c9de384) into master (d152d9a) will decrease coverage by 5.59%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master      #79      +/-   ##
==========================================
- Coverage   90.14%   84.55%   -5.60%
==========================================
  Files          48       49       +1
  Lines        2152     2182      +30
==========================================
- Hits         1940     1845      -95
- Misses        212      337     +125

Impacted Files
Coverage Δ

etna/analysis/outliers/hist_outliers.py
0.00% <0.00%> (ø)

etna/transforms/pytorch_forecasting.py
26.38% <0.00%> (-73.62%)
:arrow_down:

etna/models/nn/tft.py
43.58% <0.00%> (-56.42%)
:arrow_down:

etna/models/nn/deepar.py
45.94% <0.00%> (-54.06%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d152d9a...c9de384. Read the comment docs.

how about tests? 🙂

Add docs for methods please


No space in my 4GB sd card

Hi,
So I was pretty happy that it worked till part 3 and  want to complete the last steps.
But I run out of space ( downloaded games) and now I ordered a new micro sd card 32GB.
Can someone tell me what are the steps to safely transfer everything from my old sd card( until part 3) to the new one to continue the process?(+the hidden partition which I don't know how to move it)
Thank you!
(Plailect, thank you for everything again)

This will take some time to do if you want to keep your games save data.

Download and install JKSV
Backup games save data by selecting SD/CIA, then the game you want to backup, then export save data(and ext data if you want to)
Delete the games as you back them up to free space.
Make a RedNand backup on 4 GB
Make a RedNand on 32 GB
Copy emunand.bin to 32 GB
Restore RedNand from emunand.bin on the 32 gb

Steps 4-7 where a modification from @maorninja so thanks him :3
If you don't understand by Make a RedNand on 32 GB or Restore RedNand from emunand.bin on the 32 gb tell me and ill tell you how.

thank you!My games are new so I don't care If I lose my saves I can restart them again :-)
So I backup rednand with decrypt9,make a new one as done in part 3,I copy the backup of rednand I made(emunand.bin) and I restore it to my new sd card?
what about sysNAND?I simply drag and drop everything to the new sd card?

Yes first drag and drop everything to the other sd then restore the RedNand from the back up. I hope this helped! Also you will need to reinstall menuhax

On Jun 28, 2016, at 11:15 AM, PrometheusG<EMAIL_ADDRESS>wrote:
thank you!My games are new so I don't care If I lose my saves I can restart them again :-)
So I backup rednand with decrypt9,make a new one as done in part 3,I copy the backup of rednand I made(emunand.bin) and I restore it to my new sd card?
what about sysNAND?I simply drag and drop everything to the new sd card?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

thanks!I am just backing up redNAND and then sysNAND(just to have an extra sysNAND backup) as the guide 9.2.0 update says from steps 18-25.
I don't understand  the step 5 you write above though,Can't I just simply restore from emuNAND.bin?

Ok this is how you make a RedNAND on the new sd card and restore from it

Keep the 4gb sd card in the 3ds
Boot EmuNAND9 in the HomeBrew channel
Click Complete RedNAND Setup ALL DATA WILL BE ERASED FROM THE NEW SD CARD!
Take out your 4gb sd card and put in your 32gb one
Confirm you want to RedNAND format it.
Wait for it to finish
Go back to the main menu of EmuNAND9 and shut down your 3ds(Do not reboot!)
Copy all of the folders from the 4gb sd card to the 32gb sd card Including the emunand.bin
Boot your 3ds with the new 32gb sd card in the 3ds
Open the HomeBrew channel
Boot decrypt9
EmuNAND Options > EmuNAND Backup/Restore > Emunand Restore > restore from emunand.bin
Wait for it to finish
Reboot and get back to the Home Brew Channel
Boot luma
Cross fingers
If it boots into the home menu it worked! If it didn't tell me what happened when you tried to boot luma/what you got confused on.
Hoped this helped!

Also, if you got the new EmuNAND9/Decrypt9 you will need emunand.bin.sha

Thank you very much for you time!
I think I understand now how to do it,yes I have the new Decrypt9(started the whole process 2 days ago) thanks for telling me to have the bin.sha too  :-)
My sd card will be here tomorrow so I can't wait to try it.
One last question:sysNAND will be in version 9.2.0-20E as it was?I am not restoring any sysNAND.bin backup,I just drag and drop all the files to the new sd card
Again thank you

Your SysNAND is not on your sd card look at this amazing diagram not made by me(I would link to whoever made it but I don't know who)

So yes your SysNAND will say in 9.2 as long as you don't update.
Also, if you got any more questions just ask I'm here almost all day long :ghost:

@nickthegamer5: You've been doing an absolutely stellar job replying to questions like this, thank you very much :D

No problem I got nothing to do all day so I enjoy doing this :3

I am so noob, thank you for all!I will re-read the whole guide again and maybe search all the "new words" I am seeing here.
THank again!

If you don't care about your games/saves that are on you 3ds(Not the card ones, once you finish the guide you can dump the cards and install along with transfer the save data(Wich means no more lugging around 10 game cards)) you can redo the whole guide with the new sd but just skip the downgrading part.

So since I am already on sysNAND 9.2.0-20E ,(upgraded from 7.2.0-12E with plailect's guide-9.2.0 update) you say that I can just remove the old sd card, since I don't care about the installed games,make it like it never happened and do all the steps again from the start with new 32gb sd card?
Nice,thank you again!

Yes once your done the guide.

what do you mean?I am ok if i just do this?

It's ok as long as it's related with the guide(like this issue) but once you follow the instruction I posted and finish the guide you can just use the 4gb card anywhere else.

you mean the plailect's guide?So I first follow your instructions to move to the new 32gb sd card and then finish part 4 and 5 from plailect's guide to get arm9loaderhoax?

Yes also make sure to update the emunand NOT SYSNAND and then just finish the guide

On Jun 28, 2016, at 7:18 PM, PrometheusG<EMAIL_ADDRESS>wrote:
you mean the plailect's guide?So I first follow your instructions to move to the new 32gb sd card and then finish part 4 and 5 from plailect's guide to get arm9loaderhoax?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

My RedNAND is now on emu 11.0.0-33E, I am happy I didn't screw this up :-P
I can't thank you enough for your help.I hope one day I will help others too :-)
This is my last post
Thank you again and have a good night!
(and thanks again Plailect for your guide)

I just got my new sd card,followed your instructions step-by-step but there is one "problem".
Decrypt9 is not in my homebrew and maybe I have to access it via my browser on sysNAND.what shall I do?

Download the .3dsx version of decrypt9

On Jun 29, 2016, at 10:57 AM, PrometheusG<EMAIL_ADDRESS>wrote:
I just got my new sd card,followed your instructions step-by-step but there is one "problem".
Decrypt9 is not in my homebrew and maybe I have to access it via my browser on sysNAND.what shall I do?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

So I Delete the Decrypt9WIP.dat+Launcher.dat+Decrypt9.log from my sd card and put the .3dsx from the Decrypt9 folder of the zip file, into the root of my sd card?
I was using the old Decrypt9WIP 20160607-215224.zip .Is there a problem if I just drag and drop the new .3dsx version from the new zip file?(20160626-231556)

Ok download a new decrypt9, there should a a 3ds folder in there open it, now with your sd card in the PC open the 3ds folder on the sd card and drag in the .3dsx and .shmd(something like that can't remember rn) in to the 3ds folder. From there boot the HB channel and open Decrypt9

On Jun 29, 2016, at 11:05 AM, PrometheusG<EMAIL_ADDRESS>wrote:
So I Delete the Decrypt9WIP.dat+Launcher.dat+Decrypt9.log from my sd card and put the .3dsx from the Decrypt9 folder of the zip file, into the root of my sd card?
I was using the old Decrypt9WIP 20160607-215224.zip .Is there a problem if I just drag and drop the new .3dsx version from the new zip file?(20160626-231556)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

It worked!Thank you so much for helping me!
I will go finish the rest of Plailect's Guide but I wouldn't be able to move to the next part (4) if you were not here to help me!
Thank you again!
(sorry Plailect for continuing the topic you closed but I didn't want to open a new "issue")

People usually continue posting there issues even after its closed so don't worry :3

On Jun 29, 2016, at 11:45 AM, PrometheusG<EMAIL_ADDRESS>wrote:
It worked!Thank you so much for helping me!
I will go finish the rest of Plailect's Guide but I wouldn't be able to move to the next part (4) if you were not here to help me!
Thank you again!
(sorry Plailect for continuing the topic you closed but I didn't want to open a new "issue")
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

If it works please tell me so if another person asks I can show them this.

yes, your instructions worked for me!
I have put also the.3dsx and .smdh files from the new Decrypt9WIP.zip to my 3ds folder and left Decrypt9WIP.dat+Launcher.dat+Decrypt9.log that I used for Plailect's Guide untouched.I succesfully installed a new sd card thanks to you without even losing my game saves or bricking RedNAND.(or even sysNAND) :-)
I am going to make another backup of my entire new SD Card + sysNAND + RedNAND just in case :-)
Again, thank you

Thanks for telling me that it works :3

So glad I found this page, I'm having the same issue as PrometheusG. I'm stuck at the CTRTransfer page of Pailect's guide, unable to create a sysNAND backup due to lack of space on my SD. I'm trying to RedNAND my new SD card, but EmuNAND9 isn't working. When I try to start it up, the bottom screen on my 2DS flashes red twice and then returns me to the Homebrew Launcher.
I found somewhere that you couldn't run EmuNAND9 from within CFW, and that if you boot it outside CFW it should work. Others confirmed this, but I have no idea how to boot outside the CFW unless I totally undo my work thus far....
Sorry I'm such a noob with this. Could anyone help me with getting started?

In what part of the guide are you? My issue was that I didn't know how to move RedNAND,which I had already created, to a new SD card to continue with the old guide, an "issue" that was solved with the help of nickthegamer5.What is your problem exactly? You don't need RedNAND with the new guide,you can start with copying your files to a new sd card to finish the ctrtransfer backup and then install Luma+arm9loaderhax . You can later boot emunand as a payload if you want RedNAND

Thanks for responding! Your response makes me think perhaps there's an easier way...
Essentially, I got to this part of Plailect's guide (in Dekrypt9):

Go to “SysNAND Options”, then “CTRNAND Transfer”, then “Auto CTRNAND Transfer”
Input the key combo given, then select the 2.1.0 ctrtransfer image when prompted by pressing (A)
If you do not already have a NAND backup, backup SysNAND to NANDmin.bin when prompted by pressing (A)

When I pressed "A," it showed up that I didn't have enough space on my SD card to create my NAND backup. I just assumed I'd have to do some sort of special formatting or something to swap out SD cards, so I went searching and found this thread. I downloaded EmuNAND9 and tried to go from there with creating a RedNAND on a new card but EmuNAND9 won't open.
Is there an easier way? Could you expand upon the last couple sentences from your response above?
Thanks!

Actually...I may not need to, I think I found a way. I'll post if I run into another issue related to this, but I think I worked it out. Thank you for your response, though!

RedNAND takes space from your SD card because it is installed in a hidden partition and won't help you with your problem.the easiest way is to copy and paste all your files from your sd card to a bigger one and continue with the guide without creating a RedNAND.follow the guide from where you left off and you won't have any problems.don't forget to post if you have any other issues or if something you tried worked :-)


chore: update smithy-protocol-test-traits from 1.45.0 to 1.46.0

About this PR
📦 Updates software.amazon.smithy:smithy-protocol-test-traits from 1.45.0 to 1.46.0
📜 GitHub Release Notes - Changelog - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "software.amazon.smithy", artifactId = "smithy-protocol-test-traits" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "software.amazon.smithy", artifactId = "smithy-protocol-test-traits" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #268.


prompt_embeds and negative_prompt_embeds must have the same shape when passed directly, but got: prompt_embeds torch.Size([1, 452, 4096]) != negative_prompt_embeds torch.Size([1, 226, 4096]).

Error occurred when executing CogVideoSampler:
prompt_embeds and negative_prompt_embeds must have the same shape when passed directly, but got: prompt_embeds torch.Size([1, 452, 4096]) != negative_prompt_embeds torch.Size([1, 226, 4096]).
File "/data/ComfyUI/execution.py", line 317, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/data/ComfyUI/execution.py", line 192, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/data/ComfyUI/execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "/data/ComfyUI/execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
File "/data/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper/nodes.py", line 289, in process
latents = pipeline["pipe"](
File "/etc/anaconda3/envs/comfyui/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/data/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper/pipeline_cogvideox.py", line 391, in call
self.check_inputs(
File "/data/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper/pipeline_cogvideox.py", line 225, in check_inputs

diffusers==0.30.1

The prompt is probably just too long.

The prompt is probably just too long.

tks, It is really too long. Is there any way to use long prompt words?

Here are my hint words，1047 characters only
In the blue underwater world, an elegant turtle leisurely shuttles between coral reefs. The turtle's shell is covered with exquisite textures, and the sun shines through the water with mottled light and shadow, making its shell shine with a mysterious luster. The turtle swings its flippers slowly, light and graceful, as if soaring through the water. The underwater world around is colorful, and corals of different shapes are like sculptures of nature, decorating this peaceful ocean paradise. Colorful tropical fish swam in groups among the corals. Some fish stopped beside the turtle as if to say hello to it. The whole scene was full of vigor and vitality. In the distance, the dark blue water gradually became dark and mysterious, as if hiding countless undiscovered secrets. And the turtle is so calm and comfortable in this vast underwater world, as if it is the guardian of this ocean. The whole picture shows a quiet and harmonious natural beauty with soft colors and smooth lines, showing the fantastic world in the depths of the ocean.

226 tokens ，I found

226 tokens ，I found

Yes, I think it's best to stay within that limit. I did modify the code a bit to allow it to run with the longer prompt too, but the results seem worse, if you update you can test it yourself.

226 tokens ，I found

Yes, I think it's best to stay within that limit. I did modify the code a bit to allow it to run with the longer prompt too, but the results seem worse, if you update you can test it yourself.

tks，bro


Handle user information in URI

Without this change encoding an URL like
<EMAIL_ADDRESS>results in
git+ssh://git%40github.com/Microsoft/vscode.git.

@keegancsmith,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

Thanks


Columns with tabular data

Please always provide the GitHub issue(s) your PR is for, as well as test URLs where your change can be observed (before and after):
Fix #67
Test URLs:

After: https://feat-features--bitdefender--hlxsites.hlx.page/drafts/scoman/features

@rhudea , I think you are right with authoring. Best experience would be with h4. However, how do we author the tags in this case?  would no longer work. I've tried  with italics in it however it is not rendered as .
In this case we should use a convention for tags as implemented in https://github.com/hlxsites/bitdefender/pull/102

@rhudea , I think you are right with authoring. Best experience would be with h4. However, how do we author the tags in this case? \<em\> would no longer work. I've tried <h4> with italics in it however it is not rendered as \<em\>.
In this case we should use a convention for tags as implemented in #102. In this case how do we escape [tag] in the text or do we want to escape it at all?

Maybe it makes sense to drop support for tags in this PR and add it only as part of #102, clarifying there the authoring experience.


synchronize language with store

sessionStorage, localStorage and save to file

done
this part of the code is pretty bad, not scaling, just spaghetti - but hey, it works


chore(node): try @2060.io/ffi-napi fork as alternative to original ffi-napi

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.29%. Comparing base (969c177) to head (d3ce0ce).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #255   +/-   ##
=======================================
  Coverage   82.29%   82.29%
=======================================
  Files          11       11
  Lines        3152     3152
=======================================
  Hits         2594     2594
  Misses        558      558

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Withdrawals processor delete empty account

PR description
This fixes an issue that was found in the withdrawal reference tests where we update an account with a balance of zero. Ensure that the withdrawals processor clears empty accounts so that we conform to eip-158.
Fixed Issue(s)

Documentation

[x] I thought about documentation and added the doc-change-required label to this PR if
updates are required.

Changelog

[x] I thought about the changelog and included a changelog update if required.

Needs a javadoc
/home/runner/work/besu/besu/evm/src/main/java/org/hyperledger/besu/evm/worldstate/WorldUpdater.java:150: warning: no comment
default void clearAccountsThatAreEmpty() {

Task :evm:javadoc
^

@jframe update your branch and enjoy the 5 minute build time!


SSL connection

Hi koshimas.
I tried to migrated some personal scripts to kysely and noticed that SSL is not support yet.
I'm using postgres on Heroku, and they require connect over SSL.
It would be nice we could pass more options to node-pg https://node-postgres.com/features/ssl

Thanks! Added in the latest version.

https://github.com/koskimas/kysely/commit/90dd9e03328ccfe222caa79862c8d8f066747725


Multiple start URLs including sitemap.xml

I have a site that has ajax pagination but I don't want to drag PhantomJS into this. Luckily the sitemap has been set up to have all the links to the articles I need to crawl. However I couldn't get sitemap.xml set up with the following config. It bounds a sitemap.xml not found error.
<sitemapResolverFactory class="$sitemapFactory" ignore="false" lenient="true">
  <tempDir>$workdir/sitemaps</tempDir>
</sitemapResolverFactory>

Alternatively I set up sitemapResolverFactory ignore="true" and add the sitemap as a startURL.
<startURLs stayOnDomain="true" stayOnPort="true" stayOnProtocol="true">
   <url>${my-url}</url>
   <url>${my-url}/sitemap.xml</url>
</startURLs>

But it seems Norconex ignores the URLs in the sitemap file. So I changed it to:
<startURLs stayOnDomain="true" stayOnPort="true" stayOnProtocol="true">
   <url>${my-url}</url>
   <sitemap>${my-url}/sitemap.xml</sitemap>
</startURLs>

Now the base URL seems to be ignored and links in the pages are not picked up. Is there any way I can make both base URL and sitemap work simultaneously?

They should both be supported.  Is the base URL having the ajax pagination?  If so, that would explain why the links for that one are not picked up.  Another reason is if the URLs are the same between the sitemap and base URL, they will not be processed twice.  Can you share your full config?

I was mistakenly looking into different URLs. I can confirm that it supports both urls and sitemaps; so the third config option works. Thank you as always!


Visibility widget needs to support image services

Widget
Visibility Widget
Version of widget
1.0
Bug or Enhancement
Visibility widget needs to support image services.
Repo Steps or Enhancement details

This issue was moved to Esri/visibility-addin-dotnet#227


Extend sort-order, add option to sort specific mixins by name

Example:
{ "sort-order": [ [ "$variable" ], [ "$include" ],  [ "$include foo-mixin" ],[ "top", "padding" ] ,  [ "$include bar-mixin" ]] }

Some mixins are useful to place after all declarations at the bottom of the rule.

I've found the use case like this:
.module {
    color: red;

    .child {
        color: blue;
    }

    @include bp(m) {
        color: yellow;

        .child {
            color: green;
        }
    }
}

Which is exported out as:
.module {
    color: red;

    @include bp(m) {
        color: yellow;

        .child {
            color: green;
        }
    }

    .child {
        color: blue;
    }
}

Which makes the child always blue and not green at the m breakpoint like I would expect.
Ideally I would love to be able to sort the selectors like:
property, @include (except bp), children, @include bp

+1
I use a similar breakpoint mixin in SCSS which is ideally placed at the bottom, while other includes are best placed at the top so it's possible to override them if needed. I agree with dkeeghan and would love to be able to sort like: property, @include (except bp), children, @include bp

+1
This is a must have since sass has poor support to extend the @media syntax. @media #{breakpoints()} {} is not valid for example. This requires you to use a mixin for your breakpoints if you want to extend it's functionality, such as respond / respond-to / range mixins do. To combat this shortcoming a setting to leave alone mixins would be helpful — or maybe configure what mixins get positioned where. I am thinking about doing it myself in a fork, but still need to figure out the source.

I've made similar changes and split $include keyword to $include (@include rule), $exntend (@exntend) and $include-content (@include { @content}). The last one is useful for sorting @media mixins that have @content as parameter.
https://github.com/denisborovikov/csscomb.js/commit/7c1214deecf5da1b4634bf789d4ada9a849b95c2

Hey @denisborovikov, nice addition. I like your implementation, although I don't think it's enough to group mixins which use @content. My commit shows you can use the following syntax:
 "sort-order": [
        [
            "$include"
        ],
        ...
        [
          "$include respond-to",
          "$include media"
        ],

For a lot of use cases your implementation is absolutely awesome. However, I would rather have a bigger amount of power of the order.
Possible cons of @denisborovikov approach which come to mind:

What about the order of the different @content based mixins? We're using multiple types of media generating mixins.
It could be desirable to have certain @content based mixins at the top of your order.
I understand it is mainly aimed at respond based mixins, but what about different use cases.
In some projects we are using offset mixins to improve our margins/paddings, which contain media queries with no @content but should be placed at the bottom.

Possible cons of my approach which come to mind

Possibly more configuration (although real world usage would probably mean using about 1-3 rules)

I hope we can come with the best solution, it would be great to have this in the master.

@jeroenransijn  Now I like you implementation more than mine. One more cons: you can split different $include by empty line. In my case they are goes with no spaces and now there is no option to add new lines between them.

@jeroenransijn I'd happy to see your implementation working with mixins parameters similar to https://github.com/csscomb/csscomb.js/issues/356#issuecomment-92327556

+1
Need this to support my preferred sorting of breakpoint related mixins to bottom.

@denisborovikov I don't think CSSComb should sort order for mixins with named parameters.

@jeroenransijn In most cases I agree with you. But the most used mixins for me are breakpoints media and I'd like to sort them in a predefined order. Seems I should make a fork.

@jeroenransijn I like your solution. Have you submitted a PR?

@hudochenkov's implementation is merged into dev, send him some love for this feature :hearts:
You can take a look at tests as examples: https://github.com/csscomb/csscomb.js/commit/6617b5b4134e10cf5a2883f391223068e7357e5e

@hudochenkov You rock! Love to see this feature implemented. Thank you!

Hi, apologies if this has been discussed in another place, but will this feature be added to the grunt version of csscomb? Its exactly what I need! But doesn't seem to be working currently. Thank you!

@jimbateson, yes, it will be added to grunt plugin once stable<EMAIL_ADDRESS>is released.

Quick question, is this working with the sublime text version?

@zellwk not yet. This feature only in dev branch for now. For Sublime Text this will be available when CSSComb 4.0 will be released.

Okay cool. Thanks. Any idea when it'll be rolling around?

@zellwk no idea :(

:(
Alright cool. Thanks!

@zellwk, @hudochenkov, linters! lint methods for options are blocking the release. I've added just one for always-semicolon so far, and can't finish remaining ones.

Hey guys, any updates?

I want to use the dev version of csscomb because of the sort-order feature adjustments. How can I manually drop the dev version into my node modules, but with the right csscomb-core for it? Currently if I use csscomb dev version in my node modules I end up with a gulp warning. I can only imagine thats because I don't really have the right dependencies for it. Help would be appreciated!

@Skoks did you found a solution for your problem?
Thanks!

@SeBuDesign No, do you have any ideas?

@SeBuDesign @Skoks you can use PostCSS plugin postcss-sorting which allow you to sort mixins by name.


Unable to create random alias

Hi,
The App only has an option to create a named alias.
There is no option for a random alias (UUID or words) can this be added as it's my number 1 use-case for the app.
I've been using the mobile site to this.
Thank you

Hello, you can shake the phone on homepage and the app will let you random an alias by word or UUID. I know this is not so obvious and I'm working on renewing the whole UI.

O wow haha, yeah that works!
Would be great to get a physical button as I personally don't like the shake gesture.
And from a UX perspective the discoverability is low.

Absolutely! FYI I'm working on rewriting the whole app in SwiftUI. It'll take some time but I hope the result will be worth it :)

I wish I knew this sooner 🙄
I like the uniformity of having totally random UUID address for all acounts in my password manager but I couldn't find a way to make those within the iOS app so some of them are random UUID's and some of them are named. Oh well, #firstworldproblems I guess 😅.


Disambiguate parameter load and import error messages.

I was looking at the debug outputs and the strings were ambiguous as to which line the error was coming from. This makes each output unique so as to be directly attributable.

This came from, I was looking at the startup, and I'd like to not have an "Error" show up which seems to be expected.
creating new parameters file
creating new dataman file

______  __   __    ___
| ___ \ \ \ / /   /   |
| |_/ /  \ V /   / /| |
|  __/   /   \  / /_| |
| |     / /^\ \ \___  |
\_|     \/   \/     |_/

px4 starting.

ERROR [param] importing from 'rootfs/eeprom/parameters' failed (-1)
Command 'param' failed, returned 1

The default parameter file is created and then fails to import. This seems like it shouldn't be classified as an error. If anyone has a suggestion of a way to avoid printing this error please let me know. The two approaches I though were to fill the default parameter file with some place holder, or to only try to import if the file exists with non-zero size.

That approach sounds reasonable to me. The other option would be to make the importer ok with loading from an empty file. I think adding an extra flag in this function that would not implicitly consider zero iterations an error might resolve this.

I think adding an extra flag in this function that would not implicitly consider zero iterations an error might resolve this.

Yes that should do it.


Abstraction Layer for Messaging

Currently the messaging functionality connecting core-data with export-distro is tightly coupled to ZeroMQ. We need to put this behind an interface and implement with the appropriate type. The initial thought was to do this in a manner commensurate with how we've integrated database providers (interface in the respective service, type in an internal package). However during the Application Services face-to-face in Boston, an additional requirement has become apparent. That is, this same capability needs to be usable by the soon to be created App Services SDK. A discussion was had w/r/t implementing this in an externally usable package or module which would then be consumed by the SDK and edgex-go. Initial thoughts are that this could be hosted within the edgex-go repo at the following location:
/pkg/modules/messaging
The package would contain a go.mod designating it as a module which can be consumed by any number of 3rd party apps/services.

Reworked the issues for this effort onto the appropriate repos. The following issues will address this issue:

edgexfoundry/go-mod-messaging#5
#1148
edgexfoundry/app-functions-sdk-go#27


Publication filter with multiple types

Hi,
I've just started using hugo-academic today and I really like it so far.
I would like to divide my publications into two parts by using widgets with different filtering rules. Basically, what I want to do is to separate peer-reviewed and non-peer-reviewed publications. I thought I could for example use one widget with the filter set to conferences and journals, and another widget with the type set to for example uncategorized. However, I couldn't figure out how to select multiple types for the filter. Is it possible or does the publication_type variable only accept one integer number?

Hi, I would say "non-/peer-reviewed" does not really belong to the "conference / letter / journal" list. You may imitate the "first-author" feature in #323 to create an additional filter for "peer-reviewed", just by renaming the variables and texts.
A much better solution to thi issue will be #352,  although someone shall spend time to implement it.

I see.
At first, I didn't realize that you can easily add your own publication types. I've solved it by defining two additional types for peer-reviewed publications, and other publications and reports.


added navigation bar

Please, go through these steps before you submit a PR.

[x] My Pod Leader knows I'm working on this Pull Request
[x] I've explained what the Pull Request is adding.
[x] I've explained why this is important.

Before

Added navigation bar for moving to different sections of the webpage and also added glow feature on hover.

@varsharathore16 Review this PR


BTS-1077: bulk test data creation

Scope & Purpose
reduce test resource usage by properly bulking test data creation

[x] :hankey: Bugfix

tests green except unrelated errors.


feat: catch unexpect multi-values

:tada: This PR is included in version 0.294.0 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Possible to change BrowseImageURL to subdirectory?

I am using S3 to store my images.  I have a separate process that is used to upload images via the django admin so I only use CKEditor to use images that are already uploaded.
My application is also multitenant to I need to store images by tenant so that each tenant can only see and use their own files.
I am currently using the structure of
BUCKET/images/COMPANY_NAME to store the images.  I have the browse path set to /images/ in settings because the subdirectory will vary by tenant.
Is it possible using js or some other method to dynamically change the browse directory to go down to a subdirectory based on the company name?

You will likely have to modify the URLs for the browse views: https://github.com/django-ckeditor/django-ckeditor/blob/master/ckeditor_uploader/views.py and use your version of the view instead of the built in default one.

Is it possible to override that and still use the remainder of the package or do I need to duplicate the package and make my change?

You only have to change the view - put it in your code and then change the urls.py so that the browse urls point to your views and not the default ones.

Newbie here...
Thank you for your help.  How would I change the urls to point to my views.py?

I got it!  Thank you so much for pointing me in the right direction :)


Support trybuild (take 2)

Fixes #32
The following patch is needed until dtolnay/trybuild#123 is merged:
[patch.crates-io]
trybuild = { git = "https://github.com/taiki-e/trybuild.git", branch = "target" }

Tested in the same way as https://github.com/taiki-e/cargo-llvm-cov/pull/44#issuecomment-894656727

bors r+

bors r+