Add a flow.dispatch.workgroups op

The current flow.dispatch.region is meant to model entire tensor -> tensor operations. This is useful for things that start this way like when forming regions from traditional tensor-based ops, but if trying to perform tiling and then forming dispatch regions it creates a layering issue where a dispatch tile body has to be shoehorned into a non-tiled region and then back out again during code generation.
Instead, a native flow.dispatch.workgroups that had the body region specified as per-tile and attached the related tiling information/functions (like one to calculate the workgroup count) would let us avoid going back in-to/out-of untiled form as well as help with #3737 by eliminating layer inversion that is causing the workgroup calculation generation to be done too-late. Instead, as the flow dialect is target-agnostic we could have the flow.dispatch.workgroups op completely parameterized on workgroup size values such that all targets could use the same calculation (possibly with modifications to the indexing/tile size/etc ops).
The current workload as used by the dispatch region ops could be removed as we would (possibly) not perform fusion of such ops and instead rely on whatever produced the tiled dispatch to have done that beforehand. There may still be utility in exposing a tile map to allow per-region dependencies (tile 0 in dispatch B only consumes outputs from tile 0 in prior dispatch A), but that's strictly an enhancement.
Sketch:
%ret0, %ret1 = flow.dispatch.workgroups(%arg0 : tensor<?xf32>, %arg1 : tensor<?xf32>, %runtime_param : index) -> (tensor<?xf32>, tensor<?xf32>)
  apply(%runtime_param : index) {
    %0 = flow.interface.workgroup.count // placeholder for total workgroup count xyz
    %1 = flow.interface.workgroup.id // placeholder for current workgroup ID xyz
    %4 = flow.interface.load.tile arg[0], offsets = [...], sizes = [...], strides = [...] : tensor<1x3xf32>
    %5 = flow.interface.load.tile arg[1], offsets = [...], sizes = [...], strides = [...] : tensor<3x1xf32>
    %7 = <some op working on the tile like linalg.matmul>
    flow.interface.store.tile %7, ret[0], offsets = [...], sizes = [...], strides = [...] : tensor<1x1xf32>
    flow.return
  },
  count(%shape0 : !shapex.ranked_shape<?>, %shape1 : !shapex.ranked_shape<?>) -> !shapex.ranked_shape<?> {
    // shape math --- arguments could also include data values for data-dependent shapes
    flow.return %computed_shape : !shapex.ranked_shape<?>
  }

The goals here are to make it easy to lower into the HAL, be completely parameterized such that it's agnostic to the target backend (which then may pick one or more tile sizes/iteration types/etc), and exposing flow-aware information that can be used for device placement and stream creation (like knowing the workgroup count computation, tile mapping across dispatches, etc), and have a place to put data-dependent size computations that we want to then emit additional executables to use to compute indirect dispatch parameters.
The primary difference with the HAL is that we don't yet have a fixed interface (that's computed/refined later per-target) and instead have fixed argument/results that are referenced to load/store the tiles. This is different than taking tile inputs/outputs intentionally so that each workgroup in our grid does not have operate on specific tiles and can instead load/store multiple tiles, have imperfectly nested loops, etc. In many cases a workgroup in the dispatch may be 1:1 with tiles loaded/stored, but that should just be treated as a lucky case where we may be able to emit less indexing math in the end.
So, the new ops are:

[ ] flow.dispatch.workgroups
[ ] flow.interface.workgroup.* (to match the hal.interface.workgroup.* ops being added now)
[ ] flow.interface.load/store.tile (that are a subset of hal.interface.load/store.tensor.tile, as here we only have tensors and don't have byte offsets yet)

A straightforward conversion could today take a flow.dispatch.region and turn it into a 1x1x1 flow.dispatch.workgroups by a simple conversion pattern. This may still be useful to support coming from frontends that may not be tiled or may be using custom ops that don't have tiling concepts - the region then is just shorthand for a single thing.

I'm still fiddling with the IR but another idea is to have a new type:
%ret0, %ret1 = flow.dispatch.workgroups(%arg0 : tensor<?xf32>, %arg1 : tensor<?xf32>, %runtime_param : index) -> (tensor<?xf32>, tensor<?xf32>)
  apply(%arg0 : !flow.dispatch.input<?xf32>,
        %arg1 : !flow.dispatch.input<?xf32>,
        %runtime_param : index,
        %ret0 : !flow.dispatch.output<?xf32>) {
    %shape0 = flow.dispatch.shape %arg0 : !flow.dispatch.input<?xf32> -> !shape.ranked_shape<?>
    %0 = flow.interface.workgroup.count // placeholder for total workgroup count xyz
    %1 = flow.interface.workgroup.id // placeholder for current workgroup ID xyz
    %4 = flow.interface.load.tile %arg0, offsets = [...], sizes = [...], strides = [...] : tensor<1x3xf32>
    %5 = flow.interface.load.tile %arg1, offsets = [...], sizes = [...], strides = [...] : tensor<3x1xf32>
    %7 = <some op working on the tile like linalg.matmul>
    flow.interface.store.tile %7, %ret0, offsets = [...], sizes = [...], strides = [...] : tensor<1x1xf32>
    flow.return
  }

I think that would play a bit better in the MLIR system (get us nicer verifiers) and ensure we exclusively use the interface load/store ops and preventing the risk of making assumptions too early about byte layouts.

Thoughts for @nicolasvasilache: basically you'd at the flow level emit the tiled loads/stores, the flow.interface.load/store.tile could take the !shapex.ranked_shape as an argument (I think in the issue I just copy/pasted the offsets/sizes/strides but that should really be shape), and when lowered into the HAL set that shape information is available for use. There wouldn't be a use of hal.interface.load.tensor at all, so no need to thread through metadata into hal.interface.load.tensor.tile manually (the flow->hal tile load/store ops should be straightforward lowerings).
There still may be cases where you want to do an entire tensor, though, such as gathers where the output may be tiled writes but the inputs are not tiled (you're gathering from the entire input tensor). You may also want to perform multiple tile loads/stores in the same workgroup, or as in reduction load multiple tiles to store a single tile, etc. This should give you that flexibility.

Generally LGTM, thanks!
The remaining issue I have for correct execution (after locally taking liberties with pass ordering) is that the X.interface.load.tile op wants to bufferize into a subview and for this it needs a "shaped source buffer" (i.e. something from which to get the base ptr + offset + sizes + strides).
Starting from your latest version:
%ret0, %ret1 = flow.dispatch.workgroups(%arg0 : tensor<?xf32>, %arg1 : tensor<?xf32>, %runtime_param : index) -> (tensor<?xf32>, tensor<?xf32>)
  apply(%arg0 : !flow.interface.input<?xf32>, ... ) {
    %shape0 = flow.interface.shape %arg0 : !flow.interface.input<?xf32> -> !shape.ranked_shape<?>
   ...
    %4 = flow.interface.load.tile %arg0, offsets = [...], sizes = [...], strides = [...] : tensor<1x3xf32>

It is still unclear to me here how:
flow.interface.load.tile %arg0, offsets = [...], sizes = [...], strides = [...] : tensor<1x3xf32>

gets me the maximally statically typed source buffer shape information from just %arg0.
To make it more concrete, here is what I currently generate (manually adapted adapted to your current flow IR, I used a 2-D tile size 1x1 to simplify):
flow.dispatch.workgroups(%arg0 : memref<?xf32>, %arg1 : memref<?xf32>, %arg2 : memref<?xf32>, %runtime_param : index)
  apply(%arg0 : !flow.interface.input<?xf32>,
        %arg1 : !flow.interface.input<?xf32>,
        %arg2 : !flow.interface.input<?xf32>,
        %runtime_param : index,
        %ret0 : !flow.interface.output<?xf32>) {
    %c2 = constant 2 : index
    %shape0 = flow.interface.shape %arg0 : !flow.interface.input<?xf32> -> !shape.ranked_shape<?>
    %0 = flow.interface.workgroup.count // placeholder for total workgroup count xyz
    %1 = flow.interface.workgroup.id // placeholder for current workgroup ID xyz

    %b0 = flow.gimme_the_most_static_shape_you_can_op %ret0 : memref<?x?xf32>
    %b1 = flow.gimme_the_most_static_shape_you_can_op %arg0 : memref<?x2xf32>
    %b2 = flow.gimme_the_most_static_shape_you_can_op %arg1 : memref<?x?xf32>
    %b3 = flow.gimme_the_most_static_shape_you_can_op %arg2 : memref<?x?xf32>

    %9 = alloca() : memref<1x1xf32>
    scf.for %arg0 = %1#0 to %TODO(%dim b0, 0) step %0#0 {
      scf.for %arg1 = %1#1 to %TODO(%dim b0, 1) step %0#1 {
        %6 = subview %b3[%arg0, 0] [1, 2] [1, 1] : memref<?x?xf32> to memref<1x2xf32, affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>>
        %7 = subview %b2[0, %arg1] [2, 1] [1, 1] : memref<?x?xf32> to memref<2x1xf32, affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>>
        %8 = subview %b1[%arg0, %arg1] [1, 1] [1, 1] : memref<?x?xf32> to memref<1x1xf32, affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>>
        linalg.copy(%8, %9) : memref<1x1xf32, affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>>, memref<1x1xf32>
        linalg.matmul ins(%6, %7 : memref<1x2xf32, affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>>, memref<2x1xf32, affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>>) outs(%9 : memref<1x1xf32>)
        %10 = subview %b0[%arg0, %arg1] [1, 1] [1, 1] : memref<2x2xf32> to memref<1x1xf32, affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>>
        linalg.copy(%9, %10) : memref<1x1xf32>, memref<1x1xf32, affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>>
      }
    }
  }

Don't mind the copies as they get fused away on tensors when appropriate.
What should this op look like?
    %b0 = flow.gimme_the_most_static_shape_you_can_op %ret0 : memref<?x?xf32>

In particular I started from the minimal amount of reasonable information:

I magically know the rank here (I can infer it from #sizes in flow.interface.load.tile)
I assumed the allocated memory region is contiguous for now (no layout map on memref<?x?xf32>)
buffer size information seems type erased so I have to go for  memref<?x?xf32> (more static typing would always be better but not a blocker; I do not see how to extract more static info from the IR you gave me here).

flow.gimme_the_most_static_shape_you_can_op could potentially turn into a view to give a bare pointer some indexing semantics.

AFAICS, getting a memref view of the operands to the dispatch region seems fairly reasonable to me. I am assuming the contract of dispatch region is that all tensors operands and result get their own buffer

noooo no memrefs please :)
(in meetings, will respond afterward)

@benvanik it doesn't have to be at the flow level :)
But there is going to be a bufferization and memrefs will come in existence at some point.
Connecting the IREE representation for base pointer + offset + shape to a memref type has to happen somewhere.
I am looking for that particular bridge of the abstraction gap that is not yet clear.
If you're going as far as saying we should never try to have load/store from/to buffer granularity in codegen, I'd refer to this section: LLVM will spill us into oblivion and we want control over buffer aspects (although we are agressively moving away from traditional C + array-based inputs and analysis).
Anyway, let's discuss in our VC, much higher BW :)

@MaheshRavishankar what are your thoughts on having the workgroup count as a function expression vs having it as just arguments to the op?
Example:
%x = shape_math_stuff
%y = shape_math_stuff
%foo = flow.dispatch.workgroups[%x, %y](%arg0 : tensor<...>)...

My thinking here is that this will make it easier at the flow level to aggregate shape logic across multiple dispatches - especially when indirect - as it's possible to just use the SSA use-def chain and normal op reordering to get all the shape logic to the same place (dominating block, function entry, etc) at once. We can then do the things as a separate bit of logic for indirect/predicated dispatch, vs. having the workgroup count logic separate already.
Sound ok?
Also, I'm going to make the flow-level workgroup dimensions [1, inf) (not limited to a 3D xyz) - it can be a flow->hal step to take the N-dimensional grid and map it to the 3D grid when workgroup size is known (it's not in flow level as we don't know the target yet).

So you are saying we keep the shape match stuff "outside" of flow.dispatch.workgroup and when we create that we just insert whatever logic we need there and the flow.dispatch.workgroup just uses these values. That would indeed be easier to generate.
What about doing the same for workgroup size as well? So it would actually look like the CUDA <<<...>>> semantics in IR?

At the flow level we shouldn't have a workgroup size yet (you could still reference the workgroup size symbolically inside the region, but there's no way to specify it).

I dont fully follow why that should be the case (we have the number of workgroups but not workgroup size?)

What would you use to compute it? At this point, you don't know the target device.
Once you know it (when lowering flow->hal) you can factor that into the workgroup count calculation per-device easily. You can also factor in per-device + per-dispatch and dynamic shape math logic (clamping to maximum workgroup count along particular dimensions, padding, etc).
For example:
%x = constant 128 : index
%y = constant 48 : index
flow.dispatch.workgroups[%x, %y](...)

->
%x = constant 128 : index
%y = constant 48 : index
hal.device.switch...
  match<"device_0">() {
    %workgroup_size_x_dev0 = constant 4 : index
    %workgroup_size_y_dev0 = constant 2 : index
    %x_dev0 = divi %x, %workgroup_size_x_dev0 : index
    %y_dev0 = divi %y, %workgroup_size_y_dev0 : index
    %z = constant 1 : index
    hal.dispatch %x_dev0, %y_dev0, %z
  },
  match<"device_1">() {
    %workgroup_size_x_dev1 = constant 64 : index
    %workgroup_size_y_dev1 = constant 8 : index
    %x_dev1 = divi %x, %workgroup_size_x_dev1 : index
    %y_dev1 = divi %y, %workgroup_size_y_dev1 : index
    %z = constant 1 : index
    hal.dispatch %x_dev1, %y_dev1, %z
  }

(way back in the day flow.dispatch.region could take multiple dimensions for the workload but then we simplified it to just a single one, this is effectively the same thing as the original one was)

I think there is still a gap in my head. Loosely this is how I see everything working out.

You use some %tilesize to tile (+fuse) the Linalg operation on tensors.
The tile size is also used in the calculation of the workgroup count since for all dimensions as ceil(%dim / %tilesize).
In the 1D case on GPU you would do something like this

%offset = mulf %workgroup_id, %tilesize
%step = mulf %workgroup_count, %tilesize
scf.for %i0 = %offset to %ub step %step { // <- loop that is distributed across workgroups
  %wg_offset = mulf %local_invocation_id, %workgroup_size
  %wg_step = mulf %local_invocation_id, %workgroup_size
  scf.for %i1 = %wg_offset to %tilesize step %wg_step {  // <- loop distributed across items within workgroup

  }
}

For most cases %workgroup_size == %tilesize and we can get rid of the second loop.
I guess we could avoid having %workgroup_size, but we will still need the %tilesize both with the flow.dispatch.workgroup and outside that to get the %workgroup_count. I am afraid that this will only work out cleanly if %tilesize is compile-time constant without which we will have to make sure somehow the two %tilesizes are consistent. Having a %workgroup_size, we will be able to compute %tilesize is terms of that.
I am mostly thinking loud here and trying to internalize this. But if you believe we dont need %workgroup_size then we can go with that and revisit if we really have a concrete case where we need it.

I think I get what you're saying. Here at the flow level the tile size is parametric and not known either on the inside of the region or the outside. For the inside you can use a flow.dispatch.workgroup.size op that lets you get the placeholder for the workgroup size (which will later end up as hal.interface.workgroup.size) so any math that you need to do inside of a workgroup using that for indexing into tiles/etc can still be done.
The difference is that on the outside the flow.dispatch.workgroups op is not yet tied to any device (and that's super important) as device placement (and stream formation/etc) happens after the formation of these. And, for particular devices there may be a fan-out - so one flow.dispatch.workgroups may turn into any number of hal.dispatch ops. Conceptually you could also have a way to get the future symbolic workgroup size outside of the region, but then how would you track that?
%workload_x = ...
%workload_y = ...
%workgroup_count_x = divi %workload_x, %???? : index
%workgroup_count_y = divi %workload_y, %???? : index
flow.dispatch.workgroups[%workgroup_count_x, %workgroup_count_y](...)

Because of the one-to-many relationship between these ops and the HAL ops where we pick the actual workgroup sizes anything we would do there - like say being able to get some symbolic value - would only make things difficult:
%workload_x = ...
%workload_y = ...
%workgroup_size_x = flow.make_up_workgroup_size_value[0]
%workgroup_size_y = flow.make_up_workgroup_size_value[1]
%workgroup_count_x = divi %workload_x, %workgroup_size_x : index
%workgroup_count_y = divi %workload_y, %workgroup_size_y : index
flow.dispatch.workgroups[%workgroup_count_x, %workgroup_count_y](...)

But the workgroup size used is tied to the dispatch and that dispatch may move around (into streams/etc), so if you had two dispatches and CSE ran and deduped the sizes now what do you do? They may still end up on totally different devices (or the same device but with switches based on runtime-parameters, like vulkan device limits):
%workload_x_0 = ...
%workload_y_0 = ...
%workload_x_1 = ...
%workload_y_1 = ...
%workgroup_size_x = flow.make_up_workgroup_size_value[0]  // was CSE'd
%workgroup_size_y = flow.make_up_workgroup_size_value[1]
%workgroup_count_x_0 = divi %workload_x_0, %workgroup_size_x : index
%workgroup_count_y_0 = divi %workload_y_0, %workgroup_size_y : index
%workgroup_count_x_1 = divi %workload_x_1, %workgroup_size_x : index
%workgroup_count_y_1 = divi %workload_y_1, %workgroup_size_y : index
flow.dispatch.workgroups[%workgroup_count_x_0, %workgroup_count_y_0](...)
flow.dispatch.workgroups[%workgroup_count_x_1, %workgroup_count_y_1](...)

That gets really messy. Even if we could resolve it (put tags on the make_up_workgroup_size_value op or prevent it from being CSEd) it's still just more to track. It gets even more complex when we think of the additional math we'd want to use to do things like stay within device limits (which can vary per dimension from 64 to INT_MAX), which will happen both before and after the use of the workgroup size. By not factoring in the workgroup size here we also allow any arbitrary number of dimensions on the flow.dispatch.workgroups and only flatten down (or expand out) to the 3D XYZ grid as part of the flow->HAL lowering where we'd be inserting other math anyway, and if we've already got the workgroup size math in the IR it's really hard to undo that (if, for example, we wanted to change the tile size or distribute along a different axis).
So what's being done here is saying that the region inside the flow.dispatch.workgroups op represents a single workgroup and that single workgroup can access a symbolic workgroup size to do any indexing math it needs (all symbolically), but on the outside we are in raw loop ranges (in your example, without %step applied yet). It's only when we go to a device that we can introduce the logic to take that invocation range down into a workgroup range.
(happy to vc for higher bandwidth - figured having some IR examples would be useful first :)

+1 for VC, I am available starting in 30 minutes.
I have the impression the discussion is shifting a bit towards usage tradeoffs and getting more and more disconnected at the type level as the signatures and bbarg types now disappear :)
I am wondering whether this needs to be resolved right away to make progress on reordering passes and getting 1 e2e example running.
Given the current https://gist.github.com/nicolasvasilache/6d98a7dd5f2bd4e71a2d887e1ad2dd0d I am looking at where I would start and the only reasonable answer I see atm is:
  func @tensor() -> (tensor<2x4xf32> {iree.reflection = {}}) attributes {iree.module.export, iree.reflection = {f = "I1!R8!B5!d2d4", fv = "1"}} {
    %cst = constant dense<[[1.000000e+00, 2.000000e+00, 3.000000e+00], [4.000000e+00, 5.000000e+00, 6.000000e+00]]> : tensor<2x3xf32>
    %cst_0 = constant dense<[[1.000000e+00, 2.000000e+00, 3.000000e+00, 4.000000e+00], [5.000000e+00, 6.000000e+00, 7.000000e+00, 8.000000e+00], [9.000000e+00, 1.000000e+01, 1.100000e+01, 1.200000e+01]]> : tensor<3x4xf32>
    %cst_1 = constant dense<1.000000e+03> : tensor<2x4xf32>
    %0 = iree.do_not_optimize(%cst) : tensor<2x3xf32>
    %1 = iree.do_not_optimize(%cst_0) : tensor<3x4xf32>
    %2 = iree.do_not_optimize(%cst_1) : tensor<2x4xf32>
    %3 = linalg.matmul ins(%0, %1 : tensor<2x3xf32>, tensor<3x4xf32>) init(%2 : tensor<2x4xf32>)  -> tensor<2x4xf32>
    return %3 : tensor<2x4xf32>
  }

From that I would go to:
  func @tensor() -> tensor<2x4xf32> {
    %c2 = constant 2 : index
    %c4 = constant 4 : index
    %0 = iree.unfoldable_constant dense<[[1.000000e+00, 2.000000e+00, 3.000000e+00], [4.000000e+00, 5.000000e+00, 6.000000e+00]]> : tensor<2x3xf32>
    %1 = iree.unfoldable_constant dense<[[1.000000e+00, 2.000000e+00, 3.000000e+00, 4.000000e+00], [5.000000e+00, 6.000000e+00, 7.000000e+00, 8.000000e+00], [9.000000e+00, 1.000000e+01, 1.100000e+01, 1.200000e+01]]> : tensor<3x4xf32>
    %2 = iree.unfoldable_constant dense<1.000000e+03> : tensor<2x4xf32>
    %3 = "flow_workgroup_id"() {dim = 0 : i64} : () -> index
    %4 = "flow_workgroup_count"() {dim = 0 : i64} : () -> index
    %5 = "flow_workgroup_id"() {dim = 1 : i64} : () -> index
    %6 = "flow_workgroup_count"() {dim = 1 : i64} : () -> index
    %7 = "flow_workgroup_despetch"() ( {
      %c1 = constant 1 : index
      %8 = scf.for %arg0 = %5 to %c2 step %6 iter_args(%arg1 = %2) -> (tensor<2x4xf32>) {
        %9 = muli %3, %c2 : index
        %10 = muli %4, %c2 : index
        %11 = scf.for %arg2 = %9 to %c4 step %10 iter_args(%arg3 = %arg1) -> (tensor<2x4xf32>) {
          %12 = subtensor %0[%arg0, 0] [1, 3] [1, 1] : tensor<2x3xf32> to tensor<1x3xf32>
          %13 = affine.min #map(%arg2)
          %14 = subtensor %1[0, %arg2] [3, %13] [1, 1] : tensor<3x4xf32> to tensor<3x?xf32>
          %15 = affine.min #map(%arg2)
          %16 = subtensor %arg3[%arg0, %arg2] [1, %15] [1, 1] : tensor<2x4xf32> to tensor<1x?xf32>
          %17 = linalg.matmul {__internal_linalg_transform__ = "workgroup"} ins(%12, %14 : tensor<1x3xf32>, tensor<3x?xf32>) init(%16 : tensor<1x?xf32>)  -> tensor<1x?xf32>
          %18 = subtensor_insert %17 into %arg3[%arg0, %arg2] [%c1, %15] [%c1, %c1] : tensor<1x?xf32> into tensor<2x4xf32>
          scf.yield %18 : tensor<2x4xf32>
        }
        scf.yield %11 : tensor<2x4xf32>
      }
      flow.return %8 : tensor<2x4xf32>
    }) : () -> tensor<2x4xf32>
    return %7 : tensor<2x4xf32>
  }

None of this has the magical types I see in
%ret0, %ret1 = flow.dispatch.workgroups(%arg0 : tensor<?xf32>, %arg1 : tensor<?xf32>, %runtime_param : index) -> (tensor<?xf32>, tensor<?xf32>)
  apply(%arg0 : !flow.interface.input<?xf32>,
        %arg1 : !flow.interface.input<?xf32>,
        %runtime_param : index,
        %ret0 : !flow.interface.output<?xf32>) {
    %shape0 = flow.interface.shape %arg0 : !flow.interface.input<?xf32> -> !shape.ranked_shape<?>

I would love to solve the semantic gap concretely (and get the reordered phases running) and then iterate to the different ways we can create tiled and fused dispatch regions.
To me the problem of Connecting the IREE representation for base pointer + offset + shape to a memref type has to happen somewhere is still not solved concretely and the biggest source of remaining unknown atm.

I think what you're talking about is all in the inside of the region whereas we're talking about the outside part. So I think nothing really changes for what you're doing. But I may also be dividing things up mentally in a different way so it would be useful to get our terminology/etc lined up :)
I'm free after 12:30PST.

Lets do 12:30. I have a slot with Nicolas for that time. We can use that.

Looked through the example above. I see that there is a much longer term aspects to these things. It will be easier to reason about this with a simple example and seeing all the pieces. The only contract that is really needed is
If a dynamic tile size to tile (+fuse) + distribute the loop, that tile size is used compute the number of workgroups. The resolution of tile size -> workgroup size is internal to the dispatch region.
If that is consistent with what you are saying, I think we are on the same page w.r.t things that are nailed down at this level

@nicolasvasilache / @MaheshRavishankar update here of what I've got so far: https://gist.github.com/benvanik/454d9638eedf6bd4056a2e7fba52ec24
This carries through all dynamic shape information, allows for the tile load/store behavior, and has symbols for getting the workgroup info (count/etc) that can be carried across. For static dimensions the shape queries will fold during canonicalization (so shapex.ranked_dim[1] on ?,4 => constant 4), while the dynamic dimensions will be transformed into additional operands (and eventually push constants on the HAL interface).
The builder for the dispatch workgroups op will synthesize the needed region arguments so as you are forming it you can just pass what you need (input tensors/result types) and the rest will be handled for you (dealing with shape ties and such).

Here's the code for that: https://github.com/google/iree/commit/dd892305474f7aa5f90e6786e3ab3d1119fb1992
I'm going to start getting the transformations in now (outlining/etc) and then send the both for review (I have a feeling I may need to iterate a bit with respect to the shape/shapex stuff).

Looks overall fine. Just to clarify, the flow.dispatch.shape is a way to just get the shape of the argument/result tensors right?

Also is there a limit/way to know the extent of the indices in flow.dispatch.workgroup.(id|count|size) ? I am assuming that this would be generated in the tiled+fused+distributed loop to flow.dispatch.workgroup creation. We dont need to restrict that to 3D or something right. It could be any dimension. Is this information to be derived from the arguments/results of the dispatch region or will there be an explicit instruction to query that.

Looks overall fine. Just to clarify, the flow.dispatch.shape is a way to just get the shape of the argument/result tensors right?

yep! it's just a pseudo op for shapex stuff (so you don't have to insert the tie_shapes and do the shape resolution)

Is there a limit/way to know the extent of the indices in flow.dispatch.workgroup.(id|count|size) ?

No limit (just >=1) - the idea is that the mapping to 3D XYZ can be target-dependent and we may want to tune that, so with this as it is you can just take however many loops you want to shove into the operation and that'll be it.
Good idea on querying the count - right now when I need that I just look at workgroup_count().size(), but having an op that makes it easier to handle when the executable has been outlined would be good. flow.dispatch.workgroup.rank or something.

Here's what I have so far, which I think may be almost enough to start wiring up at either end (forming the flow.dispatch.workgroups op and then lowering the hal.executable.target op to codegen):
https://gist.github.com/benvanik/a2c1915c71dfb611f8e6d7ddcdc96539
Example shows a static case from the flow.dispatch.workgroups op through to the hal.executable immediately prior to giving the executable to the codegen backend. There's still some flow ops/types in there intentionally: the backend can trivially turn hal.interface.binding.subspan ... : !flow.dispatch.input<7x?x24x?xf32> -> hal.interface.binding.subspan ... : memref<?xi8> if it wants via type conversion, and the flow.dispatch.input.load can become whatever subview ops are desired via simple 1:1 replacement.
Regardless of what we end up doing with shapes this should be possible to move between with simple op replacement/type expansion (tensor<... shape_only=true> <-> ranked_shape, etc). What's there now uses the shape ties and assumes they are present but if we start without them we can insert them pretty easily from whatever form we have.
With a simple pass or canonicalization pattern that propagates the shape informationinto the loads/stores we can remove the ties/shapex types/etc - I put this IR together and then decided to do that, though, so I just tagged on at the bottom what that will look like :)
Major things missing:

me: some patterns for things like flow.dispatch.workgroup.count -> hal.interface.workgroup.count, etc that backends can add to their import pipeline
me: push shape info into flow.dispatch.i/o.load/store and remove use of shapex/ties from IR
@nicolasvasilache: building the tiled dispatch region (vs. one that returns the full tensor/has subtensor_insert/etc)?
@MaheshRavishankar: turning hal.interface.binding.subspan into something in the backend? (it replaces iree.placeholder)

Actually, before I change the shape stuff; @nicolasvasilache: do you think you want shapes on your ViewLikeInterface? Given that offsets/sizes/strides are parsed/printed and such in there it may make more sense to just add shape to that set (so offsets/sizes/strides/shape) - right now I have a mismatch of assemblyFormat handling and other things to try to make it work. Functionality-wise the current tie shape should give you the info you need, but it'd be nice if we could just have a ViewLikeInterface on the load/store ops that had shape as well and then nothing additional is required.

Not sure why
%0 = hal.interface.binding.subspan @legacy_io::@arg0[%c0] : !flow.dispatch.input<8x4xf32>

cant just be
%0 = hal.interface.binding.subspan @legacy_io::@arg0[%c0] : memref<8x4xf32>

then
%2 = flow.dispatch.input.load %0 : !flow.dispatch.input<8x4xf32> -> tensor<8x4xf32>

can just be
%2 = load_tensor %0 : memref<8x4xf32> -> tensor<8x4xf32>

that's what you'll be doing :) At the point we are still in flow land memrefs don't exist - that's a purely backend concept you'll be introducing. The ops are what they are so that you can do a simple 1:1 conversion.

(and, in my dreams, one day memrefs aren't a thing and instead that returns opaque_bag_of_bytes<?>)

Actually, before I change the shape stuff; @nicolasvasilache: do you think you want shapes on your ViewLikeInterface? Given that offsets/sizes/strides are parsed/printed and such in there it may make more sense to just add shape to that set (so offsets/sizes/strides/shape) - right now I have a mismatch of assemblyFormat handling and other things to try to make it work. Functionality-wise the current tie shape should give you the info you need, but it'd be nice if we could just have a ViewLikeInterface on the load/store ops that had shape as well and then nothing additional is required.

It might be worth exposing the parser method (here), the printer method (here) and the verifier (here) to allow this. I was drafting an init_tensor based on discussion in morning (With canonicalizers and folders, etc.) and ended up doing that

If those were exposed that'd be great! (unless view always wants shape, then throw it there :)

well, lets see what @nicolasvasilache  says about those. Will send out patch for my init_tensor draft (have internal CL with the op spec. Adding canonicalizers now). The view doesnt need shape. The size is effectively the shape... (so I am missing why there is an extra thing for the flow op here, but I have to look deeper)

Also, its a little strange that after lowering from flow to hal we still have flow types. But its more ergonomics than a technical issue

size is the tile shape (IIRC?), but what's the total shape? that's what the tie_shape is used for above (to allow us to resolve the total shape of the input/output tensor independent of the size of the tile being loaded/stored)

There's no need to have additional hal ops as you'll just immediately be lowering them to something else (like load_tensor) - the ergonomics improvement here is no duplication of ops that are otherwise the same besides their dialect - MLIR is great because it allows for any fragment of IR to have multiple levels and not just be level->level :)

me: some patterns for things like flow.dispatch.workgroup.count -> hal.interface.workgroup.count, etc that backends can add to their import pipeline

Fiddling with this a bit, I think maybe just not having the hal ops at all is best - if we do flow->hal inside of the target backends that removes the extra bounce through the hal dialect. Will let it be as it is now until we decide otherwise.

me: push shape info into flow.dispatch.i/o.load/store and remove use of shapex/ties from IR

As mentioned above, going to not touch this (keep the tie shapes that can be resolved) until we have more together.
I'll put what I have into a PR.

Ok, ill take a look probably later today, but mostly tomorrow. Thanks!

If those were exposed that'd be great! (unless view always wants shape, then throw it there :)

These are exposed in https://reviews.llvm.org/D93374. If you need it I can separate that out as a separate patch and submit it.

No rush - but looks good!
That code would be a nice candidate to switch to the custom parser/printer directive format in the future, as then anyone would be able to compose it nicely without having to go to full custom printers/parsers in C++: https://mlir.llvm.org/docs/OpDefinitions/#custom-directives

Oh nice idea. I will take a stab at that.

Didn't have a single cycle to devote to this this week, unfortunately.
I'll pick it up and connect to the dispatch region creation when I come back in 10 days.

Closing this. All issues discussed on this bug has been resolved...


LDO2 Restart SD failing

Paul,
below is minimal code to reproduce an issue whereby writing to SD fails  following toggle of LDO2. Appreciate your thoughts.
#include <SPI.h>
#include "BDL_.h"
#include <Arduino.h>
#include <esp_adc_cal.h>
#include <soc/adc_channel.h>

BDL bdl;
String myFilePath = "/myFile.txt";
File myFile;

void setup() {

  Serial.begin(115200);
  while (!Serial) {}
  Serial.println("Starting...");

  bdl.begin();

  if(!SD.begin()){
    Serial.println("Card Mount Failed");
  }
  else {
    Serial.println("SD card mounted");
  }

  myFile = SD.open(myFilePath, FILE_WRITE);
  if (!myFile){
    Serial.println("File Open 1: FAIL");
  }
  else {
    Serial.println("File Open 1: PASS");
  }
  myFile.close();

  bdl.setLDO2Power(false);
  delay(3000);  //----DO SOME WORK HERE----

  bdl.setLDO2Power(true);

  delay(10000);

  myFile = SD.open(myFilePath, FILE_WRITE);
  if (!myFile){
    Serial.println("File Open 2: FAIL");
  }
  else {
    Serial.println("File Open 2: PASS");
  }
  myFile.close();
}

void loop() {
  // put your main code here, to run repeatedly:
}

so a few things with that code, which maybe you just left out by mistake when making that example, but there is no FS.h, SD.h and your BDL.h has an underscore for some reason.
when i added those to the code it worked as intended.
#include "FS.h"
#include "SD.h"
#include <SPI.h>
#include "BDL.h"
#include <Arduino.h>
#include <esp_adc_cal.h>
#include <soc/adc_channel.h>

BDL bdl;
String myFilePath = "/myFile.txt";
File myFile;

void setup() {

  Serial.begin(115200);
  while (!Serial) {}
  Serial.println("Starting...");

  bdl.begin();

  if(!SD.begin()){
    Serial.println("Card Mount Failed");
  }
  else {
    Serial.println("SD card mounted");
  }

  myFile = SD.open(myFilePath, FILE_WRITE);
  if (!myFile){
    Serial.println("File Open 1: FAIL");
  }
  else {
    Serial.println("File Open 1: PASS");
  }
  myFile.close();

  bdl.setLDO2Power(false);
  delay(3000);  //----DO SOME WORK HERE----

  bdl.setLDO2Power(true);

  delay(1000);

  myFile = SD.open(myFilePath, FILE_WRITE);
  if (!myFile){
    Serial.println("File Open 2: FAIL");
  }
  else {
    Serial.println("File Open 2: PASS");
  }
  myFile.close();
}

void loop() {
  // put your main code here, to run repeatedly:
}

if this code doesnt work yet still, have you tried more than one of the boards?

that is really strange. what version of the Arduino bee data logger library are you using? im assuming the latest version 2.0.6?
that result you shared shows it fails to open both times? so even before turning the LD02 off as well? if you have a multimeter can you very that the 3v2 pin is going high and low when its turns off and on in code?

Post edited for clarity. Sorry I am travelling typing some of this on my phone.
Just to summarise,

the SD always mounts fine with LDO2 on. It doesnt matter if LDO2 has been on and off previously.
The file will always open fine the first time LDO2 is on wherever it is located in the routine. - If LDO2 is powered down and back up the file is no longer accessible.
if LDO2 is turned on once and remains on the the file always remains accessible.
BDL.h  2.0.6 taken from this github.

Hope this helps. For now I will workaround by rearranging my routine to only turn on ldo2 and write to SD just before deep sleep.


Type error

OS: MacOS Catalina
Deno version: 1.9.2
Argon version: 0.9.2
Hi @fdionisi ! I am trying to use the argon library in an API which I am making, but I am getting this error:
error: Uncaught (in promise) TypeError: invalid promise id: expected type `rusty_v8::data::Integer`, got `rusty_v8::data::Value`
      let result = Deno.core.dispatch(argon2_hash, args, buf)!;
                             ^
    at Object.hash (https://deno.land/x/argon2@v0.9.2/lib/internal.ts:97:30)
make: *** [run] Error 1

You can reproduce the issue with the following code:
import { hash, Variant } from<EMAIL_ADDRESS>
const secretString = "some-secret";

const encoder = new TextEncoder();

const secret = encoder.encode(secretString);

const passwordHash = await hash("password", {
  variant: Variant.Argon2id,
  secret,
});

console.log(passwordHash);

Ah, I must have missed that notice... thanks for your quick answer!

Not sure if related, but with deno 1.10.1, I now get the error:
error: Uncaught (in promise) Error: Could not obtain symbol from the library: dlsym(0x7fb9a546f230, init): symbol not found
  return Deno.openPlugin(localPath);
              ^
    at deno:core/core.js:86:46
    at unwrapOpResult (deno:core/core.js:106:13)
    at Object.opSync (deno:core/core.js:120:12)
    at Object.openPlugin (deno:runtime/js/40_plugins.js:8:22)
    at prepare (https://deno.land/x/plugin_prepare@v0.8.0/mod.ts:69:15)

Hey @halvardssm. The error you are seeing is because the library as it is cannot work with the new Deno Extension paradigm.
I was working yesterday to upgrade the library to Deno 1.10, but I encountered some problems (progress on https://github.com/fdionisi/deno-argon2/tree/feature/upgrade-to-deno-1.10 branch).
The new Deno Extension mechanism makes it more clean and easy to integrate native modules, but when I try to run the new implementation, I get a segmentation error:
❯ deno test -A --unstable tests/lib.test.ts
Check file:///Users/federico/Code/github/deno-argon2/tests/lib.test.ts
    Finished dev [unoptimized + debuginfo] target(s) in 0.54s
[1]    86968 segmentation fault  deno test -A --unstable tests/lib.test.ts

After some digging and try-outs, I noticed that simply copying Deno's own test_plugin, updating Cargo dependencies to use deno_core = 0.87.0 and finally testing it, will result in the same segmentation error.
I was preparing a repository to report the bug when I stumble upon a comment on Deno issues that describe the exact problem (and journey) I am experiencing.
Unluckily I'm not so knowledgeable about what is the cause of the error, so I believe we'll have to wait a bit more and see if the maintainer can help/figure this out.

Hey @halvardssm - just wanted to give you a heads up.
Apparently, the Deno plugin story needs a complete rewrite because as it is now, in certain cases, may not work. In my case, this never works... So, I believe that for now, we'll need to wait longer to use the latest Deno version. Until there is a proper fix there is no way I can support >= 1.9.0...
Just for reference, I read this comment, which made me join Deno's Discord server to check out the discussion on the #dev channel.

@fdionisi Thank you for letting me know. I will keep an eye on new releases for this repo 💪🏻

Hello 😄,
Are there any updates ? Are there any plans to use the ffi to make this work again ? Would really love to use argon2 instead of bcrypt 👍🏻

@mmarkmos, not many plans right now. The project started as part of a POC I was working on, but it ended up not being pursued as Deno was still relatively young.
Is not on top of my priorities, but I'd be happy to review the code if you don't mind preparing a PR for ffi.


Make it possible to supply an existing py4j gateway object

Always spawning a new JVM doesn't make sense if the application has other uses for py4j too. This would also make integration possible with projects like asphalt-py4j.

Hi @agronholm, this is already a feature! You can pass an existing gateway object to py4jdbc.connect (See https://github.com/massmutual/py4jdbc/blob/master/py4jdbc/dbapi2.py#L16). Please give it a try and let me know if it doesn't work the way you expected.

My apologies, I missed this!


Trigger on hover

How could I trigger transition on hover?
Can I customize how the transition is triggered?

You can add mouseenter and mouseout listeners to the fab and invoke the speed dial / morph open() and close() respectively. If you want to disable the transition for the click event, you can add a mousedown listener to the fab and cancel the event.

Thanks!


TODO

To-Do

[x] Tokenize Keywords in a single pass
[x] Tokenize Joint Symbols (+=) as a secondary pass
[x] Tokenize all valid ASCII-based symbols
[x] Token Metadata for error reporting (line position & cursor position)
[x] Reorganize Tokenizer
[x] Parser error handling improvements
[x] Reorganize Parser
[x] Rewrite Parser
[x] Caller Syntax (Main; = Main();)
[x] Return statement
[x] Qualified Declarations
[x] "const" assignment is in-place
[x] Parser can parse conditionals (if, else if, else)
[x] Code generator can emit conditionals
[x] Type casting
[ ] Arbitrary scoping
[ ] Parser can parse loops (while, for)
[ ] Code generator can emit loops
[ ] Parser can parse classes
[ ] Code generator can emit classes
[ ] Parser can parse uniforms
[ ] Code generator can emit uniforms
[ ] Parser can parse switch-case
[ ] Implicit case numbering in switches
[ ] Switch Range Case (case 4...7)
[ ] Code generator can re-order function parameters (defaults)

Very promising project. Here are my thoughts:

guard the standard output header stuff in some kind of #ifdef statement
should the generated C printf add \n at the end if needed?
I still see other issues with printf; maybe call a custom VA_ARGS C function?
comments are not copied to the C output

Hey, thanks for the feedback - I will consider this! The project was created for a college assignment and I'm not sure how far I'll really go with it.

Hey, thanks for the feedback - I will consider this! The project was created for a college assignment and I'm not sure how far I'll really go with it.


Clear editor history?

Hi there,
I use this library to render an editor in a single page app.  The editor renders code in a pop-up for editing.  On the first page load, Undo/Redo works fine, but if I close the pop-up and render a different piece of code the Undo history is there for my previous rendered code (causing an Undo to revert to the other codebase).
So I'd like to know how I can programmatically clear the Undo/Redo history, so we get a new history for each rendering of the editor.
Thanks!

We can do it on destroy but if you use editor inside keep-alive el than you should remove the history with a method or sth. If u'd like to contribute let me know(on destroy clear history) or I can do it later this week.

Thanks, I decided to switch to Codemirror.


[PG16] Fix unnecessary expression assignment

blocked by https://github.com/muhammadshoaib/age/pull/20


[Suggestion] Config to allow stack size number UI to appear even on < 64 stack items

Not sure if this is simple/worthwhile to implement given the scope of the mod may focus on only stacks > 64

No, this mod made to combines drops without changing the vanilla stack mechanic.
Change how stack work usually break a lot of stuff

Oh I miss understood what you're talking about lol


Fix bug where mm is mistakenly replaced with hmm in e.g. 20mm

The English normaliser mistakenly replaces 20mm to 20hmm. In this case "mm" is the unit postfix millimetre.
This was caused by treating 10 as number and thus splitting "10" and "mm". The "mm" token was then further replaced with "hmm" according to the English.json mapping.
Removing "mm" from the mapping shouldn't be a problem, since there is already a condition before that that would remove "mm" words entirely from the input.
"mhm" and "mmm" could probably be removed for the same reason.

Thanks. Those replacers are from the post-processing scripts for the CHiME dataset:
https://github.com/kaldi-asr/kaldi/blob/ae8cbe8858f2a66a9b193c82dbe3b0479364165f/egs/chime5/s5/local/wer_output_filter#L19-L21
but I agree it'd be not very relevant to keep it at this point.


How implemented the PageCoordinator ?

I have the root coordinator, HomeTabCoordinator, and this router I set two tabbar items, AboutController and NewsController and created for them coordinate classes.
For NewsCoordinator I have next router:
enum NewsListRouter: Route {
    case newsList
    case selectedNews(newsID: Int)
}

and I open the NewsDetailViewController next way:
func detailNewsTrigger(_ id: Int) {
        router.trigger(.selectedNews(newsID: id))
    }

For this screen I have the next design, buttons for news paging:

How do I implement the function, when tapped for NextButton, showing next news and also, the previewButton, showing preview news ?
I know about the PageCoordinator, but no idea how it implemented for my project, although looking at your example HomePageCoordinator. Please, help

PageCoordinator seems to be an interesting idea here, although I'd say it would require a custom UIPageViewControllerDelegate in which the next or previous view controllers are loaded (at least, if lazy loading is required, which it probably should).
I'm first of all ignoring data flow, since I don't know how the existing project is set up.
For changing the currently displayed page from the tap of a button use PageTransition.set(_:direction:) with the to-be-shown viewController(s) and the direction in which the animation should be shown (just like ´UIPageViewController.setViewControllers`).
If swiping is required/wanted, use the UIPageViewControllerDelegate methods in your delegate that you are providing in the initializer to generate viewControllers depending on the swipe direction. (If you are overriding generateRootViewController this delegate might not be set though).

For a route type with cases left and right, the following code illustrates a way, where one would only need to implement a UIPageViewControllerDataSource and connect the buttons to the respective route cases.
case .left:
    guard let currentViewController = rootViewController.viewControllers?.first,
        let previousViewController = rootViewController.dataSource?
            .pageViewController(rootViewController, viewControllerBefore: currentViewController) else {
                return .none()
    }
    return .set(previousViewController, direction: .reverse)
case .right:
    guard let currentViewController = rootViewController.viewControllers?.first,
        let nextViewController = rootViewController.dataSource?
            .pageViewController(rootViewController, viewControllerAfter: currentViewController) else {
                return .none()
    }
    return .set(nextViewController, direction: .forward)

I'm closing this now - if the question / problem persists or there are follow-up questions, feel free to reopen.

thanks for the answer!

@pauljohanneskraft, I need implement a PageCoordinatorDataSource ? And if so, how I create controller and use the pages stored properties ?

It would be cool to see an example with a similar implementation that I brought

With the most recent version of XCoordinator, you can also create a PageCoordinator without being constrained to the PageCoordinatorDataSource, but you can instead use any UIPageViewControllerDataSource.

In UIPageViewControllerDataSource how add controller to 'pages' stored properties ?

In UIPageViewControllerDataSource there is no stored property pages, this is only the case for our implementation PageControllerDataSource.
Please refer to the UIKit documentation for further questions regarding the UIPageViewControllerDataSource protocol.
If you have further questions regarding the XCoordinator framework, feel free to ask, but please keep in mind, that without elaborating on your problem, it is really hard to understand the problem and as a result of that, it is hard to find a solution.


Update CI for emacs 28

There are a few updates here, but I think they can go in a single PR:

After upgrading to 28.1.90, Emacs decided to create a completely new elpa-28.1.90 directory and reinstall all packages. I've extended .gitignore to account for that.
Update to CI to include Emacs 28.1.
Update CI so it will pass tests with my taps.
Update CI to make macOS more likely to pass. Apparently there was a race condition when melpa packages are installed. It, however, cannot be resolved in a batch mode - the ask-user-about-lock signals error. Documentation for the latter suggest it's intended for the user to override it. I decided to take a strategy of waiting for 5s then grabbing the lock anyway (and keeping fingers crossed that the other thread has already released the lock). N.B. from the way it works I gather the stylised lock is not a lock like i.e., mutex. It's more likely a marker that some resource is being used, that can be overwritten by the opponent.
Update the exordium-flip-string-quotes. The Emacs 28 has changed how python-mode parser sees universally delimeted strings (the ones with triple quotes). In Emacs up to 28 it was a single string, now it there are three: one made with first two opening quote characters, one starting from the third opening quote character up to the first closing quote character, and one made with the last two closing quote characters (try (forward-sexp) with point at the strategic places). I updated the function to account for that (and still working in Emacs 27).

Looks good to me. I’ll merge.


seems gpu not been used

I run python3 train.py --model='Tacotron' with default parameters in hparams.py. But when I check GPU using with nvidia-smi, Gpu-Util is 0%.
Is there any one got same question?
Thank you very much!

Check out how to find the connected gpu devices on tensorflow- https://stackoverflow.com/questions/38009682/how-to-tell-if-tensorflow-is-using-gpu-acceleration-from-inside-python-shell.
If the gpu device is not in there, make sure you have tensorflow-gpu installed

@sid-sundrani  Thank you very much!
This issue troubled me few days.
I will try as your suggestion!

Hi @pandaGst ! I am facing the same issue. Have you found a solution yet?

Thanks. @pandaGst It worked for me as well now. I agree GPU utilization is really low.

This issue is inactive for a long time, close now. And reopen it if necessary.


Can't change the color if LED strip is configured on Output other than one.

Please see configuration. If LED strip is set on the Output other than one, can't change the colors, same time can change bigness  and on/off. With Output one it works, but I dot's want to use Input 1, as it can reset the device.

Configuration file: z_led.txt

@DJTerentjev It seems that it is a bug if Z2M frontend. You can change the color from MQTT explorer if you send a command like:
Topic: z2m/0x00124b0022055177/set
JSON: {"color_l2":{"hex":"#FFFF41"}}


Aggregation simplify declarations

Found an issue.  I'll update once I address that.

Looks good. Just a few minor comments on naming, and please fix the commit message typos.


Owls 76524: implement DBPing for JRF domain integ test

Implement DBPing for JRF domain integ test during test class prepare phase.
Internal Jenkins run: http://wls-jenkins.us.oracle.com/view/weblogic-operator/job/weblogic-kubernetes-operator-javatest/2473/consoleFull
There are 2 DBPing (searching "DB Service is Ready to accept Connection" ) happened for 2 JRF test classes.

@vanajamukkara @anpanigr after you review the change, I will sync to the latest dev branch and remove the below commented out lines
// TODO: reconsider the logic to check the db readiness
// The jrfdomain can not find the db pod even the db pod shows ready, sleep more time
//logger.info("waiting for the db to be visible to rcu script ...");
//Thread.sleep(20000);


Generating RLayerVector objects outside of RMap

Hello,
I am using this library in a React app where I am required to create components outside of my the RMap object. Is there any way, using RContext maybe, to generate RLayerVector objects outside of RMap and pass them in as children using the spread operator?

Currently, an RLayerVectordoes not support being detached and reattached to another RMap- this is the missing feature

I think you need useRef for each RLayerVector in parent component, i'm working on that also, but the problem is when using RControl.Rlayer and you put RLayerVector inside or outside there with custom event handler yourself it's not work, I have code like this.
on parent component
const vector1 = useRef()
<RMap>
   <RLayerVector vector1Ref={vector1}/>
</RMap>

and then on your child component define ref on RLayerVector, and also mostly you can see that ref console.log(props.vector1Ref)


Issue63

login with google works, but with Github does not...

thank you for your comments, Fred.


Win7 compatibility

Is it possible to use new tracker on windows7?
api-ms-win-core-path.dll error on launch
or maybe you have that previous version before you converted everything to Py 3.9, bc thats probably the case, Py 3.9 unsopported on win7

The 3.8 series still works on windows 7, you could try installing python 3.8.9 and running the tracker yourself from source if rchardon decides not to support windows 7

Yeah, its just what i did, a bit complicated, but at least working solution

Yeah I approve what hyphen said.


[Question] mouse double-click?

Hi,
Under the Mouse section of Cheet Sheet -> Functions, I cannot find double-click. Is there a command for that?
https://pit-ray.github.io/win-vind/cheat_sheet/functions/#mouse
Thanks!

You can double-click like 2FF or 2o.
Thanks.

Makes sense!
This might seem like a strange follow-up question: I see that the caret gets the mouse position when I switch from GUI NORMAL to EDIT NORMAL, would it be possible to have the reverse option? I.e, to have the mouse jump to where the caret is located when switching from EDIT NORMAL to GUI NORMAL? In Word, there are times when this would be very helpful, such as when double-clicking a footnote to jump to it and back.

I am not sure if it is possible to get the coordinates of the caret. Previously, I tried to get the coordinates using the Windows API, but some applications could not get the coordinates. I will try to see if can use UIA, or minor APIs etc. This issue will be left open.
Thanks.


Ensure ToggleVisibility interacts with separators and bleed correctly

Release
Renderer status
Tasks status

1.2
🔄 .NET (#2604)🔄 Android (#2605)🔄 iOS (#2606)✔ TS (#2607)🔄 UWP (#2608)
✔ Shared (#2602)✔ Designer (#2603)

Many of our renderers likely don't handle scenarios around ToggleVisibility and separators/bleed. We should ensure this works correctly on all our renderers. See below...
So it looks like ToggleVisibility's effect is solely to change the Visibility property of the framework element associated with the specified Adaptive element Id. I just wanted to bring attention to a couple things:

Does the framework element include the target element's potentially visible separator? If not, then ToggleVisibility may result in an orphaned separator
Does setting the framework element's visibility trigger a re-layout of the card to make sure that bleeding elements bleed correctly? For instance, take the following payload:

{
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "type": "AdaptiveCard",
    "version": "1.0",
    "body": [
        {
            "type": "TextBlock",
            "id": "title",
            "text": "This is a TextBlock at the top of the card",
            "wrap": true
        },
        {
            "type": "Container",
            "style": "emphasis",
            "bleed": true,
            "items": [
                {
                    "type": "TextBlock",
                    "text": "This is a bleeding container",
                    "size": "extraLarge"
                },
                {
                    "type": "ActionSet",
                    "actions": [
                        {
                            "type": "Action.ToggleVisibility",
                            "title": "Click me",
                            "targetElements": [
                                "title"
                            ]
                        }
                    ]
                }
            ]
        }
    ]
}

Initially it renders like this:

Once the button is clicked, it should look like this:

Auto-generated task status

[x] Shared
[x] Designer
[ ] .NET
[ ] Android
[ ] iOS
[x] TS
[ ] UWP

Approving. Please investigate + fix. thanks

@almedina-ms please close once all related bugs are addressed in this area. Assigning to you for tracking. Tx


🎋 Fix #11: Id clash when using multiple Trees

Based on #13 (will update and rebase once that's in), to have the egui/serde dependency enabled.
This PR is an implementation to solve issue #11.
This is achieved by storing an egui::Id for each tree. In order to get the TiledId::egui_id  the Tree's id is passed. The value of the Tree's id is determined by a user passed value, thus this is a breaking change.
I opted to allow serde to serialize and deserialize the passed value, as adding a id_source kind of function, which needs to be called after deserialization felt bad. I also opted to have the user pass an Into<egui::Id> rather than obtaining it from the egui::Ui passed in the Tree::ui function, as that will cause the TileIds to break when the user draws the same Tree within a different Ui context.

Ran into this issue today - I've pulled that PR in my fork - works. Thanks :)

I'll rebase this once #13 lands.

I'm finishing this up in https://github.com/rerun-io/egui_tiles/pull/32

Sorry, this slipped my mind! I thought I enabled you to push to this branch, but fixing it in #32 is fine!


next/release

:tada: This PR is included in version 1.14.19 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Facing issues while launching prometheus and the adapter using the walkthrough

Hi,
Im trying to follow the walkthrough to deploy the custom-metrics-adapter, but i run into a couple of issues.

When creating the resource-lister clusterrole i get an error as mentioned the below:
src/github.com/prometheus/prometheus# kubectl create clusterrole resource-lister --verb=list --resource="*"
the server doesn't have a resource type "*".

Also for now i use --resource=pods just to be able to create the clusterrole to proceed.

While trying to create the deployment and expose it as a service it fails as shown below:
src/github.com/prometheus/prometheus# kubectl -n prom create -f prom-adapter.deployment.yaml
The Deployment "prometheus" is invalid: spec.template.spec.containers[0].volumeMounts[0].name: Not found: "prom-config"

Has anyone run into something similar? Is there something missing in my cluster setup/configs?

Thanks, il do the change to the yaml.
Yes i was following the walkthrough and i did create the configmap with the same prom-cfg.yaml that was mentioned above(https://github.com/DirectXMan12/k8s-prometheus-adapter/blob/master/docs/walkthrough.md) as it said it might just be the above.
I also tried the example deployment(https://github.com/DirectXMan12/k8s-prometheus-adapter/tree/master/deploy). Modified as necessary to point to my prometheus server. And executed the below commands:
export PURPOSE=serving
openssl req -x509 -sha256 -new -nodes -days 365 -newkey rsa:2048 -keyout ${PURPOSE}-ca.key -out ${PURPOSE}-ca.crt -subj "/CN=ca"
echo '{"signing":{"default":{"expiry":"43800h","usages":["signing","key encipherment","'${PURPOSE}'"]}}}' > "${PURPOSE}-ca-config.json"
kubectl create secret tls cm-adapter-serving-certs --cert=./serving-ca.crt --key=./serving-ca.key
kubectl create namespace custom-metrics
cd deploy
kubectl create -f manifests/

The pod shows up with status
NAMESPACE        NAME                                        READY     STATUS              RESTARTS   AGE
custom-metrics   custom-metrics-apiserver-66db884599-2xp4c   0/1       ContainerCreating   0          49s

On doing a describe, i see a volume mount error as shown below:
# kubectl describe pod custom-metrics-apiserver -n custom-metrics
Name:           custom-metrics-apiserver-66db884599-2xp4c
Namespace:      custom-metrics
Node:           XXXXXXXXXXXX
Start Time:     Tue, 24 Apr 2018 00:53:49 -0400
Labels:         app=custom-metrics-apiserver
                pod-template-hash=2286440155
Annotations:    <none>
Status:         Pending
IP:
Controlled By:  ReplicaSet/custom-metrics-apiserver-66db884599
Containers:
  custom-metrics-apiserver:
    Container ID:
    Image:         directxman12/k8s-prometheus-adapter
    Image ID:
    Port:          6443/TCP
    Args:
      /adapter
      --secure-port=6443
      --tls-cert-file=/var/run/serving-cert/serving.crt
      --tls-private-key-file=/var/run/serving-cert/serving.key
      --logtostderr=true
      --prometheus-url=http://localhost:9090/
      --metrics-relist-interval=30s
      --rate-interval=5m
      --v=10
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from custom-metrics-apiserver-token-pppk8 (ro)
      /var/run/serving-cert from volume-serving-cert (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  volume-serving-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  cm-adapter-serving-certs
    Optional:    false
  custom-metrics-apiserver-token-pppk8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  custom-metrics-apiserver-token-pppk8
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                 Age               From               Message
  ----     ------                 ----              ----               -------
  Normal   Scheduled              1m                default-scheduler  Successfully assigned custom-metrics-apiserver-66db884599-2xp4c to XXXXXXXX
  Normal   SuccessfulMountVolume  1m                kubelet, XXXXXXXX MountVolume.SetUp succeeded for volume "custom-metrics-apiserver-token-pppk8"
  Warning  FailedMount            25s (x8 over 1m)  kubelet, XXXXXXXX  MountVolume.SetUp failed for volume "volume-serving-cert" : secrets "cm-adapter-serving-certs" not found

I dont really know where have i gone wrong. Any help would be appreciated.

I also faced the issue here:
https://github.com/DirectXMan12/k8s-prometheus-adapter/blame/master/docs/walkthrough.md#L219
$ kubectl create clusterrole resource-lister --verb=list --resource="*"
the server doesn't have a resource type "*"

But if I do kubectl create -f it works with the following
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: resource-lister
rules:
- apiGroups:
  - ""
  resources:
  - '*'
  verbs:
  - list```

Thanks @zioproto .
Also I noticed that the prom-adapter.deployment.yaml file needs a change as the volumeMounts and volumes need to have the same name.
volumeMounts:
        # you'll mount your ConfigMap volume at /etc/prometheus
        - mountPath: /etc/prometheus
          name: prom-config
      volumes:
      # make your configmap available as a volume, so that you can
      # mount in the Prometheus config from earlier
      - name: config-volume
        configMap:
          name: prometheus

closing, since it seems like we've fixed these issues


erro file read

now it is possible to generate the dll using swing
but to an error in it using the convert method generate the following error in the image

also always returns the error
Attempt to read or write to protected memory. This is usually an indication that other memory is bad.

I rebuilt the DLL try this...
https://drive.google.com/file/d/1JsUlTYjofIw45tisZ4RNVLB7Vp1RVvjL/view?usp=sharing
it works for me
On Mon, Sep 13, 2021 at 11:27 AM Fabricio @.***> wrote:

also always returns the error
Attempt to read or write to protected memory. This is usually an
indication that other memory is bad.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Embroidermodder/libembroidery/issues/76#issuecomment-918363310,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADBZHGETKYCBT4CTK67N6LUBYQ53ANCNFSM5D53GYBA
.


Incorrect filter field documented for the query string.

The two issues below include a filter query for the field space_guid. This is a duplicate of the guid field in the path and should be removed.
http://apidocs.cloudfoundry.org/226/spaces/list_all_service_instances_for_the_space.html
http://apidocs.cloudfoundry.org/226/spaces/list_all_managers_for_the_space.html

This should be addressed by our fix for issue #496

http://apidocs.cloudfoundry.org/230/spaces/list_all_service_instances_for_the_space.html
http://apidocs.cloudfoundry.org/230/spaces/list_all_managers_for_the_space.html
Looks good. Closing.


Fix path for d.ts files

:tada: This PR is included in version 1.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


ENH: multi-output class weights

Currently, we handle single-output class weights. Keras itself should have support for multi-output class weights, but that feature is broken (https://github.com/keras-team/keras/issues/4735) and there doesn't seem to be any plan to fix it.
Since we have access to all of sklearn's tools, we can relatively easily implement class_weights for multiple outputs, specifically, we can implement class weights via sample weights (Keras also supports a sample weight vector per output). In fact, sklearn implements a utility to convert class weights to sample weights, and it even supports multiple outputs, but it assumes n_outputs = y.shape[1], which isn't generally true with Keras data. This can be remedied by taking sklearn's compute_sample_weights function and modifying it slightly; it shouldn't be too hard.
Do you think this is worth implementing in SciKeras @stsievert ?

I'm not seeing the need to write any code. This sounds like an upstream implementation issue, but there is some (small) value in providing an implementation to convert between class weights to sample weights. If I were developing this library, I'd provide that implementation in the docs.
Why are multi-output models worth focusing on? I've only seen single output models, and to me, the value of SciKeras is in providing a Scikit-learn API to simple Keras models and handling serialization.

I'm not seeing the need to write any code. This sounds like an upstream implementation issue, but there is some (small) value in providing an implementation to convert between class weights to sample weights. If I were developing this library, I'd provide that implementation in the docs.
Why are multi-output models worth focusing on? I've only seen single output models, and to me, the value of SciKeras is in providing a Scikit-learn API to simple Keras models and handling serialization.

Thank you for the quick reply.

This sounds like an upstream implementation issue

I do really agree on this point: this should be fixed in Keras itself. I also don't like the idea of using SciKeras as a crutch for Keras features.

I'd provide that implementation in the docs

That's certainly an option, it does reduce the amount of helper code in the library.

I've only seen single output models

Do you mean that you've only seen SciKeras users use single output models? I've seen SciKeras users use multi-output models (eg: https://github.com/dask/dask-ml/issues/764).

The implementation looks pretty easy. The main problem is going to be that computing sample weights requires having the target y already split up into a list of inputs. Currently, we do this before any user-defined transformers, so this would not be possible. I think this could be resolved by #167 since the sample weights could then be yielded as a Dataset (like here).

The implementation looks pretty easy. The main problem is going to be that computing sample weights requires having the target y already split up into a list of inputs. Currently, we do this before any user-defined transformers, so this would not be possible. I think this could be resolved by #167 since the sample weights could then be yielded as a Dataset (like here).


Added Windows 2022 and Ubuntu 22.04 configuration

boost/1.7.9 is the minimum supported version for both Windows 2022 and Ubuntu 22.04 For Windows, msvc toolset is chosen, otherwise install-boost will randomly choose between msvc or mingw. At the moment, cmake configuration doesn't work with boost when mingw is selected.
Signed-off-by: Kevin Lim<EMAIL_ADDRESS>

@nikhalim, I'm not sure what GitHub thinks is blocking this PR? Maybe its because we've changed the matrix for the PR/CI that it is getting confused...

If you're unaware of something, maybe we can force merge and monitor for failures in the CI and next PR.

Ah, GitHub Actions doesn't report status if the job uses a matrix (IMHO this is an unfortunate decision).
We have to list each strategy we want to wait on in the required status checks.
I'll merge this PR and configure the repo to use them.


Feature: preorders

I want to merge this change because it adds preorder feature to Saleor.
Only ProductVariants could be in preorder state; staff user can set global threshold and/or channel threshold. Customer can not purchase more preorder variants than threshold. When order is created, preorder variant is allocated; when preorder is finished, preorder allocations are removed and regular allocation objects are created instead.

Impact

[x] New migrations
[x] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations
[x] Documentation needs to be updated

Pull Request Checklist

[ ] Privileged queries and mutations are guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migration files are up to date
[ ] The changes are tested
[ ] GraphQL schema and type definitions are up to date
[ ] Changes are mentioned in the changelog

Task linked: SALEOR-2998 Preorders


Failing to install apxs

I got error: apxs binary is not executable
sudo apt-get install apache2-dev helped there

Thanks, this helped me.

on linux debian 7.11 wheezy you need to install apache2-threaded-dev which provides apache2-dev

Thanks. In Fedora is:
sudo dnf install httpd-devel


A warning is printed when running tests using testerina

Description:
When we run the tests after the new Package API change, there is a warning printed:
Could not write to Rerun Test json. Rerunning tests will not work:<rpoject_source>/target/caches/json_cache/ballerina/reflect/0.5.2/rerun_test.json (No such file or directory)

The tests are passing even this warning is printed.
You can check the log from here:
https://github.com/ballerina-platform/module-ballerina-reflect/pull/24/checks?check_run_id=1430679466
Building ballerina-reflect module can reproduce this issue.

Fixed with : https://github.com/ballerina-platform/ballerina-lang/pull/27081


This or That Game App[Android]

Requirements for the app:-

User can see two images
User can select it's favorite from the two images
After a selection is made, the other 2 images are displayed

Requirements for the README.md file:-

Concepts used
Documentation and resources used
Future Aspects of the app

Note:- Add the app to the Android Projects section

@kunal299 Can I work on this?

Sure, You can start working on it.

@kunal299 I have made a pr #524 , Please have a look
Thanks!


makeOct1axis function throws file not found error when Radiance is not installed

I was trying to run the bifacial_radiance Python package for the first time today, and got the following error:

After discussing it with Silvana on teams, it turns out I wasn't following the install instructions and needed to install Radiance.
Fix the error message to notify users that they may need to install Radiance if this function fails.

This is fixed now by pull #537 in Main.  It should go into our next release post 0.4.3 - either 0.4.4 or 0.5.0 depending on which one we get out first...


Hi, I added my new page

Your pull request seems good, but must be checked by a human maintainer for the following reason(s):

This contains potentially hazardous tags that need manual review.

Please be patient until a maintainer has time to look over your contribution.
This is an automated response, beep boop.
if you think there is a mistake, contact a maintainer

e9a0260f1f525cf16c8ba4d2b5031e8b2ed7c543

LGTM, thank you for your contribution!


[BUG] Mocking ollama endpoint got - Unexpected end-of-input: expected close marker for Object

Describe the bug

When testing Ollama generate endpoint ('/api/generate') with help of mockserver to mocking response. I got error
java.lang.RuntimeException: com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])

Log and Stack trace

java.lang.RuntimeException: com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 2]
        at dev.langchain4j.model.ollama.OllamaJsonUtils.toObject(OllamaJsonUtils.java:30)
        at dev.langchain4j.model.ollama.OllamaClient$1.onResponse(OllamaClient.java:124)
        at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:161)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 2]
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportInvalidEOF(ParserMinimalBase.java:585)
        at com.fasterxml.jackson.core.base.ParserBase._handleEOF(ParserBase.java:535)
        at com.fasterxml.jackson.core.base.ParserBase._eofAsNextChar(ParserBase.java:552)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipWSOrEnd(ReaderBasedJsonParser.java:2491)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:673)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4905)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3848)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3816)
        at dev.langchain4j.model.ollama.OllamaJsonUtils.toObject(OllamaJsonUtils.java:28)
        ... 6 more

To Reproduce

To reproduce we have to get mockserver and JSON file with mocked endpoints
https://www.mock-server.com/mock_server/getting_started.html
JSON file
[
  {
    "id": "1",
    "httpRequest": {
      "method": "POST",
      "path": "/api/generate"
    },
    "httpResponse": {
      "statusCode": 200,
      "headers": {
        "Content-Type": "application/json"
      },
      "body": {
        "model": "llama2",
        "created_at": "2023-08-04T08:52:19.385406455-07:00",
        "response": "Mocking response from model",
        "done": true
      }
    }
  }
]

Then method with connection to model
 public String generate(String mockServerEndpoint) {
 StreamingLanguageModel ollama = OllamaStreamingLanguageModel.builder()
            .baseUrl(mockServerEndpoint)
            .modelName("llama2")
            .timeout(Duration.ofSeconds(120L))
            .build();

        CompletableFuture<String> future = new CompletableFuture<>();
        ollama.generate("tell me a joke?", new StreamingResponseHandler<String>() {
            @Override
            public void onNext(String token) {
                message[0] += token;
            }

            @Override
            public void onComplete(dev.langchain4j.model.output.Response<String> response) {
               future.complete(response.toString());
            }

            @Override
            public void onError(Throwable error) {
                error.printStackTrace();
            }
        });
        future.join();

        return message[0];
}

and test
@Test
testGenerateEndpoint() {
 var response = given()
                .when()
                .body(mockserverUrl)  //passing mockserver url to method
                .post("/generate")
                .then()
                .statusCode(200)
                .extract().body().as(String.class);
        assertThat(response).isNotNull();
        assertThat(response).isEqualTo("Mocking response from model");
}

Expected behavior

No exception is thrown during tests
Please complete the following information:

LangChain4j version: 0.35
LLM(s) used: llama2
Java version: 17
Quarkus version: 3

Additional context

Mocking endpoint works on version 0.32, I updated to 0.35 due to fix other problems

@bbrynski Hi! After running your code in real Ollama endpoint, I think the problem is not in our side.
Could you please check the streaming repsonse using MockServer? I think the response is different from real Ollama endpoint. In real Ollama endpoint, every streaming partial response is like:
{
  "model" : "tinydolphin",
  "created_at" : "2024-10-29T02:30:20.632152678Z",
  "response" : " Why",
  "done" : false
}

I tried with your response and I got the same error

I tried with your response and I got the same error

Could you please debug in OllamaClient#streamingCompletion method and see the value of variable partialResponse?


Threadfin Will Not Update

Please first read the documentation. Problems affecting the client, e.g. Plex or Emby should be reported there in the forum.
Describe the bug
A clear and concise description of what the bug is.
Server (please complete the following information):

OS: Windows 10 64bit
Threadfin Version: 1.0 (23)

To Reproduce
Steps to reproduce the behavior:

Open Threadfin
Terminal gives error: 2024-05-30 12:44:46 [Threadfin] Start Update:           Branch: Main
2024-05-30 12:44:46 [Threadfin] [ERROR] bad status: 404 Not Found (Update failed) - EC: 6002

Expected behavior
Threadfin to update automatically.
Desktop (please complete the following information):

OS: Windows 10 64bit
Browser: Chrome or Brave

Additional context
I have tried downloading and running later exe files, but they fail during the terminal boot process.

Join the Discord and ping me please
https://discord.gg/CNaSkER2zD


BUG: ImportError: undefined symbol: ndarray_from_pyobj

Describe your issue.
I am trying to run SciPy's test suit to create a pull request, but following the steps in https://scipy.github.io/devdocs/dev/contributor/conda_guide.html
errors at step 5. python dev.py test -v.
The test yields hundreds of errors of the kind
E   ImportError: PATH/scipy/build-install/lib/python3.10/site-packages/scipy/sparse/linalg/_isolve/_iterative.cpython-310-x86_64-linux-gnu.so: undefined symbol: ndarray_from_pyobj

I added the output to the file: output.txt
Reproducing Code Example
git submodule update --init
mamba env create -f environment.yml
mamba activate scipy-dev
python dev.py build
python dev.py test -v

### Error message

```shell
_ ERROR collecting build-install/lib/python3.10/site-packages/scipy/cluster/tests/test_disjoint_set.py _
ImportError while importing test module 'PATH/scipy/build-install/lib/python3.10/site-packages/scipy/cluster/tests/test_disjoint_set.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
PATH/mambaforge/envs/scipy-dev/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
scipy/cluster/__init__.py:27: in <module>
    from . import vq, hierarchy
scipy/cluster/vq.py:72: in <module>
    from scipy.spatial.distance import cdist
scipy/spatial/__init__.py:110: in <module>
    from ._kdtree import *
scipy/spatial/_kdtree.py:4: in <module>
    from ._ckdtree import cKDTree, cKDTreeNode
_ckdtree.pyx:10: in init scipy.spatial._ckdtree
    ???
scipy/sparse/__init__.py:289: in <module>
    from . import csgraph
scipy/sparse/csgraph/__init__.py:185: in <module>
    from ._laplacian import laplacian
scipy/sparse/csgraph/_laplacian.py:7: in <module>
    from scipy.sparse.linalg import LinearOperator
scipy/sparse/linalg/__init__.py:120: in <module>
    from ._isolve import *
scipy/sparse/linalg/_isolve/__init__.py:4: in <module>
    from .iterative import *
scipy/sparse/linalg/_isolve/iterative.py:9: in <module>
    from . import _iterative
E   ImportError: PATH/scipy/build-install/lib/python3.10/site-packages/scipy/sparse/linalg/_isolve/_iterative.cpython-310-x86_64-linux-gnu.so: undefined symbol: ndarray_from_pyobj

SciPy/NumPy/Python version and system information
$ python -c "import sys, numpy; print(numpy.__version__, sys.version_info);"

1.24.2 sys.version_info(major=3, minor=10, micro=10, releaselevel='final', serial=0)

$ python -c "import scipy"
Traceback (most recent call last):
  File "/home/andreasw/workspace/code/develop/fix_scipy/scipy/scipy/__init__.py", line 111, in <module>
    from scipy.__config__ import show as show_config
ModuleNotFoundError: No module named 'scipy.__config__'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/andreasw/workspace/code/develop/fix_scipy/scipy/scipy/__init__.py", line 116, in <module>
    raise ImportError(msg) from e
ImportError: Error importing SciPy: you cannot import SciPy while
        being in scipy source directory; please exit the SciPy source
        tree first and relaunch your Python interpreter.

Addendum: Git SHA: `85e0d3ed4f0d88c97c832251fbc17e3688ab519a`

ndarray_from_pyobj comes from numpy.f2py, but I've never seen an issue with it. @DerWeh do you happen to have a self-built numpy version installed? Or PYTHONPATH set up to point to some numpy build, or another such thing?

Thanks for the hint. Never did any self-building, and NumPy seems to be rightly detected as the one installed by mamba:
$ python -c "import numpy as np; print(np.__file__)"
PATH/mambaforge/envs/scipy-dev/lib/python3.10/site-packages/numpy/__init__.py

I have not set PYTHONPATH, and sys.path evaluates to:
$ python -c "import sys; print(sys.path)"
[
    '',
    'PATH/mambaforge/envs/scipy-dev/lib/python310.zip',
    'PATH/mambaforge/envs/scipy-dev/lib/python3.10',
    'PATH/mambaforge/envs/scipy-dev/lib/python3.10/lib-dynload',
    'PATH/mambaforge/envs/scipy-dev/lib/python3.10/site-packages',
    'REPOPATH/scipy'
]

Admittedly, I have a plethora of different Python installations, but to me, it seems like everything is cleanly pointing to the SciPy environment.
I'll try if a clean micromamba environment would work.

Ok, a fresh installation of micromamba works after all.
Took me way too long to figure out that I should run rm -r build after changing the environment, maybe a command python dev.py clean would be helpful?

Ah that makes sense as a cause. I don't think that python dev.py clean is helpful though, because there's no way to clean every possible root cause. E.g., you may need git clean -xdf and we definitely can't run that automatically from dev.py -setup.py clean was always unreliable too for similar reasons.
The better way is to detect potential problems and raise an informative error message, rather than letting an obscure error about missing symbols bubble up. I'm still not 100% clear on how to do that in this case. Can you tell me what the old and new numpy and python versions in your environment were? If I can reproduce with that, I should be able to improve the error message.

python dev.py won't re-run when the meson version changes. If that's the same but environment changes, there may be ways to get around this. That's user error though - I'm not quite sure how to prevent it, or even to reproduce this. So I'll close the issue. With a clean reproducer this can be improved perhaps (but not 100% sure about that).


Update README.md to reflect the latest version

Update readme to reflect the latest version, as following the readme no longer works

Thank you

Thank you


Add openbazaar as an optional service

Installing with dietpi-software was not successfull on first try, so here is good guide: https://www.zokos.com/blog/site/public/2019/01/12/Self-Host your Own OpenBazaar Store/

https://github.com/openoms/bitcoin-tutorials/tree/master/openbazaar


Upload, install and hide doesn't work

it goes to <IP_ADDRESS> everytime and installs the virus it doesn't work

Ow. Ok. I'll fix it soon

Hey, download the latest release (https://github.com/perfilrobo/hexbot/releases/tag/v<IP_ADDRESS>) and try again. Please, inform me if this is working or not.

OK. I've just found the bug. It is fixed. Download the latest version: https://github.com/perfilrobo/hexbot/releases (v<IP_ADDRESS>)

It works now but almost after each log clearing it stops and I have to refresh for it to complete

Does it always happen? It happens for me too but just once in a while. I think it's due the ocasional internet slow. When there is progress bar running, the bot does nothing. It just waits for page to reload. Anyway once it happens again, check the browser console, copy and paste the jSpaghetti logs here.


Image validation

Dear,
We implemented this easy to use library but we noticed an issue when applied for a certain customer of ours. Most of the time, a captcha appears where you have to check a box saying "I'm not a robot" and in that case, everything works fine... Only, from time to time, a captcha appears where you have to validate an image and as a result, the captcha could not be validated.
There is no such mention of this option so my question is, is there a possibility to support image validation? Or disable this option so it always results in the checkbox? Or is there a possibility within the Google Recaptcha website to do something about it?
Kind regards,
Kevin Cocquyt

Please read how it works here, it will try to validate automatic the user, if it can't it will prompt a challenge:
https://www.google.com/recaptcha/intro/


API - Font Measurements: Make smoothing type explicitly required

This is the revery-side fix for: https://github.com/onivim/oni2/issues/1364
The measurement / metrics of character width are impacted by the font smoothing settings (specifically, whether subpixel text is allowed).
Because this is an important consideration for measurements, we should make our measurement APIs take a required argument for this - to ensure the caller has considered the smoothing type.

@bryphe Do you know the the GHA also format code.

@Et7f3 - oh, neat, didn't realize! That's very cool


Support for Kotlin Multiplatform and Ktor

Are you requesting automatic instrumentation for a framework or library? Please describe.

Framework or library name :  KMP, Ktor
Library type: KMP library
Library version: ktor version. 2.3.4

Is your feature request related to a problem? Please describe.
We are currently using Ktor in one of our KMP modules. None of the network calls are reported in Datadog. All the other network calls that are done with retrofit are reported.
Describe the solution you'd like
If Datadog is installed in the App module it should automatically trace the network requests that are done with the Ktor HttpClient
Describe alternatives you've considered
We didn't find an alternative solution

hi @ThomasRichtsfeld, thanks for this feature request. We don't currently have plans to support KMP. Regarding Ktor we will discuss and see whether it is possible.

From our POV we don't need support for KMP, as long as tracing works for Ktor out of the box.
Thanks for the update @jonathanmos

Any updates on this ussue? It looks like we are not the only ones with this need: https://www.reddit.com/r/ktor/comments/10584a3/ktor_client_tracing_and_monitoring_for_a_kotlin/

Hello @tassilo-posegga! There is no update on this topic so far, it is not at the top of our list, but if we see that this feature is highly requested this will definitely change its priority for us.

Ktor is growing bigger and bigger, it would be great to have support for this.

Hi everyone 👋
Good news: we recently released a KMM SDK in private beta. It currently supports

RUM (incl. crash reporting, network request tracking through Ktor, and webview tracking),
Logs,
and frontend-to-backend APM distributed tracing,

both for iOS and Android.
There are some caveats and the feature set isn't 100% complete obviously, but we're actively looking for more customers to partner with us and help us shape the roadmap for this new SDK.
So if you're welling to give it a try, could you reach out to your account manager (or to support) to get this enabled for your organisation please (not through this thread for obvious confidentiality reasons 😊)?

Hi everyone 👋
Good news: we recently released a KMM SDK in private beta. It currently supports

RUM (incl. crash reporting, network request tracking through Ktor, and webview tracking),
Logs,
and frontend-to-backend APM distributed tracing,

both for iOS and Android.
There are some caveats and the feature set isn't 100% complete obviously, but we're actively looking for more customers to partner with us and help us shape the roadmap for this new SDK.
So if you're welling to give it a try, could you reach out to your account manager (or to support) to get this enabled for your organisation please (not through this thread for obvious confidentiality reasons 😊)?

That's great news 🥳
Would we also need to update our SDK on the clients (Android and iOS) or is this only a setting in our admin panel on the web

Hello @ThomasRichtsfeld! It is a new SDK (which is using Android/iOS SDKs) under the hood, so you will have to integrate it in your app.

@0xnm can you share the version with us?
I guess we can then just integrate the SDK on Android/iOS and test if the data shows up correctly on the web without changing anything else. Am I right?

@ThomasRichtsfeld Please contact our support, because anyway KMP SDK functionality on web needs to be activated per organization basis and we don't have any public docs yet.

We now have Kotlin Multiplatform SDK repository opened as public and we also published the necessary documentation.
Kotlin Multiplatform SDK still stays in the PREVIEW, so if you want to use it open a ticket with our support to activate it for your org.


HTTP Errors

ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).
I've upped the time limit to 2400 and it still didn't work? I'm running on AWS, and I don't believe I have a slow connection, so I'm at a loss. Any suggestions?

Try running it locally first?

OK, I'll give that a try. I should have mentioned that it did work for a couple of days just pulling it.

I've tried to build it locally with git clone, but seem to encounter the same problem. I don't know how to setup debugging or provide you with any logs. :(

The build succeeded? What about testing 2.2 image locally with compose?

Get the example in the project and use 2.2 image.
Boot it on foreground with DMS_DEBUG=1 env.
Paste the entire log here.
If compose does not work, it's probably not related to this image.
Try with another one.

OK, the problem isn't with this container, it seems to be with docker itself having lost it's configuration file.

docker.auth.auth.load_config: File doesn't exist

OK, I'm going to close this issue  but feel free to add a comment you have a solution, It may help other users


Collapse long stack traces in the logs into a disclosure widget

Sometimes Flutter exceptions are hundreds of stack frames deep. It'd be good if when Atom detected a stack trace was more than a few lines long, it hid the rest under a disclosure triangle or something.

Just to add some color, showing the first few (5?) stack frames is good. Everything after that would be better collapsed.

Going to close this in favor of the IntelliJ plugin. Not sure how that will display long stack traces, but something to keep an eye on.
cc @devoncarew


void sleep(uint16_t t) is way too fast in 1T mode

It is 10 times faster than it should be.
For a 11.0592MHz oscillator, replacing void _sleep_1ms(void) with the code below will make a more reasonable delay.
void _sleep_1ms(void)
{
    __asm
        push ar5
        push ar6
        mov ar5,#9
        mov ar6,#145
    NEXT:
        djnz ar6,NEXT
        djnz ar5,NEXT
        pop ar6
        pop ar5
        ret
    __endasm;
}

Thanks for your feedback.
Because the STC11 series uses 1T instruction set, the sleep function written by assembly takes less time than the 12T instruction set.
we will fix it soon.

Apologies I didn't get my stc board back to test the code changes till this morning.
When I tried to commit a new PR I found the changes had been merged. I didn't know it can be done from upstream... so does it mean no further changes are needed?

Apologies I didn't get my stc board back to test the code changes till this morning.
When I tried to commit a new PR I found the changes had been merged. I didn't know it can be done from upstream... so does it mean no further changes are needed?

The new changes have been tested and passed,  so there is no further changes needed

Cool! Thanks~


Version 0.4.30

Landed in https://github.com/metarhia/impress-cli/commit/35764158e05947eefd36fac11b242c0f97a0e5a3


AddedArrayCode

Added c++ code for finding an array element appears once while others are appearing twice.

Please star this repository to show your support, in the meantime I am reviewing your code.