Error when compiling the Bert large model

Describe the bug
Hello, I have an error trying to compile the Bert large model. The error is as below
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: error: 'util.global.store' op global type mismatch; global __iree_flow___sm_node20__optimizer.iter is 'tensor<i64>' but store is 'tensor<i32>'
/home/vivian/.local/lib/python3.9/site-packages/keras/optimizer_v2/optimizer_v2.py:671:0: note: called from
/home/vivian/iree-samples/ModelCompiler/nlp_models/bert_small_gen.py:60:0: note: called from
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1136:0: note: called from
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py:3831:0: note: called from
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py:677:0: note: called from
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1161:0: note: called from
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py:3130:0: note: called from
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py:3292:0: note: called from
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py:3036:0: note: called from
/home/vivian/.local/lib/python3.9/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: note: see current operation: "util.global.store"(%2036) {global = @__iree_flow___sm_node20__optimizer.iter} : (tensor<i32>) -> ()
compilation from source to vm failed

To Reproduce

Generate the Bert large model using the script from here https://github.com/google/iree-samples/blob/main/ModelCompiler/nlp_models/bert_large_gen.py
Run command with CUDA NVIDIA V100
./iree-compile -iree-input-type=mhlo -iree-mlir-to-vm-bytecode-module -iree-hal-target-backends=cuda -iree-cuda-llvm-target-arch=sm_70 -iree-hal-cuda-disable-loop-nounroll-wa /home/vivian/iree-samples/ModelCompiler/nlp_models/model.mlir -o model.vmfb

Additional context

IREE was built with this commit cbea6b2670f56ba20134408b222d41c911f36cf8
IREE pip packages 20220325.87

Looks like yet another i64 issue :-/ @raikonenfnu looks like you added this model. Can you reproduce?

@GMNGeoffrey thanks, I'll take a look. since it's also involving hal global, I was wondering if @benvanik can also help take a peek :)

I have reproduced it and can fix with this
`~/github/iree-samples/ModelCompiler/nlp_models$ diff bert_small_mar25_i32.mlir bert_small_mar25.mlir
54c54
<   "iree_input.global"() {initial_value = dense<0> : tensor, is_mutable, sym_name = "__iree_flow___sm_node20__optimizer.iter", sym_visibility = "private", type = tensor} : () -> () loc(#loc0)

"iree_input.global"() {initial_value = dense<0> : tensor, is_mutable, sym_name = "__iree_flow___sm_node20__optimizer.iter", sym_visibility = "private", type = tensor} : () -> () loc(#loc0)
140c140
<     %31 = "mhlo.constant"() {value = dense<1> : tensor} : () -> tensor loc(#loc25)

%31 = "mhlo.constant"() {value = dense<1> : tensor<i64>} : () -> tensor<i64> loc(#loc25)

183c183
<     %74 = "iree_input.global.address"() {global = @__iree_flow___sm_node20__optimizer.iter} : () -> !iree_input.ptr<tensor> loc(#loc0)

%74 = "iree_input.global.address"() {global = @__iree_flow___sm_node20__optimizer.iter} : () -> !iree_input.ptr<tensor<i64>> loc(#loc0)

1141,1143c1141,1143
<     %763 = "iree_input.global.load.indirect"(%74) : (!iree_input.ptr<tensor>) -> tensor loc(#loc506)
<     %764 = "chlo.broadcast_add"(%763, %31) : (tensor, tensor) -> tensor loc(#loc506)
<     "iree_input.global.store.indirect"(%764, %74) : (tensor, !iree_input.ptr<tensor>) -> () loc(#loc506)

%763 = "iree_input.global.load.indirect"(%74) : (!iree_input.ptr<tensor<i64>>) -> tensor<i64> loc(#loc506)
%764 = "chlo.broadcast_add"(%763, %31) : (tensor<i64>, tensor<i64>) -> tensor<i64> loc(#loc506)
"iree_input.global.store.indirect"(%764, %74) : (tensor<i64>, !iree_input.ptr<tensor<i64>>) -> () loc(#loc506) `

Still exists in the latest nightly

140c140
<     %31 = "mhlo.constant"() {value = dense<1> : tensor<i32>} : () -> tensor<i32> loc(#loc25)
---
>     %31 = "mhlo.constant"() {value = dense<1> : tensor<i64>} : () -> tensor<i64> loc(#loc25)

If this is an i64 constant issues with small values, it may be fixed in https://github.com/google/iree/pull/8738

oh cool. I see it hasn't landed yet. So I need to pass a flag once it lands ?

I don't think so. The flag is on by default in the PR.
// TODO(#8745): remove these flags when the -iree-flow-demote-* flags can be
// used without tripping upstream verifier issues.
static llvm::cl::opt<bool> clDemoteI64ToI32(
    "iree-mhlo-demote-i64-to-i32",
    llvm::cl::desc(
        "Converts all MHLO i64 ops and values into i32 counterparts."),
    llvm::cl::init(true));
static llvm::cl::opt<bool> clDemoteF64ToF32(
    "iree-mhlo-demote-f64-to-f32",
    llvm::cl::desc(
        "Converts all MHLO f64 ops and values into f32 counterparts."),
    llvm::cl::init(true));

I don't have full context about this today. The i64->i32 behavior has bee on on by default, this looks like an NFC to MHLO path.
There may be some issues get addressed in the PR. I think it's better to start another look after landing the PR.

^ correct - the flag now allows the existing default behavior to be disabled (pass -iree-mhlo-demote-i64-to-i32=false) - we can look at flipping the default in the future but I didn't want to do that all in the initial PR.

(iree_env) 1 anush@shared-a100-debian:~/github/iree$ ./build/iree/tools/iree-translate --iree-mlir-to-vm-bytecode-module --iree-hal-target-backends=cuda --iree-llvm-target-cpu-features=host --iree-input-type=mhlo  --iree-vm-target-extension-i64 --iree-vm-target-truncate-unsupported-integers --iree-vm-target-truncate-unsupported-floats --iree-vm-target-extension-f64 --iree-mhlo-demote-i64-to-i32=true --iree-vm-target-extension-f32 /home/anush/github/iree-samples/ModelCompiler/nlp_models/model.mlir  -o bert_small_apr12_cuda.vmfb
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: error: 'util.global.store' op global type mismatch; global __iree_flow___sm_node20__optimizer.iter is 'tensor<i64>' but store is 'tensor<i32>'
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:675:0: note: called from
bert_small_gen.py:60:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1143:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3831:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:677:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1161:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3140:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3292:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3036:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: note: see current operation: "util.global.store"(%2036) {global = @__iree_flow___sm_node20__optimizer.iter} : (tensor<i32>) -> ()
compilation from source to vm failed`
````(iree_env) 1 anush@shared-a100-debian:~/github/iree$ ./build/iree/tools/iree-translate --iree-mlir-to-vm-bytecode-module --iree-hal-target-backends=cuda --iree-llvm-target-cpu-features=host --iree-input-type=mhlo  --iree-vm-target-extension-i64 --iree-vm-target-truncate-unsupported-integers --iree-vm-target-truncate-unsupported-floats --iree-vm-target-extension-f64 --iree-mhlo-demote-i64-to-i32=true --iree-vm-target-extension-f32 /home/anush/github/iree-samples/ModelCompiler/nlp_models/model.mlir  -o bert_small_apr12_cuda.vmfb
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: error: 'util.global.store' op global type mismatch; global __iree_flow___sm_node20__optimizer.iter is 'tensor<i64>' but store is 'tensor<i32>'
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:675:0: note: called from
bert_small_gen.py:60:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1143:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3831:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:677:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1161:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3140:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3292:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3036:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: note: see current operation: "util.global.store"(%2036) {global = @__iree_flow___sm_node20__optimizer.iter} : (tensor<i32>) -> ()
compilation from source to vm failed

Steps to reproduce:
1: git clone [iree-samples ](https://github.com/google/iree-samples)
2: git submodule update --init
3: ./setup_venv.sh  #setup your virtual env
4:  source /home/anush/github/iree-samples/iree-samples.venv/bin/activate #source your venv
5: export PYTHONPATH=$PYTHONPATH:$PWD/ModelCompiler/tf_models  #setup python path
6: cd ModelCompiler/nlp_models/
7: python bert_small_gen.py #generate MLIR file model.mlir you can use with translate / run-module
Model: "bert_classifier"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to
==================================================================================================
 input_word_ids (InputLayer)    [(None, None)]       0           []

 input_mask (InputLayer)        [(None, None)]       0           []

 input_type_ids (InputLayer)    [(None, None)]       0           []

 bert_encoder (BertEncoder)     [(None, None, 768),  15250176    ['input_word_ids[0][0]',
                                 (None, 768)]                     'input_mask[0][0]',
                                                                  'input_type_ids[0][0]']

 dropout_1 (Dropout)            (None, 768)          0           ['bert_encoder[0][1]']

 sentence_prediction (Classific  (None, 5)           3845        ['dropout_1[0][0]']
 ationHead)

==================================================================================================
Total params: 15,254,021
Trainable params: 15,254,021
Non-trainable params: 0
__________________________________________________________________________________________________
2022-04-13 14:46:35.729494: W tensorflow/python/util/util.cc:368] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:absl:Found untraced functions such as dropout_2_layer_call_fn, dropout_2_layer_call_and_return_conditional_losses, logits_layer_call_fn, logits_layer_call_and_return_conditional_losses, self_attention_layer_call_fn while saving (showing 5 of 64). These functions will not be directly callable after loading.
<class 'bytes'>
Wrote MLIR to path '/home/anush/github/iree-samples/ModelCompiler/nlp_models/model.mlir'

8: Run with iree-translate like:
(iree_env) 141 anush@shared-a100-debian:~/github/iree$ ./build/iree/tools/iree-translate --iree-mlir-to-vm-bytecode-module --iree-hal-target-backends=cuda --iree-llvm-target-cpu-features=host --iree-input-type=mhlo  --iree-vm-target-extension-i64 --iree-vm-target-truncate-unsupported-integers --iree-vm-target-truncate-unsupported-floats --iree-vm-target-extension-f64 --iree-mhlo-demote-i64-to-i32=true --iree-vm-target-extension-f32 /home/anush/github/iree-samples/ModelCompiler/nlp_models/model.mlir  -o bert_small_apr12_cuda.vmfb
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: error: 'util.global.store' op global type mismatch; global __iree_flow___sm_node20__optimizer.iter is 'tensor<i64>' but store is 'tensor<i32>'
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:675:0: note: called from
bert_small_gen.py:60:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1143:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3831:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:677:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1161:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3140:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3292:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3036:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: note: see current operation: "util.global.store"(%2036) {global = @__iree_flow___sm_node20__optimizer.iter} : (tensor<i32>) -> ()
compilation from source to vm failed

It will fail compilation.
You can also run directly from python with bert_small_run.py :
Model: "bert_classifier"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to
==================================================================================================
 input_word_ids (InputLayer)    [(None, None)]       0           []

 input_mask (InputLayer)        [(None, None)]       0           []

 input_type_ids (InputLayer)    [(None, None)]       0           []

 bert_encoder (BertEncoder)     [(None, None, 768),  15250176    ['input_word_ids[0][0]',
                                 (None, 768)]                     'input_mask[0][0]',
                                                                  'input_type_ids[0][0]']

 dropout_1 (Dropout)            (None, 768)          0           ['bert_encoder[0][1]']

 sentence_prediction (Classific  (None, 5)           3845        ['dropout_1[0][0]']
 ationHead)

==================================================================================================
Total params: 15,254,021
Trainable params: 15,254,021
Non-trainable params: 0
__________________________________________________________________________________________________
2022-04-13 14:48:58.271306: W tensorflow/python/util/util.cc:368] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:absl:Found untraced functions such as dropout_2_layer_call_fn, dropout_2_layer_call_and_return_conditional_losses, logits_layer_call_fn, logits_layer_call_and_return_conditional_losses, self_attention_layer_call_fn while saving (showing 5 of 64). These functions will not be directly callable after loading.
Traceback (most recent call last):
  File "bert_small_run.py", line 68, in <module>
    flatbuffer_blob = compile_str(compiler_module, input_type="mhlo", target_backends=["cuda"])
  File "/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/iree/compiler/tools/core.py", line 293, in compile_str
    result = invoke_immediate(cl, immediate_input=input_bytes)
  File "/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/iree/compiler/tools/binaries.py", line 200, in invoke_immediate
    raise CompilerToolError(process)
iree.compiler.tools.binaries.CompilerToolError: Error invoking IREE compiler tool iree-compile
Diagnostics:
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0: error: 'util.global.store' op global type mismatch; global __iree_flow___sm_node20__optimizer.iter is 'tensor<i64>' but store is 'tensor<i32>'
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:675:0: note: called from
bert_small_run.py:61:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1143:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3831:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:677:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1161:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3140:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3292:0: note: called from
/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3036:0: note: called from
compilation from source to vm failed

Invoked with:
 iree-compile /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/iree/compiler/tools/../_mlir_libs/iree-compile - --iree-input-type=mhlo --iree-vm-bytecode-module-output-format=flatbuffer-binary --iree-hal-target-backends=cuda --iree-mlir-to-vm-bytecode-module --iree-llvm-embedded-linker-path=/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/iree/compiler/tools/../_mlir_libs/iree-lld --mlir-print-debuginfo --mlir-print-op-on-diagnostic=false

Need more information? Set IREE_SAVE_TEMPS=/some/dir in your environment to save all artifacts and reproducers.

It starts of valid in the generated .mlir file:
iree_input.global"() {initial_value = dense<0> : tensor<i64>, is_mutable, sym_name = "__iree_flow___sm_node20__optimizer.iter", sym_visibility = "private", type = tensor<i64>} : () -> () loc(#loc0)

Seems like print-before-all flag is not plumbed through in iree-translate (or needs additional option)

It starts of valid in the generated .mlir file:
iree_input.global"() {initial_value = dense<0> : tensor<i64>, is_mutable, sym_name = "__iree_flow___sm_node20__optimizer.iter", sym_visibility = "private", type = tensor<i64>} : () -> () loc(#loc0)

Seems like print-before-all flag is not plumbed through in iree-translate (or needs additional option)

Are you thinking of print-ir-before-all, which after latest integrate is mlir-print-ir-before-all ?

You are correct, there is the variant with and without -ir- going to the different parts and I was using the wrong one. I added the crash reproducer in this section too as that should make it simpler, but dumping should be sufficient to get to jsut before pass that failed.

module {
  iree_input.global private mutable @__iree_flow___sm_node20__optimizer.iter = dense<0> : tensor<i64>
  func @main() {
    %0 = mhlo.constant dense<1> : tensor<i64>
    %1 = iree_input.global.address @__iree_flow___sm_node20__optimizer.iter : !iree_input.ptr<tensor<i64>>
    %2 = iree_input.global.load.indirect %1 : !iree_input.ptr<tensor<i64>> -> tensor<i64>
    %3 = arith.addi %2, %0 : tensor<i64>
    iree_input.global.store.indirect %3, %1 : tensor<i64> -> !iree_input.ptr<tensor<i64>>
    return
  }
}

post ./iree/tools/iree-opt --iree-util-demote-i64-to-i32 results in
module {
  iree_input.global private mutable @__iree_flow___sm_node20__optimizer.iter = dense<0> : tensor<i64>
  func @main() {
    %0 = mhlo.constant dense<1> : tensor<i32>
    %1 = iree_input.global.address @__iree_flow___sm_node20__optimizer.iter : !iree_input.ptr<tensor<i64>>
    %2 = iree_input.global.load.indirect %1 : !iree_input.ptr<tensor<i64>> -> tensor<i32>
    %3 = arith.addi %2, %0 : tensor<i32>
    iree_input.global.store.indirect %3, %1 : tensor<i32> -> !iree_input.ptr<tensor<i64>>
    return
  }
}

where one has a store of an i32 to an i64 location.

Oh global store indirect doesn't consider element type during verification, so an error is only flagged later when converted to global store. That doesn't seem correct, I'd have expected global store indirect to also care about the element type here.

The util pass only works on util ops - not iree_input

So there is another pass needed here for the iree_input?

But here this pass did work on iree_input.global.load.indirect

(sorry for brevity, was on phone)
It worked by chance as it's just going through ops and reinterpret casting types. It's a pass intended to be used via the -iree-flow-demote-i64-to-i32 flag or on IR that's already been lowered into IREE's internal dialects. So I think this issue is that the -iree-flow-demote-i64-to-i32 flag needs to be used instead of manually running the pass on the origin IR.

This is local reproducer of what happens when IREE compiler is invoked via --iree-mhlo-demote-i64-to-i32=true. The op gets converted into an invalid state due to running this before lowering to internal dialects. And indeed once lowered to Util dialect the pass works and converts the ops.
--iree-mhlo-demote-i64-to-i32=false --iree-flow-demote-i64-to-i32 works (so disabling the MHLO demotion and leaving it until flow demotion). At least when I did
iree/tools/iree-compile --iree-mlir-to-vm-bytecode-module --iree-hal-target-backends=dylib --iree-llvm-target-cpu-features=host --iree-input-type=mhlo  --iree-vm-target-extension-i64 --iree-vm-target-truncate-unsupported-integers --iree-vm-target-truncate-unsupported-floats --iree-vm-target-extension-f64 --iree-mhlo-demote-i64-to-i32=false --iree-flow-demote-i64-to-i32 --iree-vm-target-extension-f32 ../iree-samples/ModelCompiler/nlp_models/model.mlir  -o bert_small_apr12_cuda.vmfb

it worked (I don't have CUDA backend installed and the above failed at CUDA backend stage)

@jpienaar Thank you so much. That work around works for bert_small_gen.py. I tested both cpu and cuda.
However with bert_large_gen.py it compiles ok but it fails at runtime.
(iree_env) anush@shared-a100-debian:~/github/iree$ ./build/iree/tools/iree-benchmark-module --driver=cuda --function_input=1x512xi32 --function_input=1x512xi32 --function_input=1x512xi32 --function_input=1xi32 --entry_function=learn --benchmark_repetitions=10  --module_file=bert_large_apr15_cuda.vmfb
NOTE: IREE is not a BLAS library and this tool is intended
to be used when benchmarking entire programs. Precise timing
of automatically generated device-side execution is not
available here.
I /home/anush/github/iree/iree/tools/utils/vm_util.cc:154] Creating driver and device for 'cuda'...
/home/anush/github/iree/iree/hal/cuda/native_executable.c:110: INTERNAL; CUDA driver error 'CUDA_ERROR_INVALID_VALUE' (1): invalid argument; while invoking native function hal.executable.create; while calling import;

[ 1]   native hal.executable.create:0 -
[ 0] bytecode module.__init:13530 [
    "Sum:",
    "gradient_tape/bert_classifier/bert_encoder/transformer/layer_23/self_attention/softmax/Sum@__inference_learn_23167"(/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/imperative_grad.py:73:0)
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/backprop.py:1087:0
      at bert_large_gen.py:59:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1143:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3831:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:677:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1161:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3140:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3292:0      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3036:0
  ]

I think it is OOM for the HAL 250M parameter model
I /home/anush/github/iree/iree/tools/utils/vm_util.cc:154] Creating driver and device for 'dylib'...
NOTE: IREE is not a BLAS library and this tool is intended
to be used when benchmarking entire programs. Precise timing
of automatically generated device-side execution is not
available here.
2022-04-15T16:56:06-07:00
Running ./build/iree/tools/iree-benchmark-module
Run on (12 X 2200.14 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x6)
  L1 Instruction 32 KiB (x6)
  L2 Unified 1024 KiB (x6)
  L3 Unified 39424 KiB (x1)
Load Average: 1.48, 1.55, 1.17
/home/anush/github/iree/iree/base/allocator.c:93: RESOURCE_EXHAUSTED; system allocator failed the request; while invoking native function hal.allocator.allocate; while calling import;
[ 1]   native hal.allocator.allocate:0 -
[ 0] bytecode module.learn:13008 [
    "AssignAddVariableOp:",
    "SGD/SGD/AssignAddVariableOp@__inference_learn_23167"(/home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/distribute/merge_call_interim.py:51:0)
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:675:0
      at bert_large_gen.py:60:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1143:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3831:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py:677:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:1161:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3140:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3292:0
      at /home/anush/github/iree-samples/iree-samples.venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py:3036:0,
    "ResourceApplyGradientDescent:",

so basically with the workaround you suggested we are good to compile. And at runtime we hit:
https://github.com/google/iree/issues/8653

Great, I'll look into flipping these so the small one tests again and we can add it to testing (unless someone already did while I was otherwise occupied 🙂). The i64 here is only loop count (and should be completely unnecessary that big), so glad this then gets to next part

I don't think this is in any test. Yes adding it in tests would be great since it is the only functional training model that I am aware of.

With the recent work on #8661 I was able to run this model. I passed all zeros and got 16.1181 as an output - assuming that's expected then this may be fixed once all the pending changes land!
I used these flags:
        -iree-mhlo-demote-i64-to-i32=false
        -iree-stream-resource-index-bits=64
        -iree-vm-target-index-bits=64

As a sanity check it'd also be useful to know what kind of memory this model uses on tf - I am embarrassed by what we're using now but would be good to know how embarrassed I should be :)

Assuming the results are correct this has been fixed in #8661 - if someone wants to verify we can close this out!

I am using the latest nightly build to test the 250M parameter model available at https://github.com/google/iree-samples/blob/main/ModelCompiler/nlp_models/bert_large_run.py
It fails with:

Traceback (most recent call last):                                                                                │
  File "/home/anush/data/anush/iree-samples/ModelCompiler/nlp_models/bert_large_run.py", line 76, in <module>                                                                                │
    flatbuffer_blob = compile_str(compiler_module, target_backends=[backend], extra_args=args, input_type="mhlo")                                                                                │
  File "/home/anush/data/anush/iree-samples/iree-samples.venv/lib/python3.9/site-packages/iree/compiler/tools/core.py", line 293, in compile_str                                                                        │
    result = invoke_immediate(cl, immediate_input=input_bytes)                                                                                │
  File "/home/anush/data/anush/iree-samples/iree-samples.venv/lib/python3.9/site-packages/iree/compiler/tools/binaries.py", line 200, in invoke_immediate                                                               │
    raise CompilerToolError(process)                                                                                │
iree.compiler.tools.binaries.CompilerToolError: Error invoking IREE compiler tool iree-compile                                                                                │
Diagnostics:                                                                                │
<unknown>:0: error: 'vm.list.get.ref' op operand #1 must be 32-bit signless integer, but got 'i64'                                                                                │
compilation from source to vm failed                                                                                │
                                                                                │
                                                                                │
Invoked with:                                                                                │
 iree-compile /home/anush/data/anush/iree-samples/iree-samples.venv/lib/python3.9/site-packages/iree/compiler/tools/../_mlir_libs/iree-compile - --iree-input-type=mhlo --iree-vm-bytecode-module-output-format=flatbuff│
er-binary --iree-hal-target-backends=dylib-llvm-aot --iree-mlir-to-vm-bytecode-module --iree-llvm-embedded-linker-path=/home/anush/data/anush/iree-samples/iree-samples.venv/lib/python3.9/site-packages/iree/compiler/t│
ools/../_mlir_libs/iree-lld --mlir-print-debuginfo --mlir-print-op-on-diagnostic=false --iree-llvm-target-cpu-features=host --iree-mhlo-demote-i64-to-i32=false --iree-flow-demote-i64-to-i32 --iree-stream-resource-ind│
ex-bits=64 --iree-vm-target-index-bits=64                                                                                │
                                                                                │
Need more information? Set IREE_SAVE_TEMPS=/some/dir in your environment to save all artifacts and reproducers.                                                                                │

iree-compiler                20220527.151
iree-runtime                 20220527.151
iree-tools-tf                20220527.151
iree-tools-tflite            20220527.151
iree-tools-xla               20220527.151

Remove the iree-flow-demote-i64-to-i32 flag

Still same error:
Diagnostics:
<unknown>:0: error: 'vm.list.get.ref' op operand #1 must be 32-bit signless integer, but got 'i64'
compilation from source to vm failed

Invoked with:
 iree-compile /home/anush/data/anush/iree-samples/iree-samples.venv/lib/python3.9/site-packages/iree/compiler/tools/../_mlir_libs/iree-compile - --iree-input-type=mhlo --iree-vm-bytecode-module-output-format=flatbuffer-binary --iree-hal-target-backends=dylib-llvm-aot --iree-mlir-to-vm-bytecode-module --iree-llvm-embedded-linker-path=/home/anush/data/anush/iree-samples/iree-samples.venv/lib/python3.9/site-packages/iree/compiler/tools/../_mlir_libs/iree-lld --mlir-print-debuginfo --mlir-print-op-on-diagnostic=false --iree-llvm-target-cpu-features=host --iree-mhlo-demote-i64-to-i32=false --iree-stream-resource-index-bits=64 --iree-vm-target-index-bits=64

Need more information? Set IREE_SAVE_TEMPS=/some/dir in your environment to save all artifacts and reproducers.

weirdly if I copy paste the command it compiles outside of Python:
(iree-samples.venv) anush@anush-large-nvme:~/data/anush/iree-samples/ModelCompiler/nlp_models$ /home/anush/data/anush/iree-samples/iree-samples.venv/lib/python3.9/site-packages/iree/compiler/tools/../_mlir_libs/iree-compile --iree-input-type=mhlo --iree-vm-bytecode-module-output-format=flatbuffer-binary --iree-hal-target-backends=dylib-llvm-aot --iree-mlir-to-vm-bytecode-module --iree-llvm-embedded-linker-path=/home/anush/data/anush/iree-samples/iree-samples.venv/lib/python3.9/site-packages/iree/compiler/tools/../_mlir_libs/iree-lld --mlir-print-debuginfo --mlir-print-op-on-diagnostic=false --iree-llvm-target-cpu-features=host --iree-mhlo-demote-i64-to-i32=false --iree-stream-resource-index-bits=64 --iree-vm-target-index-bits=64 model.mlir -o model.vmfb

dunno bout python stuff, maybe it's adding flags for you; that command looks ok to me

I believe it is printing exactly what executes, after all python-foo. https://github.com/google/iree/blob/b4967867b2eff4434c0f2ffb8d6f04e2bd1fe48b/compiler/src/iree/compiler/API/python/iree/compiler/tools/binaries.py#L182

Ashley, that is the log. Here is the exception: https://github.com/google/iree/blob/b4967867b2eff4434c0f2ffb8d6f04e2bd1fe48b/compiler/src/iree/compiler/API/python/iree/compiler/tools/binaries.py#L77

Anything to do here?

I don't think there is anything left for this case, more work needed for larger type support, part of that is captured by index conversion work. Closing, we can create new one for new model or more general support beyond this.


Add LandingSection and HeaderUnderline component

Add reusable components for the landing section of the Pitch and Get Started page.
I was careful to only affect the landing section Pitch and Get Started. I want to merge these changes before applying the changes more broadly to reduce the number of pages we have to check for breaking changes.

You will need to resolve conflicts before you can merge


Human prompts for dataset generation.

We instruct the user model with carefully designed prompts to mimic human user behavior

@ningding97 could you provide these prompts you used? Thanks!

hi, I attempted to reproduce its procedure but the human often does not raise questions but evaluates the assistants' answers.
what the prompt is appended to the dialogue history mean?
do you append the system prompt to every request's content?

Yes, we put everything under the role "user"  for openai chatgpt api. So for example, if you already have the first round of conversation as below
User: user utterance 1
Assistant: assistant response 1

Now you want to generate a new user utterance, then your input message is
{
   "role": "user",
   "content": """User: user utterance 1
Assistant: assistant response 1
Above is a conversation between a user and an intelligent assistant. Now suppose you are the user, say something to continue the conversation based on given context. Make the response short and the language casual.
"""
}


Added --auto to script

Related to  Issue: Terraform/Ansible Script Broken #734
Added this for those that want to install reconFTW as part of an automated installation.
--auto flag will skip the menu and proceed with the normal terminal based installation.
Also: potential.json file has been removed from where it was previously stored. Added a local assets directory to store it for now and copy it over during the running of the installation script.

Thanks a lot for the PR @lwears
Moved to dev branch in order to follow the deployment process :)


Add csp_server_client for Zephyr

In order to add a version of csp_server_client, I have to fix a few places.  This PR has these fixes as well as a version of csp_server_client for Zephyr.

Oops.  The semaphore change was accidentally included in this PR.  I thought I'd create a dedicated PR for the semaphore change because we'd need a discussion for it.  If you think I should create a different PR just for semaphore change, please let me know.

I've created #328 and rebased this PR on top of it.


Why Illuminate\Database\Eloquent\Builder do not use the Macroable trait ?

Hi,
I need to create a macro on the Eloquent Builder.
I thinked it could be possible to do the same as I do on Query Builder, but all macro logic of the first one, is not base on the rular trait and is not a static api.
Best regards,

It is macroable?

https://github.com/laravel/framework/blob/v5.2.29/src/Illuminate/Database/Query/Builder.php#L21

https://github.com/laravel/framework/blob/v5.2.29/src/Illuminate/Database/Query/Builder.php#L2239

Sorry @GrahamCampbell, I did not understand your answer.
The Eloquent Builder implement some kind of Macro logic but without the Macroable Trait and not on the static way :
https://github.com/laravel/framework/blob/v5.2.29/src/Illuminate/Database/Eloquent/Builder.php#L1301
https://github.com/laravel/framework/blob/v5.2.29/src/Illuminate/Support/Traits/Macroable.php#L24
The purpose of my question is to know more about the choice to use another kind of logic especially for the Eloquent Builder that is not the same as the Query Builder.
And Why Not : a way to implement static macro an the Eloquent Builder as the same as I did it on the Query Builder.

We don't process proposals to change behaviour here. Please see our contribution guide.
To be brutally honest, I wouldn't bother raising another issue, since this has already been discussed, and a decision already made.

Oh my bad. My english seem poor.
This is not a proposal. This is something more like a curiosity to fix a missed comprehension about two things that seem identical in my mind but implement opposite logics.
With a bonus question about a way to solve my needs.
Sorry to make you to lose your time.


License source URL

Since I needed the license source URL as a reference, I added it to the conversion output. If you think this is unnecessary, feel free to reject.
Thanks for a very useful module!

FYI https://github.com/shinnn/spdx-license-ids/blob/master/build.js#L13

Ahh, thanks. But I also needed the full license text, which I suppose is not available over there.

SPDX maintains full license text files. They're on spdx.org, but not yet exported.


update pass-client version to 0.8.0

update surefire plugin version to 2.22.2
update README for pass-client configuration change

After review, this PR looks good.  Builds are failing on my local machine, but they are succeeding on Travis and on @jrmartino's machine.  I suspect there may be a bug related to my platform's combination of the Docker Maven Plugin and Docker versions in use.


Gitleaks reports "LinkedIn Client ID" errors on some public LinkedIn profiles

Describe the bug
I have a Markdown file which contains links to people profiles on LinkedIn.
Some of those links are reported as errors even though their profile can be browsed anonymously.
To Reproduce
Steps to reproduce the behavior:
Inside an empty directory create a file test.md with the following contents:
# Test gitleaks

Testing some LinkedIn profiles:

* [Gianpaolo Macario](https://www.linkedin.com/in/gmacario)
* [Olivier Charra](https://www.linkedin.com/in/oliviercharra)

<!--
Screenshots
Actual result from executing the commands listed above:
gmacario@gmpowerhorse:~/tmp/test-linkedin (master)$ docker run -v ${PWD}:/my-repo zricethezav/gitleaks:latest --path="/my-repo" -v
time="2021-11-14T16:22:10Z" level=info msg="opening /my-repo\n"
{
        "line": "* [Olivier Charra](https://www.linkedin.com/in/oliviercharra)",
        "lineNumber": 6,
        "offender": "linkedin.com/in/oliviercharra",
        "offenderEntropy": -1,
        "commit": "e3ea3c0692c5d10a727a0bb332332fd8701566cd",
        "repo": "my-repo",
        "repoURL": "",
        "leakURL": "",
        "rule": "LinkedIn Client ID",
        "commitMessage": "Add file\n",
        "author": "Gianpaolo Macario",
        "email"<EMAIL_ADDRESS>        "file": "test.md",
        "date": "2021-11-14T17:22:01+01:00",
        "tags": "client, LinkedIn"
}
time="2021-11-14T16:22:10Z" level=info msg="scan time: 1 millisecond 514 microseconds"
time="2021-11-14T16:22:10Z" level=info msg="commits scanned: 1"
time="2021-11-14T16:22:10Z" level=warning msg="leaks found: 1"
gmacario@gmpowerhorse:~/tmp/test-linkedin (master)$

Basic Info (please complete the following information):

OS: Ubuntu Server 20.04.3 LTS 64-bit
Gitleaks Version: Docker latest

Additional context
N.A.
cc @zricethezav

Hello @gmacario !
I'm a little bit confused, you say first:
Some of those links are reported as errors even though their profile can be browsed anonymously.
I understand that gitleaks shouldn't report the links as a leak.
Yet, then you say:
I would expect to leaks found.
Which is actually what's happening.
If you want gitleaks not to report those links as a leaks you can always use an Allowlist, explained here:
https://github.com/zricethezav/gitleaks#example-4-using-allowlist-regex
I can see you pointed this issue to another one -> https://github.com/github/super-linter/issues/2129
Pointing to this file -> https://github.com/github/super-linter/blame/main/TEMPLATES/.gitleaks.toml#L56-L59
gitleaks is behaving as it should.
From here, possible solutions:

As said earlier, create an allowlist
If this has to do with the file pointed in that other issue, it's not an issue with gitleaks, but with that other repo instead

Cheers!

Hello @gmacario !

Hello @w0rmr1d3r, thank for your quick reply.

I'm a little bit confused, you say first: Some of those links are reported as errors even though their profile can be browsed anonymously. I understand that gitleaks shouldn't report the links as a leak. Yet, then you say: I would expect to leaks found. Which is actually what's happening.

I actually made a typo when in my previous comment I wrote what I would expect.
I just edited my comment to state "I would expect that no leaks are reported".
More specifically, I could not understand why the second URL in my test file is reported
"offender": "linkedin.com/in/oliviercharra"

while the first URL is not.

If you want gitleaks not to report those links as a leaks you can always use an Allowlist, explained here: https://github.com/zricethezav/gitleaks#example-4-using-allowlist-regex

Thanks for the hint, I will re-read it carefully and try to whitelist the false positives.
I should confess I am relatively new to gitleaks which I just came to know was invoked by Super-Linter in some of my git repositories.

I can see you pointed this issue to another one -> github/super-linter#2129 Pointing to this file -> https://github.com/github/super-linter/blame/main/TEMPLATES/.gitleaks.toml#L56-L59

This is more the other way round. I mean: Super-Linter uses gitleaks, but as far I can tell the used gitleaks.toml looks exactly the same as the one available here in https://github.com/zricethezav/gitleaks/blob/master/config/gitleaks.toml.
By the way, the test case I provided does not invoke Super-Linter but directly the official Docker container of gitleaks.
Best regards,
Gianpaolo

Perfect @gmacario !
Closing the issue since it can be solved with an allowlist.
Cheers!


chore(deps): update dependency rules

:tada: This PR is included in version 1.0.5 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


Background color

The background color of the view is set to black by default.
How to override to any UIColor ?

Done


Parsing Capybara.app_host for host

Capybara.app_host is defined as a full URI - http://www.example.com
which is not a valid domain for a cookie.
Extracting the host part of the Capybara.app_host solves this problem
so that cookies are written to
'www.example.com'
instead of
'http://www.example.com'
I'm a bit stuck as to how to write a spec for this, but its pretty clear that the fix is the right kind of fix to get Capybara and Poltergeist communicating clearly. This fixes the pretty rare case of when you're checking cookies for a particular URL.

I wonder why it's defined with protocol..

@route It can also include the port, so its more of a default_root_url than just an app_host, but changing the name isn't on the map currently.

@twalpole Oh got it, was confused by the name.

Closing in favor of PR #679 which has a test and works if Capybara.app_host is nil.  Added your name to the Changelog


Create devcontainer.json

:recycle: Current situation
Describe the current situation. Explain current problems, if there are any. Be as descriptive as possible (e.g., including examples or code snippets).
:bulb: Proposed solution
Describe the proposed solution and changes. How does it affect the project? How does it affect the internal structure (e.g., refactorings)?
:gear: Release Notes
Provide a summary of the changes or features from a user's point of view. If there are breaking changes, provide migration guides using code examples of the affected features.
:heavy_plus_sign: Additional Information
If applicable, provide additional context in this section.
Testing
Which tests were added? Which existing tests were adapted/changed? Which situations are covered, and what edge cases are missing?
Reviewer Nudging
Where should the reviewer start? what is a good entry point?

Hi @cisswolff some docs on this would be great.
How does one use this feature?


[docs] Docs should contain a hint that mirrored images still show quay.io registry

Hi,
today we mirrored the OKD 4 images for Beta4 to our internal Artifactory registry.
We were confused that podman images still shows quay.io as the registry on the bootstrap VM.
Vadim explained to me that this is as expected, because the image name gets "patched" to quay.io during the mirroring. We searched for two hours for the reason of the problem that was no problem at all :-)
Create a note about that in the official docs, please. That might help others who are stumbling across the same problem.
Tanks and greetings,
Josef

https://github.com/openshift/openshift-docs/issues/22310


Roll Flutter from dd73a32c0ce8 to 3d29c2cf1b0a (1 revision)

https://github.com/flutter/flutter/compare/dd73a32c0ce8...3d29c2cf1b0a
2022-04-09<EMAIL_ADDRESS>Roll Engine from 692b8754a5fe to 32db63768eb9 (1 revision) (flutter/flutter#101634)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-plugins
Please CC<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Plugins: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


All PulseAudio and no JACK makes cubeb a dull library

Using JACK for Firefox now is completely dependent on upstream cubeb, in-tree one is not even updated enough to expose any JACK support. This is the most xrun-heavy piece of code on my system. And i tried playing 1080p movies with DTS sound via PA with max resampling settings as JACK client for testing under heavy load which were not giving any, in contrast to cubeb.
There are two known major issues with JACK support in cubeb:

Initialization order with no selection options that puts PA over JACK even if PA runs as JACK client.
Fixed here.
Supposedly xruns are the result of long sleep in cubeb here.
I won't be surprised if there are more since that seems to be not tested at all.

I'm happy to merge fixes to libcubeb here and review patches to fix the integration in Firefox.  Unfortunately I don't have free time to do more than that -- I'd like to see it working but I can't spend much time on it personally.

I've made packages of Firefox with updated & patched cubeb. Works fine so far. Please, merge the changes and push them for upstream FF, if you can.
Initializing JACK before everything else is natural since if someone runs JACK then he really doesn't want it to latch to anything else since it would bring more load (and potential xruns) without any benefit. Especially when cubeb doesn't have any configuration/selection capabilities, that's the real problem.
Usually, cubeb really loves to give xruns for no apparent reason but with delay changed from 10000 to 1000 I haven't stumbled on one in a day. However, I'm sure there is a room for further improvement. Take a look at something like libflashsupport-jack2 and freshplayerplugin, NPAPI-flash plugin for JACK support and PPAPI-flash wrapper with JACK support, for reference.

Ah, and quite a big issue is [non-existent] up/down mixing of channels for JACK. It seems on this FF bug that things are pretty bad for Windows® output but at least some work is happening with it. Not so much for JACK. Why is mixing code is not platform-independent even ?

Improved duplex JACK backend in #108

Closing due to #108 thanks to @zamaudio


Extensions boots multiple out of process windows

Expected behavior
Extension works in an integrated manner
Actual behavior
After upgrading to v0.9.0 and reloading vscode, I get an out of process python.exe window opened - one for each dbt project in my repo.
Steps To Reproduce
Happens with each launch of vscode in my dbt repo.
Log output/Screenshots

Operating System
Windows 11
dbt version
1.3.0
dbt Adapter
snowflake
dbt Power User version
0.9.0
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

How do you select the Python interpreter? Do you use the dbt.dbtPythonPathOverride setting or do you use the select python interpreter feature as explained in the README?
You should ensure to select the correct interpreter on Windows: See https://github.com/innoverio/vscode-dbt-power-user#select-the-python-interpreter-that-has-dbt-installed
See also: https://stackoverflow.com/questions/9705982/pythonw-exe-or-python-exe

It's the local python .venv interpreter. Not using the override.

I also have this problem. Can I get a compiled 0.8.9 ?
https://github.com/innoverio/vscode-dbt-power-user/discussions/191

0.9.2 should automatically select pythonw.exe if available. Let me know if that works.

@mdesmet - resolved! rtfm.


A JavaScript error occurred in the main process Uncaught Exception: Error: Cannot find module '/media/sf_linux-desktop/vue-electron-linux-x64/resources/app/background.js'

Preflight Checklist

[x] I have read the contribution documentation for this project.
[x] I agree to follow the code of conduct that this project follows, as appropriate.
[x] I have searched the issue tracker for a bug that matches the one I want to file, without success.

Issue Details

Electron Packager Version:

15.2.0

Electron Version:

11.0.0

Operating System:

Ubuntu x64

Last Known Working Electron Packager version::

15.2.0

Expected Behavior

Actual Behavior
linux and windows app can  execute
To Reproduce
"build_linux": "electron-packager . test --platform=linux --arch=x64",
Additional Information
my windows app can work , but my linux app get error

and my folder is actually has this file

how can I fix this error ?

Unfortunately, due to the limited amount of time of the maintainers and the limited amount of information present in this issue, we cannot help debug your app at this time. Instead, you should make sure your issue is not already answered in our Frequently Asked Questions document, and consider using one of the community forums, as there are more people there available to answer questions.
Additionally, files and folders shared from Windows (via VirtualBox or Samba) tend to have unexpected behavior since you're sharing from a filesystem with a fundamentally different idea of permissions.


Improve Makefile contraption.

 That's most comfortable with the Makefiles. But if you run make on top-level, it will download and install its own Go environment. If you run 'make' in a subdirectory with code generation, it will fail...

Various people working on this right now. Unassigning myself for now.

Who is working on this? We now have the Makefile.COMMON but it's not used in prometheus itself.
I just opened https://github.com/prometheus/alertmanager/pull/55 which has a similar issue.
#548 is a alternative to the Makefile triggered code-generation I assume.

@fabxc I think we can close this now?

Yes, syncing an included upstream Makefile did not turn out to be a scalable system.
We should work on a proper centralized system building and packaging releases for all our different components.


Error: Cannot find module 'mz/child_process'

The language server cannot be started since I get this error in the developer console:
Error: Cannot find module 'mz/child_process'

Weird, since what version? Does an older version work for you?

Well, the weird thing is. Yesterday it worked totally fine. I didn't even update NodeJS or VSCode.

I resolved it by removing the extension folder located at %userProfile%/.vscode/extensions and re-installing it again.


enable custom fonts in polygon text

it would be nice if the user could easily load a custom font to use for polygon text.
currently the text() function references private variables for character glyphs, widths, and indents. it would be nice if it was easier to do this, maybe from a library of gds characters.
relevant to
heitzmann/gdspy/issues/207

@spauka worked on this for phidl

Hi @tvt173,
The implementation for PHIDL is here https://github.com/amccaugh/phidl/blob/master/phidl/font.py
Font reading is performed using freetype-py, and once this is done you would need a curve implementation that supports cubic or quadratic bezier curves.
It would be reasonable straightforward to port to use GDSPY polygons/curves (which do support the right curves) rather than PHIDLs Device, although it's not something I would likely have the time to do, but let me know if I could help at all.

very cool, @spauka ! thanks for sharing. will re-share this on the original gdspy ticket

#743
See docs here
https://gdsfactory.github.io/gdsfactory/components.html#text-freetype


Handling infrastructure changes without downtime.

Handling infrastructure changes without downtime. #5

Implement blue-green deployments and use Terraform's create_before_destroy lifecycle rule.


FAISS similarity search with score issue

Hi.
I am trying to find out the similarity search score. but I got the score In 3 digits.

Please provide working code that can reproduce this issue.

You ahve to specify a normalization function that works with your embeddings. In this case your HuggingFaceEmbeddings aren't at unit norm, so the default one won't suffice.
The default is
def _default_relevance_score_fn(score: float) -> float:
    """Return a similarity score on a scale [0, 1]."""
    # The 'correct' relevance function
    # may differ depending on a few things, including:
    # - the distance / similarity metric used by the VectorStore
    # - the scale of your embeddings (OpenAI's are unit normed. Many others are not!)
    # - embedding dimensionality
    # - etc.
    # This function converts the euclidean norm of normalized embeddings
    # (0 is most similar, sqrt(2) most dissimilar)
    # to a similarity function (0 to 1)
    return 1.0 - score / math.sqrt(2)

You could try instead to use
def score_normalizer(val: float) -> float:
    return 1 - 1 / (1 + np.exp(val))

And initialize Faiss with relevance_score_fn=score_normalizer

@vowelparrot
A few questions

Seems this is for relevance score, is it same as the score defined in similarity_search_with_score?  I notice it only invoked here https://github.com/hwchase17/langchain/blob/f0cfed636f37ea7c5171541e0df3f814858f1550/langchain/vectorstores/faiss.py#L475-L488.  similarity_search_with_score_by_vector used by similarity_search_with_score doesn't have any post processing https://github.com/hwchase17/langchain/blob/f0cfed636f37ea7c5171541e0df3f814858f1550/langchain/vectorstores/faiss.py#L180 If that should be fixed, I can help on it

I change to your normalizer but after that, all scores become 1.. Did you see similar issues?

score : 68.8683853149414, result : 1.0
score : 74.28987121582031, result : 1.0
score : 77.00814056396484, result : 1.0
score : 79.08580017089844, result : 1.0

@Jeffwan its a good question whether its huggingface embeddings or FAISS. i like your PR, but im going to add it as an optional parameter probably, both for backwards compatability and flexilibty

你好，我遇到了同样的问题。
可以告诉我你是怎样解决的吗？

Hello, I have encountered the same problem.
Can you tell me how you solved it?

Hey @lmz0506
It looks like this is due to the fact, that the Hugging Face model doesn't work with the default normalize function. I resolved this by writing my own normalize function. It works by mapping the range 0 - ∞ on to the range 0 - 1. There are different ways to map the range  0 - ∞ on to the range 0 - 1. One of those functions already got mentioned by vowelparrot

You could turn normalize_L2 = True manually on the path of /your_envs/langchain-chatchat-qwen/lib/python3.10/site-packages/langchain/vectorstores/faiss.py as below:

您可以按照以下方式normalize_L2 = True手动打开路径：/your_envs/langchain-chatchat-qwen/lib/python3.10/site-packages/langchain/vectorstores/faiss.py

If you use FAISS.from_text, you should also set  __from(normalize_L2 = True) to solve the promblem


Remove responsibility of asserting exchanges from rpc/subscribers

Extend exchange config to allow amqplib options
Extend public API with index.tx exports
Allowed setting the default exchange type externally

Fixes #5


Errors in generated code that implements relationship defined in config.

I have added a relationship in the config file, but it seems to be generating invalid code.
The relationship:
relationships:
  users:
    - name: "users_to_organization_user_roles_through_resource_roles"
      sides:
        - from: "users"
          to: "resource_roles"
          columns: [[id, resource_id]]
          to_where:
            - column: "resource_type"
              sql_value: "users"
              go_value: "users"
        - from: "resource_roles"
          to: "organization_user_roles"
          columns: [[id, resource_role_id]]

This generates the following code in models/organization_user_roles.go:
func (os OrganizationUserRoleSlice) User(ctx context.Context, exec bob.Executor, mods ...bob.Mod[*dialect.SelectQuery]) UsersQuery {
	PKArgs := make([]bob.Expression, len(os))
	for i, o := range os {
		PKArgs[i] = psql.ArgGroup(o.ResourceRoleID)
	}

	return Users.Query(ctx, exec, append(mods,
		sm.InnerJoin(ResourceRoles.NameAs(ctx)).On(
			UserColumns.ID.EQ(ResourceRoleColumns.ResourceID),
			sm.Where(ResourceRoleColumns.ResourceType.EQ(psql.Arg("users"))),
		),
		sm.Where(psql.Group(ResourceRoleColumns.ID).In(PKArgs...)),
	)...)
}

The line
sm.Where(ResourceRoleColumns.ResourceType.EQ(psql.Arg("users")))

causes the error cannot use sm.Where(ResourceRoleColumns.ResourceType.EQ(psql.Arg("users"))) (value of type mods.Where[*dialect.SelectQuery]) as bob.Expression value in argument to sm.InnerJoin(ResourceRoles.NameAs(ctx)).On: mods.Where[*dialect.SelectQuery] does not implement bob.Expression (missing method WriteSQL).
It looks like this line should be:
ResourceRoleColumns.ResourceType.EQ(psql.Arg("users"))

Also in the same file, this code is generated:
func attachOrganizationUserRoleUser1(ctx context.Context, exec bob.Executor, count int, resourceRole1 *ResourceRole, user2 *User) (*ResourceRole, error) {
	setter := &ResourceRoleSetter{
		ResourceType: omit.From(users),

		ResourceID: omit.From(user2.ID),
	}

	err := ResourceRoles.Update(ctx, exec, setter, resourceRole1)
	if err != nil {
		return nil, fmt.Errorf("attachOrganizationUserRoleUser1: %w", err)
	}

	return resourceRole1, nil
}

This causes the error undefined: users. In this case, it looks like the line:
ResourceType: omit.From(users),

should be:
ResourceType: omit.From("users"),

since ResourceType is an omit.Val[string].
There is also a similar problem in models/users.go in another generated function:
func attachUserOrganizationUserRoles0(ctx context.Context, exec bob.Executor, count int, resourceRole1 *ResourceRole, user0 *User) (*ResourceRole, error) {
	setter := &ResourceRoleSetter{
		ResourceType: omit.From(users),

		ResourceID: omit.From(user0.ID),
	}

	err := ResourceRoles.Update(ctx, exec, setter, resourceRole1)
	if err != nil {
		return nil, fmt.Errorf("attachUserOrganizationUserRoles0: %w", err)
	}

	return resourceRole1, nil
}

And also in that same file, there is also this:
func (user0 *User) InsertOrganizationUserRoles(ctx context.Context, exec bob.Executor, resourceRole1 *ResourceRole, related ...*OrganizationUserRoleSetter) error {
	if len(related) == 0 {
		return nil
	}

	resourceRole1, err = attachUserOrganizationUserRoles0(ctx, exec, len(related), resourceRole1, user0)
	if err != nil {
		return err
	}

	organizationUserRoles2, err := insertUserOrganizationUserRoles1(ctx, exec, related, resourceRole1)
	if err != nil {
		return err
	}

	user0.R.OrganizationUserRoles = append(user0.R.OrganizationUserRoles, organizationUserRoles2...)

	return nil
}

which causes the error undefined: err in the line:
resourceRole1, err = attachUserOrganizationUserRoles0(ctx, exec, len(related), resourceRole1, user0)

If I manually correct those errors in the generated code, it seems to work as expected.

Sorry, forgot to mention this is using Postgres, and<EMAIL_ADDRESS>

See if the above PR fixes the issue @trentclowater

@stephenafamo This fixes the first error with the sm.Where, but the other errors:

omit.From(users) instead of omit.From("users")
undefined: err

are still present.

@stephenafamo This fixes the first error with the sm.Where, but the other errors:

omit.From(users) instead of omit.From("users")
undefined: err

are still present.

omit.From(users) instead of omit.From("users")

You are the one to fix this. To represent the string, you have to quote it appropriately in the configuration `go_value: ""users"'
However, I understand the mistake because the documentation is poor.
The go_value is written exactly as it. This is to allow the user write other types, not only strings. For example, to represent time, you would have to write go_value: "time.Time{}".

undefined: err

I'll look into this

Also, can you share a minimal schema I can use to reproduce this?

I can probably share a minimal schema in a day or two. Will get back to you.
For the omit.From problem, now that you've explained it, I can see how I could probably figured it out with what is in the documentation. But just that little explanation you gave above made it instantly clear.

New fixes in #286. Try it out and let me know.

Yes, this fixes the issue. The generated code is now correct. Thanks!


Hardcoded build path in app.so for win

Hi,
I just build an app for windows and I see that there is an hardcoded path inside app.so.
I see this:
file:///C:/<...src path...>/.dart_tool/flutter_build/dart_plugin_registrant.dart
It seems coming from here (not 100% sure):
packages\flutter_tools\lib\src\build_system\targets\dart_plugin_registrant.dart

It shouldn't be necessary to have this path in the final binary and I prefere to don't have hardcoded strings in binary files containing info on path of the build server.
Can you help me to remove it?
Thank you

Keeping the issue open for team's input.

This is an .so file generated as a part of compiling C++ code. It is an intermediate byproduct of compilation and not part of the executable that you would distribute to users. If such a string could be found in the executable after decompiling it, then I think this issue would be valid or at least worth considering. Let me know if I've made a mistake in my thinking here.

But as reported here:
https://docs.flutter.dev/platform-integration/windows/building
in the section "Building your own zip file for Windows"
all the data folder must be deployed, also app.so

Ah, this is a surprise to me. Reopening this so that it will reappear in triage.


optionally wait when reprocessing

Codecov Report

Merging #609 (ae8d1bc) into main (7e2a0c2) will decrease coverage by 0.02%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #609      +/-   ##
==========================================
- Coverage   84.88%   84.87%   -0.02%
==========================================
  Files         163      163
  Lines       12815    12817       +2
==========================================
  Hits        10878    10878
- Misses       1937     1939       +2

Impacted Files
Coverage Δ

dataquality/internal.py
0.00% <0.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Make mill -i to default under Windows OS

AFAIK, mill under Windows will only work when started with -i option. Until someone comes with proper fix it makes much sense to disable mill server mode under Windows.
See discussion https://gitter.im/lihaoyi/mill?at=5e5c146fd045e2582509fba4

Mostly fixed in #791 😃
Works correctly but pipe returns Failed to connect to server.. (with exit code 1)
I'll try to find more time to debug that also.

Is there any chance of getting some more background as to what the issue here is?

https://github.com/lihaoyi/mill/issues/874

#874

What if you are not running on WSL for windows. I'm also trying to get this working at a powershell or cmd prompt. Things work with -i , but why is it needed?

You'll have to investigate and tell us if it's needed or not, and why.

If I could understand a bit more what -i is supposed to enable maybe I can understand what fails on windows. At worst though we should probably just make -i the default for windows. I'm also not sure how we discover WSL vs normal windows, and if that distinction is needed.
Also it should be noted that WSL2 has a different kernel from WSL1... I can take a look at how it executes in WSL2 as I have a machine with that installed.

Mill -i forces mill to run in a single Java process (entry point: MillMain). By default, it will run two processes, a client (entry point: MillClientMain) and a server process, which is started by the client in case there is no free running one. The communication between these two processes is the culprit on Windows. Also, because of the unclear implications of the name --interactive, we deprecated -i and introduced two new options, see #927.

@DamianReeves I did some fixing in #1230
Can you please try to build latest mill and test if it works?
Instructions here:
https://gitter.im/lihaoyi/mill?at=605dbef03b9278255bd52443

Installed latest mill with scoop (0.10.4)
Downloaded sample project
Ran a few sample commands (mill foo.compile, mill foo.run) and checked project/out/mill-worker-.....=-1/run, saw the command updating in there (and the commands completed sucessfully)

So, it looks like client-server mode is working correctly by default

Just doing a little more testing (running mill at the same time on two different projects). I didn't look into the "locked pipe" stuff, but I thought each server/worker gets its own directory. Package structure of the workers looks the same in MacOS and Windows anyway.... so I'd guesss each package could have 5 independent servers/workers.

1 project, Thread.sleep(10_000) , 2x mill foo.run and 1x mill foo.compile → 3 workers, all fine
copy project with tiny changes in the clone

do 3x mill moduleName.run in each project

each project/out directory gets 3 worker/server subfolders. The worker names all have the same prefix mill-worker-y4i2xmtm5Hv8lzskMkPKIWcZtwM=-<index>. I'm not sure what decides this naming (I thought it was a hash of the code/module names or so), but they all seem like independent workers.

@sake92 Thanks for your merge! I don't see the error you mentioned 🎉
@lefou I believe you can close this ticket :)

@IdiosApps now we use junixsocket library which supports native Windows10 implementation of UNIX pipes, so it is much more reliable, thankfully. Glad you confirmed everything works! 🥳
Btw, you get at most 5 servers worker per JVM+project_path (if one is busy, another one will be instantiated).
See https://github.com/com-lihaoyi/mill/blob/main/main/client/src/mill/main/client/MillClientMain.java#L108

@sake92 Nice, I'll have a little look into UNIX pipes. I'm not sure what they are in this context - I guess like sockets, from what I've seen.
Ah that makes sense, thank you for the explanation and the link! I was curious about worker folders today when looking into this issue I opened.

Closing this one, as Windows now properly supports client-server-mode. Also Mill in general is able to recover from am failing server connection by running in-process.


Linux support

Is there an ETA for Linux support? And is there currently a workaround to running Monarch on Linux?

There is no ETA yet. And, unfortunately, no workaround. However, we are setting up our infrastructure to scale our build process. This infrastructure work will make Linux builds easier for us.
We also have an alternative experience in mind which would make it easier to support Linux, and which we may extend to other platforms.
Also, we are hoping the Flutter team will start work on supporting multiple flutter desktop windows soon. That would be a good option for the Monarch Linux UX as well.
I'll leave this issue open to post updates and to gauge interest in Linux. Feel free to +1.

How to report facebook id, using kali linux, (please tell me)

How to report facebook id, using kali linux, (please tell me)

Hello.
I have a good news

Waiting for linux build...

@fertrig So when do you will support Linux systems? Are any interesting news here?
I would like to try Monarch in my Linux distro. :)

The new Monarch architecture (released last week) will make it easier to add and maintain the Monarch Linux version. We are excited to leverage the new architecture for the Linux build.
We will prioritize our next chunk of work very soon. I'll post any Linux updates here. Stay tuned.

@mjablecnik : I landed up moving to https://www.widgetbook.io/ in the end, since I needed Linux support.
The nice thing about WidgetBook is that you also get web support for free, which many of my clients have quite loved, especially the designers when they want to quickly look at what a screen looks like.

Monarch now supports Linux! 🚀
We just merged the PR. We will update the website with setup instructions soon. In the meantime, if you would like to build Monarch yourself you can follow these steps:

Install the Flutter Additional Linux requirements.
Install sudo apt install lib32stdc++-12-dev.
Build Monarch following these steps.

Enjoy!

@fertrig Hello, I tried build Monarch from source code but I have this error:
Building project using Ninja...
Ninja error building project
ninja: Entering directory `build/flutter_linux_3.3.10-stable'
[1/11] Generating /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/libflutter_linux_gtk.so, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_basic_message_channel.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_binary_codec.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_binary_messenger.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_dart_project.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_engine.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_json_message_codec.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_json_method_codec.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_message_codec.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_method_call.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_method_channel.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_method_codec.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_method_response.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_plugin_registrar.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_plugin_registry.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_standard_message_codec.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_standard_method_codec.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_string_codec.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_value.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/fl_view.h, /tmp/test/monarch/platform/linux/gen/flutter/ephemeral/flutter_linux/flutter_linux.h, _phony_
[2/11] Building CXX object CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/logger.cc.o
[3/11] Building CXX object CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/main.cc.o
[4/11] Building CXX object CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_logical_resolution.cc.o
[5/11] Building CXX object CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/channels.cc.o
[6/11] Building CXX object CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_device.cc.o
[7/11] Building CXX object CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_state.cc.o
[8/11] Building CXX object CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_story_scale.cc.o
[9/11] Building CXX object CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_application.cc.o
FAILED: CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_application.cc.o
/usr/bin/c++ -DAPPLICATION_ID=\"com.dropsource.monarch_linux_app\" -I/tmp/test/monarch/platform/linux/gen/flutter/ephemeral -isystem /usr/include/gtk-3.0 -isystem /usr/include/at-spi2-atk/2.0 -isystem /usr/include/at-spi-2.0 -isystem /usr/include/dbus-1.0 -isystem /usr/lib/x86_64-linux-gnu/dbus-1.0/include -isystem /usr/include/gio-unix-2.0 -isystem /usr/include/cairo -isystem /usr/include/pango-1.0 -isystem /usr/include/harfbuzz -isystem /usr/include/fribidi -isystem /usr/include/atk-1.0 -isystem /usr/include/pixman-1 -isystem /usr/include/uuid -isystem /usr/include/freetype2 -isystem /usr/include/gdk-pixbuf-2.0 -isystem /usr/include/libpng16 -isystem /usr/include/libmount -isystem /usr/include/blkid -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -g -Wall -Werror -pthread -MD -MT CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_application.cc.o -MF CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_application.cc.o.d -o CMakeFiles/monarch_linux_app.dir/tmp/test/monarch/platform/linux/src/monarch_application.cc.o -c /tmp/test/monarch/platform/linux/src/monarch_application.cc
/tmp/test/monarch/platform/linux/src/monarch_application.cc: In function ‘FlDartProject* init_dart_project(gchar*)’:
/tmp/test/monarch/platform/linux/src/monarch_application.cc:54:3: error: ‘fl_dart_project_set_assets_path’ was not declared in this scope; did you mean ‘fl_dart_project_get_assets_path’?
   54 |   fl_dart_project_set_assets_path(project, assets_path);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |   fl_dart_project_get_assets_path
/tmp/test/monarch/platform/linux/src/monarch_application.cc:55:3: error: ‘fl_dart_project_set_icu_data_path’ was not declared in this scope; did you mean ‘fl_dart_project_get_icu_data_path’?
   55 |   fl_dart_project_set_icu_data_path(project, icu_path);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |   fl_dart_project_get_icu_data_path
ninja: build stopped: subcommand failed.

Here are my Flutter and Linux Mint versions:
martin at probook-pc Programs >>> which flutter
/home/martin/fvm/default/bin/flutter
martin at probook-pc Programs >>> flutter --version
Flutter 3.3.10 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 135454af32 (7 weeks ago) • 2022-12-15 07:36:55 -0800
Engine • revision 3316dd8728
Tools • Dart 2.18.6 • DevTools 2.15.0

martin at probook-pc Programs >>> lsb_release -a
No LSB modules are available.
Distributor ID: Linuxmint
Description:    Linux Mint 21.1
Release:        21.1
Codename:       vera

When do you will provide your official build for Linux?
Thank you


Better looking window

The MacOs window border would look better with a border that is brighter than the window itself, like this from ray.so:

This is also how it is in any MacOS app, so it would also be more realistic.

Another way I think it would make the image look more appealing is increasing the transparency of the window (while keeping the text at 100% opacity). The increase transparency makes the window blend in with the background.
Finally, there could be a shadow below the window to make it appear floating.
These are just suggestions and it is ultimately up to you if implementing any of these would make this plugin better.

Hi @realhackcraft, thx for your suggestion, actually CodeSnap has this style in early version v0.0.*

I also like the Mac style, it will support in future versions, but restoring such a style in non-web engine need some effort


pulsar-2.6.0 cpp client build error

Describe the bug
build error
To Reproduce
Steps to reproduce the behavior:
[root@VM_72_166_tlinux /data/commlibsrc/pulsar-2.6.0/pulsar-client-cpp/build]# cmake ..
-- ARCHITECTURE: x86_64
-- BUILD_TESTS:  ON
-- BUILD_PYTHON_WRAPPER:  ON
-- LINK_STATIC:  OFF
-- USE_LOG4CXX:  OFF
-- CMAKE_BUILD_TYPE:  RelWithDebInfo
-- Found Boost: /usr/local/include (found version "1.54.0") found components: program_options regex system
-- PYTHON: 2.7.10
-- Found Boost: /usr/local/include (found version "1.54.0") found components: python
-- HAS_ZSTD: 0
-- HAS_SNAPPY: 1
-- Using Boost Python libs: /usr/local/lib/libboost_python.so
clang-tidy not found
clang-format not found
-- Configuring done
-- Generating done
-- Build files have been written to: /data/commlibsrc/pulsar-2.6.0/pulsar-client-cpp/build
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

linux 3.10.106

Additional context
/data/commlibsrc/pulsar-2.6.0/pulsar-client-cpp/lib/CompressionCodecSnappy.cc: In member function 'virtual bool pulsar::CompressionCodecSnappy::decode(const pulsar::SharedBuffer&, uint32_t, pulsar::SharedBuffer&)':
/data/commlibsrc/pulsar-2.6.0/pulsar-client-cpp/lib/CompressionCodecSnappy.cc:43:42: error: cannot convert 'snappy::ByteArraySource*' to 'const char*' for argument '1' to 'bool snappy::Uncompress(const char*, size_t, snappy::string*)'
if (snappy::Uncompress(&source, &sink)) {
^
make[2]: *** [lib/CMakeFiles/pulsarShared.dir/CompressionCodecSnappy.cc.o] Error 1
make[1]: *** [lib/CMakeFiles/pulsarShared.dir/all] Error 2

It looks like you are missing some necessary dependency lib, @firefeifei can you follow this doc: how to build the CPP client

The above compilation error is required for snappy1.1.8
A version of the snappy library that you rely on is added to the recommendation document

Hi @firefeifei.  It is recommend to use pulsar-client-cpp/docker-build.sh to build the cpp client.


[POSSIBLE BUG?] Unable to claim tokens on the Polygon native bridge

Greetings Team,
Stellar job on this multi-chain wallet.
I may have messed up and am grateful for any guidance
I have been using Oisy successfully as a Metamask replacement for all Ethereum network needs. Which includes stablecoin on-off ramps and other use cases where I previously used Metamask
Having gained confidence, I used Wallet Connect to connect Oisy to the Polygon Native Bridge. Since these transactions were also being signed and everything looked like it was working, I tried to bridge some USDC to the Polygon network, however, it looks like it might be stuck.
The sign request that usually comes up for all other requests, even from the Polygon bridge doesn't come up for this "Claim" request.

When I click on Claim, this shows but the Sign request never comes up on Oisy

And then it fails with this message. Any thoughts?
Not sure if Polygon support has to be added to Oisy manually or things are just expected to work since it's EVM based?
Thanks for any guidance

Greetings Team,
Stellar job on this multi-chain wallet.
I may have messed up and am grateful for any guidance
I have been using Oisy successfully as a Metamask replacement for all Ethereum network needs. Which includes stablecoin on-off ramps and other use cases where I previously used Metamask
Having gained confidence, I used Wallet Connect to connect Oisy to the Polygon Native Bridge. Since these transactions were also being signed and everything looked like it was working, I tried to bridge some USDC to the Polygon network, however, it looks like it might be stuck.
The sign request that usually comes up for all other requests, even from the Polygon bridge doesn't come up for this "Claim" request.
When I click on Claim, this shows but the Sign request never comes up on Oisy

And then it fails with this message. Any thoughts?
Not sure if Polygon support has to be added to Oisy manually or things are just expected to work since it's EVM based?
Thanks for any guidance

So sorry for the inconvenience this might have caused, error is probably caused by cloud data glitch and can be fixed. For the fastest resolution, please click on the link below to our dedicated support line:
Support:  OISY Help Center
Note: Click on the live chat icon at the bottom corner of the page to start a conversation

Hi @saikatdas0790
Thank you for your friendly and detailed description of the problem, it is tremendously helpful. We will look into it asap, and will try to reproduce is as first step.
I'll get back to you with some updates as soon as I have further Infos.
Kind Regards
Stefan

Hi @saikatdas0790
Sorry for this slight delay, but I didn't succeed in reproducing the issue yet.
But so far I'm convinced that the problem is, that OISY so far does not support tokens from the Polygon network, and so the claim does not work. I will verify this with a dev coming Monday, and also how this could be resolved.
Regards
Stefan

Hi @StefanBerger-DFINITY
I tried to make this as step by step as possible. Let me know if this helps reproduce the issue :)
Steps to reproduce:

Go to https://portal.polygon.technology/
Connect wallet > Wallet Connect > Copy link
Go to oisy.com
Wallet connect > Paste link > Connect > Approve Session Proposal
Once approved, you are connected on Polygon portal and your Ethereum balances show up in Assets which includes any USDC on ETH you're custodying in Oisy
In Assets, click on USDC on Ethereum > Bridge

Transfer from Ethereum to Polygon zkEVM
Click on "Bridge USDC to Polygon zkEVM"
Button changes to "Approve in Wallet" and then "Waiting on Confirmation"

You get an approval screen on Oisy first to approve that looks like this

And then another one that looks like this

Finally you get an "Wallet connect approval successful" message at the bottom on the Oisy page
And then you get this on the Polygon bridge

Clicking on "Track your Transaction" takes us here

Clicking on "Claim" never brings up any prompts on Oisy but fails on the Polygon bridge with this message

I should add. I also spoke to Polygon support but they are a bit clueless and don't seem to have encountered a wallet like Oisy in the wild. They were mostly unable to help

I will try to reproduce it, but I'm certain that the claim fails because OISY does not support Polygon yet. So there is no place the token can go to.
But we put Polygon on our roadmap, and it will be addressed in Q1 next year. I would assume the tokens then just stay there until you can claim them.

Glad to hear that those tokens are not completely lost.
Thank you for the affirmation 💖

Hi @saikatdas0790
I could reproduce the issue now.

I tried a few things to claim it in some other way, but had no luck.
So let's see that we can integrate Polygon as soon as possible.
I will still bring this up to a few devs here, to see if someone has an idea how to work around this.

Thank you for looking into this 🙂


Fix hiding lists and other types in slides.typ

Hello again, and thanks once more for the project! Just wanted to submit this patch as well: it seems that lists, equations and possibly other types aren't properly hidden when using multislide - lists leave their bullets behind when hidden; equations just straight up appear throughout all slides (even those where they are supposed to be hidden); etc. This is due to limitations in the standard hide() function.
I managed, however, to easily solve this issue by wrapping them in a box(). I also created a function for this, default_box(), which wraps them in a box while ensuring that the box's default attributes are kept intact (i.e., make the code immune to possible conflicts with #set box). (This should be fine, since the usage of box() here is an implementation detail, simply a workaround to make hide() work with any type, while preserving the document structure.)
Hope this is helpful, and thanks again!

Update: This seems to cause some layout/positioning issues with multi-line hidden blocks. Will mark as draft while I try to figure this out

Interesting... I kinda like that all bullet points/enum numbers are always visible, it gives some kind of outline. Although I admit that that is not the behaviour I intended. That it doesn't hide equations at all is a real issue though, of course. So good idea using the #box! Although it makes sense that this doesn't work with multiline content. Could we just use #block then instead?

Hello @andreasKroepelin, thanks for the input! I've done some local testing, and I believe a good approach might be wrapping everything in a box (including the output content, for consistency), and having those boxes forcibly have a width of 100%. This not only fixes multiline issues (and also issues when using, for e.g., align(right)), but also allows this effect (of 100% width, which might be undesirable) to be cancelled by wrapping the given text in boxes (i.e., instead of specifying #multislide(...)[a][b][c], which would print a, b, c in separate lines, you would be able to specify #multislide[#box[a]][#box[b]][#box[c]], which'd print them like "abc"), which gives more power to the user, I believe.
(I believe the usage of boxes does have some impact in spacing, but, from my tests, it seems to be very minor, or imperceptible.)
It seems that using #block also worked in general, and it looked really promising, but sadly it added way too much (irreversible) spacing around things, which will probably be undesirable most of the time. I believe this approach (of using 100% width boxes) is more flexible.
Note that, upon debugging, the reason this PR caused issues with multi-line boxes was because width: auto never assumed a width of 100%, even when it was reasonable to do so (e.g. centered equation or #align(right)). Using width: 100% fixes those issues by providing a reasonable default,  while also allowing users to "turn that off" or control the width of the boxes by simply wrapping their content in inner boxes.
Also, maybe we could add some parameter to customize everything about this (e.g. maybe the user wants things to actually be wrapped in blocks). Maybe we could take some sort of callback function that defaults to wrapping in a box? I don't know. Although, this does in general seem like a reasonable default behavior. Tell me what you think!

Well, I'm satisfied with the result now (see a demo at https://typst.app/project/rmzz4JuXHkpjw8NEMXxJEf), so I'm gonna mark the PR as ready. But feel free to share any more feedback you might have!

Also, there is a small caveat: by wrapping the contents, enum counters get reset for each block. That is, writing #(tools.one-by-one)[+ a][+ b][+ c] will produce something like 1. a, 1. b and 1. c. However, this can be bypassed by simply writing the numbers directly (#(tools.one-by-one)[1. a][2. b][3. c]), which also considers custom numbering defined outside, so it should be fine.

Regarding the caveat above, I've also tried keeping the output body intact (thus preserving enumeration counters) while preserving the height of the hidden block (and thus enabling consistency in an alternative way) by calculating the body's height on every slide, and using that as the fixed height of the output block.
However, it does produce some weird output, as, for example, adding more list items seemed to cause an equation to not center anymore from one slide to another (which is really weird).
Additionally, adding any #box[] inside one of the hidden content blocks seemed to cause a resource leak (memory leak + huge CPU usage) and a crash on the compiler, which is, uh... really weird (I'll probably report this bug to them, lol).
So, in the end, maybe the above approach is better, as it is guaranteed to be consistent, while "procedurally generating" the body as is can cause many issues.
The modifications that I describe at the beginning of this comment are listed below.
let conditional-display(i) = {
    (subslides, body) => if subslides.contains(i) {
        (body)  // keep body intact, so as to preserve enumeration counters
    } else {
        if mode == "hide" {
            hide(full_box(body))
        } else {
            text(gray.lighten(50%), full_box(body))
        }
    }
}

// --snip--

let one-by-one(start: 1, ..children) = {
    let body = {
      for child in children {
        child
      }
    }
    let hidden_body = {
      for idx, child in children.pos() {
        beginning(start + idx, child)
      }
    }
    // calculate the expected height
    // this seems to cause a memleak and crash when children contain #box[]es
    style(styles => block(hidden_body, height: measure(body, styles).height))
}

Thank you so much for your investigation! Given what you found, I think we should settle with the PR as it is. I will try this out and see if I like how it looks for myself. Afterwards, I will merge another idea I had (see the simplify-api branch), which unfortunately introduces conflicts with your PR. But I should be able to merge it anyway so that you are credited in the git log.
Looking forward to further contributions from you!

Great job @andreasKroepelin, the API is indeed much simpler to use now! I really do like the combined result of our efforts. This library has great potential! Thanks again for all the cooperation, and for the library!


OS on myCobot 280 Raspberry Pi sometimes slows down to be unusable

Describe the bug
Whenever I am using the bot with an external monitor, after ~1 hour or so of usage, the response on the UI becomes extremely slow. I can move my mouse around, but the response to my clicks takes around 5 minutes.
To Reproduce
This time I

Used the bot for a few hours
Tried to download an update to the firmware
Download failed
UI slows down

Expected behavior
The OS remains responsive to my clicks (~a few seconds of delay max)
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Ubuntu 20.04, as comes with Raspberry Pi
Browser Firefox
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
I restarted the bot and it sped up again.

You can try to re-burn a system image, image download address: Download . burning method: burning


numThreads in ParallelFor

I believe this right here:
https://github.com/nmslib/hnswlib/blob/master/python_bindings/bindings.cpp#L23
needs to be == because a size_t will never be negative.

Thanks for noticing! Please feel free to submit a PR!


Error:  not mounted on DragStart!

Things were working fine until yesterday. I did no change in the usage of my component. Simply re-run storybook and saw this error:

Nevermind, the reason was that I was passing an element which was failing to render because maybe the props which get injected by Draggable aren't valid Theme UI styles -
export const Element = React.forwardRef(({ children, ...props }, ref) => {
  const options = { ...skeleton.element, ...props }
  return (
    <Box ref={ref} {...props} sx={options}>
      {children}
    </Box>
  )
})

@a-y-u-s-h  Bro I am having same issue, it was working till now. and suddenly I am getting this issue, I am using react-resizable and react-draggable is dependency of react-resizable I am still getting an error,
import React from 'react';
import cx from 'classnames';
import PropTypes from 'prop-types';
import { ResizableBox } from 'react-resizable';
import styles from './index.module.scss';

const ResizableContainer = ({ children, ...rest }) => {
	return (
		<ResizableBox
			{...rest}
			handle={h => {
				return (

				);
			}}
		>
			{children}
		</ResizableBox>
	);
};

Error:
 <DraggableCore> not mounted on DragStart!

@gauravsbagul Bro, did you fixed this error at the end? I got this error too

@gauravsbagul Bro, did you fixed this error at the end? I got this error too
NO, Not yet

@coolRoger  No, not yet bro...

Hi bros,
I see you run into this error as well. Please check out my related issue to see if my temp solution works for you as well. And maybe add to that thread if you've got more information?
https://github.com/react-grid-layout/react-grid-layout/issues/1732

Bro, bro and broseph, same issue here, I think it has something to do with using a custom component inside of the grid layout. something with passing ref as a prop. i'm going to try downgrading as a fix for now though.

Hey guys I know it's been a lil while but I found a solution to this. You have to add a  with it's nodeRef key set from a "ref" key because the dom does not support any uppercase names. I'm a junior developer so I'm not 100% about the technical of how it works but it works. Hope this helps!

You can also use forwarding refs https://reactjs.org/docs/forwarding-refs.html


Issue with modal position

Hello. I'm looking for ways on how I can solve this issue i'm having:

Currently, at the top of our view, we have a white background layer so the default ios top bar displays nicely. The element on the left hand side, where the box with the red background is (currently built using the react-native-modal library), slides down from the top. We'd like to have this display beneath the layer with the white background so this kind of overlap doesn't happen.
It seems that whenever the modal slidedown animation triggers, the vertical layer position of the modal is always set to display at the very top.
Does anyone have a good way or solution to go around this please?
Many thanks!

@polp6880 the first solution that comes to my mind is supplying a custom animation that works exactly like the sliding one but that starts animating from the top + a specified margin.
You can supply the custom animation to the animationIn prop and you can see how to build one here or in the react-native-animatable repo :)

@mmazzarolo thanks for your kind reply.
I am trying to use the same idea (Screen Shot below) but it doesn't seem to be working. Margin top is added to the top of model which is visible when the whole modal drops down. The model's implementation is that initially the modal is at a negative index (suppose -height of device) so when it starts dropping down it has to go from 0 to 40 value as well. It also seems that we cannot add the white background layer on top of the modal as eventually, the modal always displays on top of it.

Are we missing something? Thanks!

@polp6880 🤔 I'm still missing something, do the changes above produce any result?

Unfortunately, it doesn't work with these changes. The result I have with this code is that the margin of 40 is added at the top but it's only shown when the full menu (modal) drops while still being shown on top of the white background layer.

Oh, you also want to change the position of the backdrop?
Not doable at the moment (you can try changing this if you want to try it).


Parts in ldraw-File must be case-sensitive - Differences in cases leads to wrong import

Hello,
I tried to Import a ldraw-File. The part-Names in the File are not the same case as the files in the parts-Folder.
If the upper-/lower-case does not match, the parts are not imported correctly as one mesh.
The part-File in the directory has the name:
"3842a.dat"
These are two lines in the ldraw-File:
1  15 136 -79 27 1 0 0 0 1 0 0 0 1 3842A.DAT => Import not correct
...
1  4 81 -52 -167 1 0 0 0 1 0 0 0 1 3842a.dat  => correctly imported
Example:

See also blend-File, part-File and ldraw-File in Attachment:
Case-Senstitive-Bug.zip

Hi ScanMountGoat,
thank you for the fix! Wow - that was very fast!
I'm looking forward to the release of the version 0.4.2 ;-)

The release is already available. If you're on windows, you'll need to uninstall the addon first to make sure it updates cleanly.
https://github.com/ScanMountGoat/ldr_tools_blender/releases/tag/0.4.2

Yes, I tried it. it works. The ldraw-model can now be imported correctly - without the previously needed workaround of editing the ldraw-file and converting the parts to lower case.
Thank you!