Compiling model including hub.KerasLayer fails in distribution strategy scope.

When trying to compile a model that includes a hub.KerasLayer I get the following error:
ValueError: Variable (<tf.Variable 'bert/embeddings/word_embeddings:0' shape=(119547, 768) dtype=float32>) was not created in the distribution strategy scope of (<tensorflow.python.distribute.tpu_strategy.TPUStrategy object at 0x7fb17c83f3c8>). It is most likely due to not all layers or the model or optimizer being created outside the distribution strategy scope. Try to make sure your code looks similar to the following.
with strategy.scope():
  model=_create_model()
  model.compile(...)

Code used to build and compile model:
with tpu_strategy.scope():
    in_id = tf.keras.layers.Input(shape=(MAX_SEQ_LENGTH,), name="input_ids", dtype=np.int32)
    in_mask = tf.keras.layers.Input(shape=(MAX_SEQ_LENGTH,), name="input_masks", dtype=np.int32)
    in_segment = tf.keras.layers.Input(shape=(MAX_SEQ_LENGTH,), name="segment_ids", dtype=np.int32)
    bert_inputs = {"input_ids": in_id, "input_mask": in_mask, "segment_ids": in_segment}

    bert_output = hub.KerasLayer(BERT_MODEL_HUB, trainable=True, signature="tokens", output_key="pooled_output")(bert_inputs)

    dense = tf.keras.layers.Dense(256, input_shape=(768,), activation='relu')(bert_output)
    pred = tf.keras.layers.Dense(len(unique_labels), activation='sigmoid')(dense)

    model = tf.keras.models.Model(inputs=bert_inputs, outputs=pred)

    model.compile(
        optimizer=tf.keras.optimizers.Adam(learning_rate=LEARNING_RATE),
        loss=tf.keras.losses.binary_crossentropy,
        metrics=["accuracy"]
    )

This is the model I'm loading in the KerasLayer: https://tfhub.dev/google/bert_multi_cased_L-12_H-768_A-12/1
The entire model is created within the scope of the distribution strategy, so I'm not sure what the cause of the error would be, other than a bug.

Yes, this may well be due to a bug in TensorFlow 2.0. Sorry about that. Please try a recent tf-nightly PIP package and reopen if the issue persists.

Also, you're using a TF1-style hub.Module. Try https://tfhub.dev/tensorflow/bert_en_cased_L-12_H-768_A-12/1 instead (see https://www.tensorflow.org/hub/tf1_hub_module vs https://www.tensorflow.org/hub/tf2_saved_model).

Strangely enough, installing tf-nightly gave me TensorFlow version 1.12. I was able to install TensorFlow v2.1.0-rc2-17-ge5bf8de by running pip install tensorflow-gpu==2.1, but then I get these errors when trying to create the TPU distribution strategy:
UnimplementedError                        Traceback (most recent call last)
<ipython-input-25-6596c0485615> in <module>
      7
      8     resolver = tf.distribute.cluster_resolver.TPUClusterResolver(TPU_ADDRESS)
----> 9     tf.config.experimental_connect_to_host(resolver.master())
     10     tf.tpu.experimental.initialize_tpu_system(resolver)
     11     tpu_strategy = tf.distribute.experimental.TPUStrategy(resolver)

~/.local/lib/python3.5/site-packages/tensorflow_core/python/eager/remote.py in connect_to_remote_host(remote_host, job_name)
     75       {job_name: [_strip_prefix(host, _GRPC_PREFIX) for host in remote_hosts]})
     76
---> 77   connect_to_cluster(cluster_spec)
     78
     79

~/.local/lib/python3.5/site-packages/tensorflow_core/python/eager/remote.py in connect_to_cluster(cluster_spec_or_resolver, job_name, task_index, protocol, make_master_device_default)
    140     context.set_server_def(server_def)
    141   else:
--> 142     context.update_server_def(server_def)
    143
    144   if make_master_device_default and isinstance(

~/.local/lib/python3.5/site-packages/tensorflow_core/python/eager/context.py in update_server_def(server_def)
   2017
   2018 def update_server_def(server_def):
-> 2019   context().update_server_def(server_def)
   2020
   2021

~/.local/lib/python3.5/site-packages/tensorflow_core/python/eager/context.py in update_server_def(self, server_def, keep_alive_secs)
    592       pywrap_tensorflow.TFE_ContextUpdateServerDef(self._context_handle,
    593                                                    keep_alive_secs,
--> 594                                                    server_def_str)
    595       self._initialize_logical_devices()
    596

UnimplementedError:
Additional GRPC error information:
{"created":"@1578592686.165655670","description":"Error received from peer","file":"external/grpc/src/core/lib/surface/call.cc","file_line":1039,"grpc_message":"","grpc_status":12}

This is the code for creating the strategy:
TPU_ADDRESS = "grpc://" + "<IP_ADDRESS>:8470"

with tf.compat.v1.Session(TPU_ADDRESS) as session:
    print('TPU devices:')
    pprint.pprint(session.list_devices())

resolver = tf.distribute.cluster_resolver.TPUClusterResolver(TPU_ADDRESS)
tf.config.experimental_connect_to_host(resolver.master())
tf.tpu.experimental.initialize_tpu_system(resolver)
tpu_strategy = tf.distribute.experimental.TPUStrategy(resolver)

So it looks like there might be a bug in TensorFlow 2.1? Or should I be creating the TPU distribution strategy differently?

Please help me investigate further. Do I guess correctly that your code runs on a Colab notebook hosted on colab.research.google.com? Please share a Colab notebook with me to reproduce the issue with TF 2.1 (possibly prerelease) and BERT from a TF2-style SavedModel, such as https://tfhub.dev/tensorflow/bert_en_cased_L-12_H-768_A-12/1.

@arnoegw I'm using Colab and a Google Cloud TPU Instance. The error I previously posted appears to be caused by tf.config.experimental_connect_to_host(resolver.master()) already having been ran. I still encounter another error however while trying to create the TPU Strategy, but it appears to be an issue with TensorFlow, not TensorFlow Hub.

Using TF-2.3, still got the same problem. I use distribute.MirroredStrategy().scope(). Any idea?

Using TF-2.3, still got the same problem. I use distribute.MirroredStrategy().scope(). Any idea?

I also have the same issue. Please help!

FWIW, https://www.tensorflow.org/tutorials/text/solve_glue_tasks_using_bert_on_tpu shows how it works for TF2.4. (It also worked for TF2.3, but in Colab you can't change the TPU worker version to try.)

I get the same error when I want to compile my model.
My model is a Deeplab.
Could you please help me with that?
Tensorflow 2.2.0
def create_model(args, model_name, output_dim):
    if model_name == "deeplabV3_plus":
        input_shape = ((args.input_size, args.input_size, 3))
        model = DeepLabV3Plus(input_shape)

Then later, in one section of my code, I want to train my model, but I get the following error.
         with strategy.scope():
            METRICS = [
                keras.metrics.TruePositives(name='tp'),
                keras.metrics.FalsePositives(name='fp'),
                keras.metrics.TrueNegatives(name='tn'),
                keras.metrics.FalseNegatives(name='fn')
            ]
            model = self.model
            model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=METRICS)

        history = model.fit(train_data,
                            verbose=2,
                            epochs=args.epochs,
                            shuffle=False)

 raise ValueError(
ValueError: Variable (<tf.Variable 'conv1_conv/kernel:0' shape=(7, 7, 3, 64) dtype=float32, numpy=
array([[[[-0.02878591,  0.03968531,  0.03641525, ..., -0.03084487,
          -0.02158457, -0.014132  ],
         [ 0.00074502, -0.0025292 , -0.04114929, ..., -0.0122269 ,
           0.02191756,  0.00397792],
         [ 0.01599922,  0.02605366,  0.02913692, ...,  0.01153038,
           0.01981341, -0.01823156]],

        [[ 0.0302609 , -0.03902362,  0.01403341, ...,  0.02662553,
           0.01796425, -0.02402215],
         [-0.00916608,  0.02252114,  0.03229741, ..., -0.02720008,
           0.04137112,  0.01507518],
         [-0.00389622,  0.01516257,  0.03228285, ...,  0.04099331,
           0.00592189,  0.0221811 ]],

        [[ 0.03253492, -0.01435366,  0.02506782, ..., -0.03665198,
          -0.00949166, -0.02421995],
         [-0.01569086, -0.04148282, -0.01675939, ..., -0.0239016 ,
          -0.02261319,  0.03264611],
         [-0.03728421,  0.01131782, -0.0091066 , ..., -0.00546361,
          -0.01797785, -0.03041088]],

        ...,

        [[ 0.01792137,  0.03674115,  0.02934234, ...,  0.03210436,
          -0.01456548, -0.03170301],
         [-0.00490872,  0.02028899,  0.03461434, ...,  0.01494849,
           0.03875599,  0.02271222],
         [-0.01961406, -0.03714995, -0.01296565, ..., -0.00358429,
           0.04021056, -0.01833691]],

        [[-0.01868246,  0.03103003, -0.01014595, ...,  0.0008074 ,
           0.02688321, -0.01458171],
         [ 0.01615499,  0.01003345, -0.03475311, ...,  0.03200821,
          -0.03148265, -0.03816338],
         [-0.02500446, -0.04130269,  0.03440503, ...,  0.00656155,
          -0.00379527,  0.01493592]],

        [[ 0.03022939,  0.0246064 ,  0.02682921, ..., -0.03900644,
          -0.00721056, -0.02661576],
         [-0.02527636,  0.02407213, -0.00924577, ..., -0.00710348,
          -0.01992491,  0.00035052],
         [ 0.02833832,  0.01586089, -0.04064951, ...,  0.02057872,
           0.00046169,  0.01369259]]],

       [[[-0.01213805,  0.03111628,  0.02673499, ...,  0.00549041,
           0.02938559, -0.01614523],
         [-0.00885064, -0.00359355, -0.00358286, ..., -0.04022623,
           0.02911928,  0.02872286],
         [ 0.03571633, -0.0010606 , -0.0220888 , ..., -0.0238349 ,
          -0.03300236, -0.03627774]],

        [[ 0.03680736,  0.00487308, -0.01098377, ..., -0.005409  ,
          -0.01626872, -0.03017656],
         [ 0.03737028,  0.01651787, -0.01228035, ..., -0.02987617,
           0.01691703, -0.03619874],
         [-0.00311079,  0.02328985,  0.03340743, ...,  0.0070854 ,
          -0.008461  ,  0.01112581]],

        [[-0.01047574, -0.04075018, -0.01167224, ..., -0.03680304,
          -0.02393692, -0.02736144],
         [ 0.00220935, -0.00096874,  0.01025746, ..., -0.01753948,
          -0.00598299, -0.04184602],
         [-0.03863832, -0.03028022,  0.02922371, ..., -0.00947168,
           0.03729448, -0.0011937 ]],

        ...,

        [[-0.01837078, -0.04027799, -0.00785745, ..., -0.00608211,
           0.02307337, -0.01391036],
         [-0.01727627, -0.03093182,  0.03744406, ..., -0.00287396,
          -0.02517046,  0.02554483],
         [ 0.01364107,  0.02183375, -0.00415699, ...,  0.01771579,
          -0.03176679,  0.0247004 ]],

        [[ 0.00443545,  0.02236081,  0.011328  , ..., -0.01656485,
           0.01892501, -0.01097592],
         [ 0.01361132, -0.01312676,  0.00155234, ...,  0.03267791,
           0.0075016 ,  0.00613252],
         [ 0.000273  , -0.02797817,  0.03450226, ..., -0.02308745,
          -0.00685496,  0.03451258]],

        [[ 0.02024524, -0.0317959 ,  0.00446649, ..., -0.0217262 ,
          -0.03863733,  0.0247216 ],
         [-0.03236089,  0.02701623, -0.01374604, ...,  0.02401371,
          -0.00909961,  0.0173627 ],
         [ 0.00756612,  0.01979894, -0.01443185, ..., -0.03781085,
           0.02710035, -0.03068993]]],

       [[[ 0.02833999, -0.00623503, -0.02557684, ..., -0.02902167,
           0.03662451, -0.0300681 ],
         [ 0.00201247,  0.00327801,  0.00134143, ...,  0.03343113,
           0.00843311, -0.02711113],
         [-0.01984301, -0.00551971,  0.02528293, ...,  0.03623073,
          -0.01419386, -0.0007051 ]],

        [[ 0.02833929, -0.01252304, -0.02989598, ...,  0.04137161,
          -0.00839223,  0.01394486],
         [-0.02659342,  0.00217635, -0.03752425, ...,  0.00908671,
           0.03528336, -0.03762647],
         [-0.02389926,  0.01050568, -0.02797867, ...,  0.00836991,
           0.01234463,  0.04017064]],

        [[ 0.00203969, -0.02852772,  0.00551932, ...,  0.00465782,
          -0.03351232, -0.03177572],
         [ 0.02206092,  0.03589064, -0.01951779, ...,  0.00248256,
           0.0077274 ,  0.01249541],
         [ 0.03517856, -0.00963541, -0.00515503, ..., -0.00965383,
           0.00070368, -0.01995948]],

        ...,

        [[-0.04234631,  0.03789089, -0.00772819, ..., -0.01769685,
          -0.03292959,  0.03434884],
         [ 0.02146093,  0.01694347,  0.00318442, ..., -0.01881772,
          -0.01668198,  0.02567464],
         [ 0.03571705,  0.03259897, -0.01294047, ...,  0.02712655,
           0.02608032, -0.01180965]],

        [[ 0.01185928, -0.04079281, -0.01112602, ...,  0.04131548,
           0.04050212, -0.00183395],
         [-0.04209455, -0.02359672, -0.00488375, ...,  0.01303448,
          -0.00892901, -0.03552562],
         [ 0.01119863,  0.01505886, -0.01598817, ...,  0.03405465,
           0.00575322,  0.03870751]],

        [[-0.03639854,  0.04030317,  0.0210116 , ..., -0.02623046,
           0.00511633, -0.00511916],
         [ 0.01786057, -0.02768354, -0.0043756 , ...,  0.03386175,
           0.02333821,  0.02404093],
         [-0.03236726,  0.02467468,  0.01458252, ..., -0.03318507,
           0.04025299,  0.01569658]]],

       ...,

       [[[-0.02181565, -0.03812791,  0.03461647, ...,  0.03866912,
           0.01784445,  0.04158403],
         [-0.02396281,  0.00245118,  0.03562684, ...,  0.0102238 ,
          -0.02127265,  0.02097363],
         [ 0.02662035, -0.032344  , -0.03497766, ..., -0.0173105 ,
           0.0330125 ,  0.02158869]],

        [[ 0.03021639, -0.03994903, -0.02158598, ...,  0.02239507,
          -0.00532783,  0.03573686],
         [-0.00708783, -0.00955907,  0.01542434, ...,  0.0060139 ,
          -0.00969837,  0.00809037],
         [-0.0187736 , -0.03951379, -0.00171888, ..., -0.03458672,
          -0.01707613,  0.03584855]],

        [[ 0.02697386,  0.0227053 , -0.02373461, ...,  0.02323208,
           0.02264023,  0.00521714],
         [ 0.03144782,  0.02519536, -0.01290952, ...,  0.01900919,
           0.01282516, -0.00580743],
         [ 0.00347913,  0.04055054,  0.0163502 , ...,  0.04182776,
           0.01810824,  0.02913269]],

        ...,

        [[-0.00699282, -0.02034857,  0.01357543, ...,  0.02949038,
          -0.01741139, -0.02793814],
         [ 0.03612771, -0.01679454,  0.00621703, ...,  0.0012925 ,
           0.01140555, -0.01587559],
         [-0.03647219, -0.0248061 ,  0.01499417, ..., -0.03298131,
          -0.01821174, -0.00639778]],

        [[-0.0124757 , -0.02934109, -0.03352599, ...,  0.01948061,
           0.01547889,  0.03470867],
         [ 0.02129364, -0.03426546, -0.02001888, ...,  0.02287591,
           0.02132068,  0.019723  ],
         [ 0.01685879, -0.0293836 , -0.03255122, ...,  0.03241389,
           0.00359154, -0.01897937]],

        [[ 0.00998483, -0.03145854,  0.025159  , ..., -0.00608719,
           0.03768683, -0.01049537],
         [-0.00681999,  0.04274154, -0.01796091, ...,  0.00209755,
           0.03404817, -0.0367528 ],
         [-0.02953758,  0.03915798, -0.00744066, ..., -0.03437715,
          -0.00139549,  0.02726422]]],

       [[[ 0.00132366, -0.01231837,  0.02457621, ..., -0.01362764,
          -0.02422343,  0.01555888],
         [ 0.03975777, -0.01529455, -0.00347792, ..., -0.02121332,
           0.02689363,  0.01563011],
         [ 0.00771964,  0.00972354, -0.00712591, ..., -0.01438173,
          -0.04188299, -0.0042484 ]],

        [[-0.00451561,  0.00983775,  0.00607958, ...,  0.01211549,
           0.03849814, -0.00082758],
         [-0.01711717,  0.02956637, -0.03246796, ..., -0.02180483,
          -0.02713996, -0.03387011],
         [ 0.01791469, -0.02935785, -0.03102269, ...,  0.02160091,
           0.01942689, -0.02052332]],

        [[-0.01667717, -0.0326683 , -0.03564648, ...,  0.0072429 ,
           0.03514059, -0.02634143],
         [ 0.01950502, -0.00975966, -0.02890214, ..., -0.00114042,
          -0.02262777, -0.03157064],
         [-0.0098084 , -0.0039121 ,  0.00276146, ...,  0.0217625 ,
           0.02369581,  0.03543634]],

        ...,

        [[ 0.00755312,  0.01343475, -0.00427934, ...,  0.00775591,
           0.02321446,  0.03337752],
         [ 0.01661244,  0.00883742,  0.01554158, ...,  0.01742725,
           0.02783858,  0.02373559],
         [ 0.01748803,  0.02117841, -0.01117309, ...,  0.02326744,
           0.04078443, -0.00090919]],

        [[-0.03060398, -0.02152558,  0.01366274, ..., -0.0021921 ,
          -0.01093575,  0.01506961],
         [-0.02631433,  0.0001437 ,  0.03135208, ...,  0.0380754 ,
           0.03520514,  0.00907693],
         [-0.03707059, -0.01206359, -0.02500631, ...,  0.02470848,
           0.02481312,  0.01542969]],

        [[-0.01821166,  0.02100131,  0.03497771, ...,  0.03766722,
          -0.02053893,  0.00021088],
         [-0.01354444,  0.03923328, -0.0168089 , ...,  0.00970965,
           0.03874869,  0.04041659],
         [ 0.02845939,  0.03721001,  0.03801313, ..., -0.01394963,
           0.0335576 ,  0.00692736]]],

       [[[ 0.03618703, -0.0138254 , -0.00453906, ...,  0.02329758,
           0.03991621, -0.01152961],
         [-0.01452164, -0.02750105, -0.03441633, ..., -0.00388454,
           0.04104266, -0.00631983],
         [ 0.02425962,  0.04094199, -0.00775501, ..., -0.00427654,
           0.0076022 , -0.0125599 ]],

        [[ 0.02671588,  0.03458586,  0.00841282, ...,  0.03032816,
          -0.02575159,  0.01461332],
         [ 0.01187777,  0.03301452, -0.03002692, ..., -0.01510497,
           0.02217172,  0.02302239],
         [ 0.01616631,  0.00667933,  0.02903796, ...,  0.03481976,
          -0.01880755,  0.012068  ]],

        [[ 0.01323244, -0.03229237, -0.02649139, ..., -0.01777507,
          -0.03302069,  0.00621729],
         [ 0.02260002,  0.0165596 ,  0.03154979, ...,  0.00254676,
          -0.00893909, -0.02884208],
         [-0.00751556,  0.03365554,  0.03511561, ..., -0.02840066,
           0.02889479,  0.00891231]],

        ...,

        [[-0.02447437,  0.03960178, -0.03086215, ..., -0.03870507,
          -0.01697602, -0.02034575],
         [ 0.03166722, -0.02755606,  0.00750558, ...,  0.02459161,
           0.02179844,  0.0252091 ],
         [ 0.0228941 , -0.00440465, -0.02639789, ...,  0.01260859,
          -0.02058532, -0.02541496]],

        [[ 0.03373916, -0.01786761, -0.0122059 , ..., -0.01751993,
          -0.01739639, -0.00601044],
         [ 0.00135932, -0.0009327 ,  0.03774614, ...,  0.02985087,
          -0.01790373,  0.01864828],
         [ 0.030805  ,  0.03011939,  0.01722021, ...,  0.03570656,
          -0.01456108,  0.0192683 ]],

        [[-0.03611086,  0.04171606,  0.0315418 , ..., -0.03692954,
           0.00130274,  0.03395432],
         [ 0.02079141, -0.03837941, -0.03237452, ...,  0.03315201,
           0.00455385,  0.02035925],
         [ 0.01785033, -0.00887285, -0.03673003, ...,  0.03627278,
          -0.00523347, -0.01229645]]]], dtype=float32)>) was not created in the distribution strategy scope of (<tensorflow.python.distribute.mirrored_strategy.MirroredStrategy object at 0x0000020E2492E970>). It is most likely because some layers, model, or optimizer was being created outside the distribution strategy scope. Try to make sure your code looks similar to the following.
with strategy.scope():
  model=_create_model()
  model.compile(...)

Update:
When instead of self.model, I import my model directly using model = DeepLabV3Plus(input_shape) everything is fine.
The difference is when I import the model directly, it generates MirroredVariable for all variables of layers, but when I use self.model it generated tf.Variable. Could you let me know how can I convert tf.Variable. to MirroredVariable. Maybe it can solve my problem.
`[MirroredVariable:{
0: <tf.Variable 'conv1_conv/kernel:0' shape=(7, 7, 3, 64) dtype=float32, numpy=
array([[[[ 2.74782069e-02, -2.23000664e-02,  4.21437509e-02, ...,

Why is this issue closed? Is there a solution??


TypeError: Converting circular structure to JSON

Just came across this issue. I can consistently reproduce this error by having my app throw an error from an unprovided service. I have a core module with a service in it that is used in my signup feature module / component (lazy loaded). When the service is removed from the providers array on the core module then this error occurs when I try and load the route to the signup component that uses it.
node v8.1.0
npm v5.4.0
Mac OS v10.12.6
Chrome Version 60.0.3112.113
99 Jeep Cherokee XJ
package.json
"@ngrx/store-devtools": "~4.0.0",
app.module.ts
!environment.production ? StoreDevtoolsModule.instrument() : [],
Error and stack trace
ERROR Error: Uncaught (in promise): TypeError: Converting circular structure to JSON TypeError: Converting circular structure to JSON     at Object.stringify (<anonymous>)     at Object.e.stringify (<anonymous>:1:9263)     at stringify (<anonymous>:1:18991)     at toContentScript (<anonymous>:1:20255)     at Function.sendMessage [as send] (<anonymous>:1:20741)     at DevtoolsExtension.webpackJsonp.../../../../<EMAIL_ADDRESS>(store-devtools.es5.js:184)     at ScanSubscriber.StoreDevtools.applyOperators.state [as accumulator] (store-devtools.es5.js:566)     at ScanSubscriber.webpackJsonp.../../../../rxjs/operator/scan.js.ScanSubscriber._tryNext (scan.js:109)     at ScanSubscriber.webpackJsonp.../../../../rxjs/operator/scan.js.ScanSubscriber._next (scan.js:102)     at ScanSubscriber.webpackJsonp.../../../../rxjs/Subscriber.js.Subscriber.next (Subscriber.js:89)     at Object.stringify (<anonymous>)     at Object.e.stringify (<anonymous>:1:9263)     at stringify (<anonymous>:1:18991)     at toContentScript (<anonymous>:1:20255)     at Function.sendMessage [as send] (<anonymous>:1:20741)     at DevtoolsExtension.webpackJsonp.../../../../<EMAIL_ADDRESS>(store-devtools.es5.js:184)     at ScanSubscriber.StoreDevtools.applyOperators.state [as accumulator] (store-devtools.es5.js:566)     at ScanSubscriber.webpackJsonp.../../../../rxjs/operator/scan.js.ScanSubscriber._tryNext (scan.js:109)     at ScanSubscriber.webpackJsonp.../../../../rxjs/operator/scan.js.ScanSubscriber._next (scan.js:102)     at ScanSubscriber.webpackJsonp.../../../../rxjs/Subscriber.js.Subscriber.next (Subscriber.js:89)     at resolvePromise (zone.js:795)     at zone.js:847     at ZoneDelegate.webpackJsonp.../../../../zone.js/dist/zone.js.ZoneDelegate.invokeTask (zone.js:425)     at Object.onInvokeTask (core.es5.js:3881)     at ZoneDelegate.webpackJsonp.../../../../zone.js/dist/zone.js.ZoneDelegate.invokeTask (zone.js:424)     at Zone.webpackJsonp.../../../../zone.js/dist/zone.js.Zone.runTask (zone.js:192)     at drainMicroTaskQueue (zone.js:602)     at <anonymous>

@jbeckton - running into this error right now. What did you do to resolve?

Nevermind, figured out the issue is triggered when any errof is thrown in the constructor of the component being navigated to. Unfortunately, the error above is swallowing the real error. Thanks for the tip, @jbeckton.

#356 is a better resolution to this problem.

@dougludlow one of the ngrx developers created a fix for this issue, the ngrx router store now handles the errors better, just try getting the latest release.

@dougludlow Had the same issue, the JSON error was swallowing the real error (missed importing the a rxjs operator used on an activatedRoute).
Left my comment here in case anyone else drops with a similar issue while Googling https://github.com/ngrx/platform/issues/356#issuecomment-336733036


[cli/serve command] operationHeaders isn't reading headers when using the array syntax

Describe the bug

When using mesh dev or mesh start with the built-in serve command, the syntax :
Authorization: "{context.headers['authorization']}"

isn't read correctly by mesh.
Using the object access syntax is working :
Authorization: "{context.headers.authorization}"

To Reproduce
Steps to reproduce the behavior:

under operationHeaders :
Authorization: "{context.headers['authorization']}"

Expected behavior

As the doc advertise for the array syntax, I guess both should work.
Environment:

OS:
"@graphql-mesh/cli": "0.34.0"
NodeJS:

Additional context

Closing this issue for now. Feel free to create a new one with a clear and minimal reproduction.


Fix Chrome Store Publishing

Publishing from CircleCI to the Chrome Store is broken. Doing manual deploys for now.

Fixed in January together with (I think) #46.


[BEAM-3415] Amend TestPipeline to work with JUnit5

This makes TestPipeline implement a JUnit5 extension, making it usable with the JUnit5 extensions mechanism (what used to be "Rules" in JUnit4).
It's semantically the same code, the only outward noticeable change would be JUnit5 uses @Tag instead of @Category, and they're string values not classes, so @Category(NeedsRunner.class) becomes @Tag("needsRunner") -- open to guidance on the value here, happy to consider the existing one a placeholder.
This also adds a separate JUnit5 test for the capablities of TestPipeline and a JUnit5 run in the build.gradle. It should be fine running side-by-side (both in theory and at least my experience) since they use different annotations etc., JUnit5 will only pick up the Jupiter annotations and JUnit4 will only use the org.junit ones.
R: @lukecwik
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[x] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Implementing JUnit5 hooks in the existing TestPipeline ended up breaking compilation across tons of components, due to the now required Jupiter API, which I figured would be unacceptable for clients.
I ended up opening up TestPipeline a bit (setting the constructor to protected, and some select fields), and extending it with a separate TestPipelineJunit5 class. This way people should hopefully not get compile problems unless they use the new class.
Would love to hear thoughts and iterate though, if this is not acceptable. Also happy to just collapse them back into TestPipeline and add the dependency to all the modules, if we think that level of breaking everyone's stuff is acceptable.

Also just noticed template says comment, not tag in description, so just in case there's behind-the-scenes automation
R: @lukecwik

@tysonjh or @kennknowles
Could you please help review this PR or find someone help this?

Run Java_Examples_Dataflow PreCommit

@boyuanzz @tysonjh @kennknowles @TheNeuralBit any feedback / changes needed? How would we go about getting this merged?

Haven't had time to look deeply, but just checking that this will still work with JUnit 4?

@kennknowles yes, all the JUnit5 stuff is additive and just enables support, nothing changes with respect to JUnit4.

@TheNeuralBit yeah I saw that, and didn't understand why they'd done it. I'd be keen to hear from the author themselves @rmannibucau if you have context, but my guess would be both JUnit5 and Beam have evolved in the 2 years since. Purely just implementing the before / after TestExecutionCAllback interfaces from JUnit5 seems to work, the added JUnit5 test suite verifies the same things the JUnit4 ones does, this includes the various calls to PAssert. Anecdotally I also did some local testing on a company project where we're looking to utilise this by copy-pasting all the relevant bits and just implementing the 2 JUnit5 extensions, and all our tests seem to pass by just swapping out the annotations and runner.
tl dr there's a test suite exercising PAssert calls and local testing would seem to confirm that the code is fine.
I don't mind an extension, since that solves the whole "clients would be required to depend on both versions of JUnit" debacle, but not sure that's justified for 2 classes (the TestPipeline and the sys prop restorer util). If that's the preferred route I'd love some thoughts on whether we want to just copy-paste the existing TestPipeline and allow for potential divergence of the JUnit4 / 5 implementations, or whether the current inheritance coupling seems appropriate.

@emilssolmanis original migration goal was to support natively junit5 (and not rely on junit4 imports which has several pitfalls in integrations layers (thinking to maven/gradle plugins and IDE) + in user land with completion packages - "why my @test does not run"...wrong package). At that time beam community kind of rejected this as not being a goal/needed for google - at that time - so got rid of this need by providing an alternate testing solution and bypassing beam native testing stack which was not moving to user direction fast enough. Was still the case a few months ago but maybe it changed in between (I don't use beam at all anymore so stopped to follow the project).

Yea, it is a tough problem. It doesn't really have to do with Google or any specific company.
Judging by any data I can find, JUnit4 is still by far the dominant version in use. Why? JUnit5 isn't so much an upgrade but just a totally different project with a shared name and some similar concepts, and also with API/functionality regressions that are not a priority for JUnit team. The migration cost for a large organization is huge, as much as migrating to an unrelated test framework, and benefits not enough to justify the investment. It reminds me of Python 2 to 3.
This is just my basic understanding of the current status.
Why does Beam need to care about what test frameworks others are using? There are real reasons:

We export our "Beam Model Compliance" test suite via JUnit4 category. So it is a critical breaking change to do that differently.
So we also do want to support people running tests in their own infrastructure to test their runners.
And of course, being part of the dominant version gives network effects of the best maintenance and support via tooling.

Of course, in the same spirit, if there are Beam users/contributors who want to run our tests in their infrastructure that uses JUnit5 it is nice to support them. So I very much support figuring out a way to help both to work.

Judging by any data I can find, JUnit4 is still by far the dominant version in use. Why? JUnit5 isn't so much an upgrade but just a totally different project with a shared name and some similar concepts, and also with API/functionality regressions that are not a priority for JUnit team. The migration cost for a large organization is huge, as much as migrating to an unrelated test framework, and benefits not enough to justify the investment. It reminds me of Python 2 to 3.

Using CI data yes because it exists but using new project stacks not really because projects start on up to date framework and junit4 said they will not upgrade the framework (until a CVE pops up) so Junit4 can be seen as dead as of today and JUnit5 the main alternative. I know it is hard to have the difference in terms of data but people doing service on new projects will surely confirm what i saw over the last year.
In terms of cost, it is generally cheap to have done it on big suites, the only cost is about migrating rules to extensions and it is not that costly in practise and rest is mainly a sed so not as bad as python even if it can look like that upfront.
Last point is that with junit 5.7, junit5 caught back its lateness on junit4 (mainly rules which were enabling more before) so guess it is time for beam to move anyway. It can't go back anyway :).

We export our "Beam Model Compliance" test suite via JUnit4 category. So it is a critical breaking change to do that differently.

JUnit 5 has groups and it does not have to break, guess junit4 should stay in a beam-junit4 module and jupiter move to a beam-junit5 one. Current beam-core ("put it all") will become a kind of bom (with an empty jar to not break deps too much?) and core can move in a beam-core-engine, no breaking change for anyone if desired.
To have done the work, there is no real reason to be that coupled with a testing framework - don't get me wrong, it was natural in a first version of the framework, I'm not blaming - so I guess it is mainly about abstracting the few basic primitives (primitives not the whole hamcrest stuff ;)).
Side note: it can be an opportunity to split beam internal harnessing and user testing API. Using direct runner enable to cover user case quite well and does not need the test pipeline or so which mainly brings constraints and as much off toggle than feature so rethinking the API and maybe splitting both makes sense to me.

@kennknowles @rmannibucau
sorry for the lack of response, been a bit snowed under at work.
So 2 points I can extract from the conversation so far:

the dominant version of Junit and all that stems from there -- I think this would require data, and I'm not sure we have that available. I'm with @rmannibucau in that most new projects will start in Jupiter, and most old projects stay in Junit4 purely by inertia. I don't think there's a right or wrong solution, and supporting either seems simple enough, at least from what I've seen so far, so being independent of the testing framework sounds like a good thing to aim for and should be achievable.

We export our "Beam Model Compliance" test suite via JUnit4 category. So it is a critical breaking change to do that differently.

Not sure I follow, is this documented somewhere I can have a read about? Mainly I don't immediately understand how this would affect clients.

@emilssolmanis I assume @kennknowles speaks about https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/ValidatesRunner.java and the related tests which validates a runner impl complies to the spec (if about app code then it is not a blocker at all since it will fall in junit5 extension impl). In this spec case the runner will have to run the tests through junit4 which is not a big deal for now (not all runners - outside the project itself -try to pass these tests and for now it is a bean internal thing anyway). So overall no blocker to support a junit5 testing solution IMHO.

what is the progress on this PR? I hope we can merge quickly :)

@MaximeFrancoeur Seems like work is suspended because of different opinions on this.
Imho, this sensitive topic requires a discussion on mailing lists (dev@ and user@) and, probably, a vote before taking a final decision.

Hope discussions on that are continuing. Would like to fully migrate my projects that use Beam away from JUnit 4.

I am not aware of a discussion going on<EMAIL_ADDRESS>You could definitely start it up!


crouton working with ubuntuGnome zesty upgraded to artful

Please paste the output of the following command here: sudo edit-chroot -all

Please describe your issue:
..working but every time I exit from ubuntu and returning to chronos shell, sudo startgnome doesn't start because of installed gnome-shell extensions, Removed the extensions, it starts again
If known, describe the steps to reproduce the issue:

@fedrox
I'm not sure I follow, please show us the output of: sudo edit-chroot -all
Is this someting you installed in the chroot or something that was installed by crouton?
-DennisL


Make it possible for Firefox to review uProxy

In order to review uProxy, Firefox requires that they be able to either (1) verify the build from source is identical to what we submitted to the web store, which may be hard because of dependencies or (2) be able to decompile the submitted bundle into a format that they can review.
One FF reviewer has used this tool before, and it's possible we could script something similar to achieve (2).

Well, (1) may be easier.  They can build the package, unzip it, and diff the contents.  If the package versions are the same, is there anything outside of metadata (build dates, etc) that'd be a problem?

I suspect making a "shallow" reproducible build from our own dependencies would not be terribly hard ... but it also wouldn't be a complete solution, insofar as many of our dependencies are already compiled internally.  Recursively building all our dependencies reproducibly from source is a bigger headache.
I do think the "shallow" solution seems like it might provide enough improved clarity, at low enough cost, to be worth trying.


Playwright: Fixed TypeScript definition of click() and dragAndDrop()

Motivation/Description of the PR

3.3.0 introduced optional parameters in methods click() and dragAndDrop(). This fix allows using them in TS tests.

Applicable helpers:

[ ] WebDriver
[ ] Puppeteer
[ ] Nightmare
[ ] REST
[ ] FileHelper
[ ] Appium
[ ] Protractor
[ ] TestCafe
[x] Playwright

Applicable plugins:

[ ] allure
[ ] autoDelay
[ ] autoLogin
[ ] customLocator
[ ] pauseOnFail
[ ] coverage
[ ] retryFailedStep
[ ] screenshotOnFail
[ ] selenoid
[ ] stepByStepReport
[ ] stepTimeout
[ ] wdio
[ ] subtitles

Type of change

[ ] :fire: Breaking changes
[ ] :rocket: New functionality
[x] :bug: Bug fix
[ ] :clipboard: Documentation changes/updates
[ ] :hotsprings: Hot fix
[ ] :hammer: Markdown files fix - not related to source code
[ ] :nail_care: Polish code

Checklist:

[x] Tests have been added
[ ] Documentation has been added (Run npm run docs)
[ ] Lint checking (Run npm run lint)
[x] Local tests are passed (Run npm test)
[x] TS tests passed (npm run dtslint)

WebDriver tests failed, but it doesn't seem to be caused by my changes. They fail in my environment even with a previous commit 29c9fc94 ("Update custom-helpers.md (#3271)", 2022-03-12)

I fixed the test "#seeTextEquals => should check text is not equal to empty string of element text" in WebDriver helper tests.
WebDriver grabs main section on https://codecept.discourse.group/ with an empty content from some reason:
<section id="main">
</section>

though if you open the page manually, e.g. in Chrome, there is a non empty content in that element:

Fixed by change of URL to main site CodeceptJS where it's possible to test the same case with the element .logo


Some changes to Provider and Property for more convenient use from Kotlin and Groovy

tl;dr
someTask.someProp.set("abc")
// becomes
someTask.someProp.value = "abc"

consumer.someInput.set(producer.someOutput)
// becomes
consume.someInput.source = producer.someOutput

someProp.get().asFile.writeText(otherProp.get())
// becomes
someProp.value.asFile.writeText(otherProp.value)

This is a proposal for some changes to to Provider and Property. Specifically:

Add read-only value property to Provider<T> to represent the current value of the provider. This is the same as get().
Add mutable value property to Property<T> to allow the current value of the property to be set and queried. This is the same as get() and set(T).
Add mutable source property to Property<T> to allow the current source of the property to be set. This is the same as this and set(Provider<T>).

The intention is that get() and set() would be deprecated and replaced by these new methods.
None of these properties are nullable. Instead, a method to explicitly discard the value would be added (not included in this change yet).
The intention is also that the Groovy DSL convenience would be deprecated.
Similar changes would be made to the other property types. For example:
someTask.input.fileProvider(legacyTask.map { it.outputFile})
// becomes
someTask.input.fileSource = legacyTask.map { it.outputFile }

Contributor Checklist

[ ] Review Contribution Guidelines
[ ] Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
[ ] Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
[ ] Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
[ ] Provide unit tests (under <subproject>/src/test) to verify logic
[ ] Update User Guide, DSL Reference, and Javadoc for public-facing changes
[ ] Ensure that tests pass locally: ./gradlew <changed-subproject>:check

Gradle Core Team Checklist

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation
[ ] Recognize contributor in release notes

I'm not super convinced: it doesn't make the DSL much better, at least not enough to justify deprecating methods IMO. I'd very much prefer we avoid the many deprecations/API changes we're doing which make the lives of plugin authors painful.

I think it makes the DSL better by removing the parentheses, favoring assignments and better communicating the intent by using actual words instead of the less meaningful get and set. Distinguishing setting a value or a source of value also makes a lot of sense to me and makes the DSL more explicit, when you read some.source = ... you know directly that's it's a lazy wiring, when you read some.value = .. you know directly that's it's an eager assignment.
I share Cédric's concerns about the impact of deprecations. Build logic often has a lot of those wiring. But I also think it's worth it because of the reasons above, especially the better communication of intent.

I'm not saying the proposal doesn't have value. I'm saying I don't see much value compared to breaking existing plugins with deprecations. I'm seeing a lot of swearing on Twitter about breaking changes in APIs we promote, typically the provider API. It's a cost to invest in those new APIs, and changing them is painful as it makes it harder for plugin authors to support a wide range of Gradle versions.

That may just be me, but I fail to see a clear improvement to what is already present. First, it as much or more typing for the users. Second, it add another layer of confusion regarding how to wire values together. I fear a snowball effect every time a user change from an eager type to a lazy type because of the value vs source. I get that it's more explicit but for users that don't care about their build system, I think it's just going to add more swearing. I think most improvement such as discarding the value can definitely be achieve through the current API. Finally, I agree that adding more deprecation would just frustrate users even more.
I think going forward with deprecating the "magic" Groovy decorator would be more beneficial then adding new APIs. However, I would argue that we should deprecate the Groovy decorator once we widely use the Provider API in the Gradle codebase to give a progressive migration path to users.

I'll echo some of the other comments.  I think my gut feelings are...

We're far down the path of adoption of get/set, so removing those APIs would be very disruptive.
The combination of getters and setters for value and source make the API more confusing to use. I think one of the good things with the existing value(...), convention(...) and set(...) methods is that they can be explained as influencing one "thing" (the eventual value of the provider) where now it looks like there are two different things (a value and a source) plus the eventual value of the provider.
Like above, I think it gets worse where we want to provide setters for other types (File -> FileSystemLocation) since it would require more getter/setter pairs to be added. This might be an argument against providing these conveniences on the Provider/Property types.
None of the examples are super compelling.  The first two could be done today with value(...):

someTask.someProp.set("abc")
// becomes
someTask.someProp.value("abc")

consumer.someInput.set(producer.someOutput)
// becomes
consume.someInput.value(producer.someOutput)

I get that this isn't an assignment, but I don't know if assignment is the right way to think about how a Provider gets a value.
OTOH, I think getValue could be added separately from this as it's slightly better without the complications above.
I think we'd be better off spending time on making it easier to use the existing Provider APIs with older APIs than to migrate to a slightly different Provider API getter/setter syntax.

Would it make sense to revive and prioritize this PR?


Error when trying to clean up spies after running jasmine tests since jasmine 2.5.0

A change in the way that jasmine is cleaning up its spies means that as of version 2.5.0 of that testing framework the test results will not display in the browser window when using the tool in conjunction with zone.js patching.
This issue can be reproduced using any of the angular 2 testing samples if the package definitions for jasmine are updated to 2.5.0 or higher. The developer of Jasmine has suggested that this issue is related to the way jasmine is patched to work with zone js so a fix needs to be implemented here rather than in jasmine itself.

@GZidar, could you provide a reproduce sample repo with the package.json included? Thank you.

repro.zip
The attached zip hopefully shows the error in action. If you want the error to go away change the version of jasmine-core in package.json back down to 2.4.1 and reinstall to get it working, but the sample as it stands will (hopefully) show the bug in action.

@GZidar , I just try your repo, and I don't think it is a zone.js problem, you can try the attached systemjs.config.js, it should work, and you can also see this page for custom_boot of jasmine.
systemjs.config.js.zip

@JiaLiPassion your changes to the systemjs.config.js do allow the tests to run using the latest version of jasmine. Thank you for looking into this.

@GZidar , thank you for confirm this, could you close this issue?

Hi again. I think this issue has resurfaced with the release of version 0.8.5. I just updated my repro to use angular 4.0.1 and the jasmine test results show correctly when using zone.js 0.8.4 but fail with the same spies clean up error when using zone.js 0.8.5... I have attached two zips - one that works as expected and the second showing the error. You should see that the only difference is the version of zone.js that is being used.
notworking.zip
working.zip

@GZidar, I see, you can try the attached one, it is a little tricky, I will figure out how to make it better, but it can work now, I modified the index.html and system.config.js
fix.zip

@JiaLiPassion Thanks for the quick response. Your changes do work, but as you say it is a bit crude looking. Would you like me to keep the issue open until you come up with a more elegant fix?

@GZidar , sure, let's keep it open until we can find a better solution:)


why doesnt webvm work?

everytime i load into the vm, it just freezes

Hi, could you provide more information about your setup? Are you trying to load our hosted online version, your own github pages or a local one? If you open the console in your browser, are you seeing any logs or any errors in the network tab?

I'm closing this issue as there is not enough information to work on this. If you're still having trouble, feel free to open a new issue with more information.


can  i use other features instead of normal?

if i have R,G,B and other features of each point, how should i put them into the octree? thks

Yes, RGB and other features can be used. Probably the results can be improved.
To this end, you just need to modify the code about octree construction and OctreeDatabaseLayer.
It is is not that difficult.
From the original code,  you can identify where the normal signal is processed, and then modify the relevant code accordingly.
If you still have any problems, feel free to mail me.

@wang-ps  thank you


Problem with online image

Hello,
I tried to use the ViewAsPDF to export one of my details view. I have the CSS, the text and image resources but I don't succeed in having online images. I tried with an image sent by a controller (with a return File) and with an online image available on yahoo. Each I have a litlle square instead of my image.
Before I used ActionAsPDF and it worked but I have a security problem and according to the #57 issue I tried to use ViewAsPDF. Could you tell me if there is a restriction with online image?
Thanks.

I am also seeing this problem.  Any updates would be appreciated!

For me this works in a local dev environment, but the images are not being rendered when my web app is published. I'm not sure why it is treated any different.

I'm facing the exact same problem, it looks like the pdf gets rendered before the images are loaded

Can I look at the image? and the HTML fragment that loads it. I would like to reproduce the problem.


example using difftool: make reference to hash; closes #329

this is hopefully more robust since then there is less risk of diffing
something that is staged and seeing zero difference

What it most useful depends on whether they staged the changes or not. In the exercise, they're told to make changes to the files and then git difftool. If they also git add . in between is not so hard to predict. I am OK with any.


mesa: Upstream removed lib.platforms.mesaPlatforms

This fixes the build with current nixos-unstable.
See https://github.com/NixOS/nixpkgs/pull/338337

Thanks for the fix, I can confirm it works.
As I can see, the platforms are now contained in mesas (mesa.meta) platforms
https://github.com/alyssais/nixpkgs/blob/ef3692d0559c3ba8b682f07fd8d19669bbee4893/pkgs/development/libraries/mesa/common.nix#L32-L36
I am unsure but but isn't it possible to inherit those platforms in the mesa-asahi-edge platforms instead of having two sources of truth?

The entire mesa package is vendored. Because of this I chose to not depend on the original at all. If @tpwrules wants to do that I can change it.

Thats a good point, let's see what @tpwrules says.

This works and looks good. I would indeed prefer to duplicate the list rather than just change to inherit from a different place. Thank you!


Replace numpy linalg with scipy linalg

NumPy's linear algebra library is subpar. SciPy's uses LAPACK which is well-verified and excellent.
It works the same way. Just import scipy.linalg instead.
https://github.com/quinngroup/ornet/blob/eigenspectrum/src/analysis/util.py#L51

Also, use the eigh function instead of the generic eig, because you are indeed using a Hermitian/symmetric matrix. As a result--and as you can see from the documentation--the eigenvalues and eigenvectors will already be ordered for you.
https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eigh.html#scipy.linalg.eigh

Don't even need sort_eigens anymore.


Add all Page Components

Add empty component files for the rest of the six pages. Name them so it’s easy to understand which one is which, try to do something like 'if-else' instead of ’second', feel free to rename ones already made. Also add temporary button and description in UpperSection.tsx for each page so it’s easy for next person to transfer over description and buttons from Figma.

Look at Conditional Brainstorm google doc for short description of all pages


FR: Every sensor as attribute on every entity (for a single container instance)

Make all data for a container accessible as attributes for every sensor. E.g every sensor as attributes on the switch.
This will make it significant easier to use auto generated entities such as auto-entities.
As of now it can't be done as e.g. auto-entities does not allow to substitute domains for entities and therefore you can't filter for switch. and reference belonging sensors.

Yes, please. Add the attributes also to the switch entities: I want to use with auto-entities the switches of monitor_docker in order to access the toggle functionality.

I've solved this by creating template sensors in HA that takes the sensor.docker_*_state as state and everything else I want/need as attributes.... It is a bit of effort, but it works very well!
Here is my config:
  - platform: template
    sensors:
      docker_adguard_info:
        friendly_name: AdGuard
        value_template: "{{states('sensor.docker_adguard_state') | float | round(1)}}"
        attribute_templates:
          status: "{{states('sensor.docker_adguard_status')}}"
          memory: "{{states('sensor.docker_adguard_memory') | float | round(1)}}"
          memory_percent: "{{states('sensor.docker_adguard_memory_percent') | float | round(1)}}"
          cpu: "{{states('sensor.docker_adguard_cpu') | float | round(2)}}"
          network_speed_up: "{{states('sensor.docker_adguard_network_speed_up') | float | round(1)}}"
          network_speed_down: "{{states('sensor.docker_adguard_network_speed_down') | float | round(1)}}"
          network_total_up: "{{states('sensor.docker_adguard_network_total_up') | float | round(1)}}"
          network_total_down: "{{states('sensor.docker_adguard_network_total_down') | float | round(1)}}"
          uptime: >-
            {% set up_time = (as_timestamp(now()) - as_timestamp(states.sensor.docker_adguard_up_time.state)) %}
            {% set days = (up_time // (60 * 60 * 24)) | int %}
            {% set weeks = (days // 7) | int %}
            {% set hours = (up_time // (60 * 60)) | int %}
            {% set hours =  hours - days * 24 %}
            {% set minutes = (up_time // 60) | int %}
            {% set minutes = minutes - (days * 24 * 60) - (hours * 60) %}
            {% set days = (days | int) - (weeks * 7) %}

            {% macro phrase(value, name) %}
                      {%- set value = value | int %}
                      {%- set end = 's' if value > 1 else '' %}
                      {{- '{} {}{}'.format(value, name, end) if value | int > 0 else '' }}
            {%- endmacro %}

            {% set text = [ phrase(weeks, 'Week'), phrase(days, 'Day'), phrase(hours, 'Hour'),
                            phrase(minutes, 'Min') ] | select('!=','') | list | join(', ') %}
            {% set last_comma = text.rfind(',') %}
            {% set text = text[:last_comma] + ',' + text[last_comma +1:] %}
            {{ text }}

You can define the sensors inside 1 sensor, yes it is not in the switch, I know that. The problem is to include it in the switch, means we need to add of code again. So I will close this ticket.


Guidance 0.5

Still getting empty answer from GGML model

No indication that is going to work, closing


Remove version: Tableau.Reader version 24.1.1132

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

For moderators or/and anyone else

[!Caution]
As a result of several personal problems I've had over the last few months + my final exams coming up very soon, I'm currently unable to look at my GitHub notifications every day and maintain my executions correctly.
If you detect a certain problem on a certain pul request:

Either you can close the pull request directly without mentioning me.
Or you can send me a message on X (Twitter): @Exorcism0666

Thank you for your understanding, and I'm sorry if some pull requests can't be approved due to the situation I'm in.

This account is automated by Github Actions and the source code was created by CoolPlayLin. If you have any questions about any pull request, don't hesitate to ping Exorcism0666.

[!warning]
With the new version of Komac, numerous Pull Requests may contain bugs or unwanted elements. If such issues arise, please tag russellbanks so that these bugs can be addressed in future updates.

[!important]
Please carefully review these Pull Request before merging. If it is a Pull Request for removing incorrect content and the URLs are issue, free upon manual checking, please close this Pull Request directly. (It is best to inform Exorcism0666 of the closure.)

良い一日をお過ごしください！

URL: https://downloads.tableau.com/tssoftware/TableauReader-64bit-2024-1-3.exe
Response Error: The remote server returned an error: (403) Forbidden.
(Automated message - build 912)


fix(fetcher): ensure proper content options for same repository coord…

…inates
fixes #60

Codecov Report

Merging #63 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master    #63   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           3      3
  Lines         137    139    +2
=====================================
+ Hits          137    139    +2

Impacted Files
Coverage Δ

src/fetchers.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65b9c97...68f6ac7. Read the comment docs.


assertion failed: g_current_activity_ == nullptr

What version of gRPC and what language are you using?
Python
grpcio==1.46.0 (debug build)
What operating system (Linux, Windows,...) and version?
Arch Linux kernel 5.17.5-arch1-1
What runtime / compiler are you using (e.g. python version or version of gcc)
Python 3.10.2, 3.8.13
gcc 11.2.0
binutils 2.38
glibc 2.35
What did you do?
Running a Flask server with eventlet monkey patching. Multiple concurrent requests with gRPC results in a assertion failed error.
Here is a reproducible example: https://github.com/AlexJMohr/grpc-sigsegv
What did you expect to see?
Concurrent requests should not crash the Flask server
What did you see instead?
flask server crashes with:

assertion failed with a debug build of gRPC
SIGSEGV with normal build

<IP_ADDRESS> - - [05/May/2022 09:26:09] "GET / HTTP/1.1" 200 -
E0505 09:26:15.737590610  200702 activity.h:154]             assertion failed: g_current_activity_ == nullptr
Fatal Python error: Aborted

Current thread 0x00007f35f7446740 (most recent call first):
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/grpc/_channel.py", line 595 in _consume_next_event
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/grpc/_channel.py", line 607 in _next_response
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/grpc/_channel.py", line 642 in _next
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/grpc/_channel.py", line 426 in __next__
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/google/ads/googleads/interceptors/response_wrappers.py", line 88 in __next__
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/google/api_core/grpc_helpers.py", line 73 in __init__
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/google/api_core/grpc_helpers.py", line 147 in error_remapped_callable
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/google/api_core/gapic_v1/method.py", line 154 in __call__
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/google/ads/googleads/v10/services/services/google_ads_service/client.py", line 3426 in search_stream
  File "/home/amohr/work/grpc-sigsegv/app.py", line 49 in test
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/flask/app.py", line 1509 in dispatch_request
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/flask/app.py", line 1523 in full_dispatch_request
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/flask/app.py", line 2077 in wsgi_app
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/flask/app.py", line 2095 in __call__
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/werkzeug/debug/__init__.py", line 311 in debug_application
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/werkzeug/serving.py", line 324 in execute
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/werkzeug/serving.py", line 335 in run_wsgi
  File "/opt/python-dbg/lib/python3.10/http/server.py", line 415 in handle_one_request
  File "/opt/python-dbg/lib/python3.10/http/server.py", line 427 in handle
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/werkzeug/serving.py", line 363 in handle
  File "/opt/python-dbg/lib/python3.10/socketserver.py", line 747 in __init__
  File "/opt/python-dbg/lib/python3.10/socketserver.py", line 360 in finish_request
  File "/opt/python-dbg/lib/python3.10/socketserver.py", line 683 in process_request_thread
  File "/opt/python-dbg/lib/python3.10/threading.py", line 946 in run
  File "/opt/python-dbg/lib/python3.10/threading.py", line 1009 in _bootstrap_inner
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/eventlet/green/thread.py", line 64 in wrap_bootstrap_inner
  File "/opt/python-dbg/lib/python3.10/threading.py", line 966 in _bootstrap
  File "/home/amohr/work/grpc-sigsegv/env/lib/python3.10/site-packages/eventlet/green/thread.py", line 43 in __thread_body

Extension modules: greenlet._greenlet, __original_module__thread, __original_module_select, __original_module_time, markupsafe._speedups, grpc._cython.cygrpc, google.protobuf.pyext._message, yaml._yaml (total: 8)

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).
See TROUBLESHOOTING.md for how to diagnose problems better.
Anything else we should know about your project / environment?

@AlexJMohr Sorry that gRPC Python does not provide support for eventlet, we do support AsyncIO and gevent.

@AlexJMohr Supporting an asynchronous library is not an easy task for gRPC Python. The socket, timer, threading are managed by gRPC Core (C/C++), and the Python is the wrapper layer. To support a flavor of Python asynchronous library, we might need to plumb through the asynchronous-ness vertically across all stacks. For example, we need to make a TCP read in C/C++ play well in eventlet context, making it capable of yielding execution right back to the Python event loop.
The gRPC Python team hope to support all async libraries via the AsyncIO interface, which will be the official async interface in Python in the long term.

@lidizheng thanks, no problem, i just updated to the latest version of gRPC on our production servers, i will create a new issue if it i got this error again with the new version, i've contributed to this ticket because it is probably related.

@lidizheng I gave gevent a try and it also crashes with assertion failed in the same way. I updated the repro example here: https://github.com/AlexJMohr/grpc-sigsegv

I started to encounter similar problem.
I noticed that it was working fine with grpcio==1.44.0, but the errors started to appear at least on 1.46.1
I am using python 3.10 with gunicorn and uvicorn workers.
Unfortunately, I can not provide a reproducible example, because in my case the error appear from time to time.

@AlexJMohr @botsman Hi, I think you might encountered the same issue related to forking. To learn more about gRPC's fork-support, see doc. Gunicorn forks the worker processes, and if the application creates gRPC objects at global scope, there is a chance of triggering it. Solutions:

Defer the gRPC objects' creation till the fork is finished;
Supply following flags to turn on fork-support:

export GRPC_ENABLE_FORK_SUPPORT=true
export GRPC_POLL_STRATEGY=poll

Tried again with those environment variables and it still crashes the gunicorn worker processes. Also crashes the Flask dev server.

In this case, it means we will need to go for solution No.1, defer the gRPC object creations. You may find this example useful: https://github.com/grpc/grpc/tree/master/examples/python/multiprocessing.
For Gunicorn, it loads the module before forking, so if there are any gRPC object (client or server) created in module initialization phase, it will likely to crash during the fork. For more background about why forking support is hard, see https://github.com/grpc/grpc/blob/master/doc/fork_support.md.
If you think gRPC as a project should prioritize on this, you can create a feature request.

I did not test proposed solutions, but pinned grpcio version for now: grpcio==1.44.0
Perhaps I will test those later

Today I've tried to defer the gRPC objects' creation till the fork is finished, but this did not work for me, unfortunately
Could it be related to that fact that I run gunicorn with uvicorn workers?
gunicorn -k uvicorn.workers.UvicornWorker


Role is not mandatory for a user when using OAuth login

Bug description
Role is mandatory when default superset login is used. But when using OAuth Login, the role becomes optional for a user. This in turn leads to 500 Internal Server Error when a user that has no role logins to superset.
How to reproduce the bug

Configure superset to use any other OAuth login.
Go to List Users and change the role of any user to None and then save it.
Login using that user whose role has been removed.
Notice 500 Internal Server Error.

Screenshots/recordings
Superset version
3.1.3
Python version
3.9
Node version
16
Browser
Chrome
Additional context

Checklist

[ ] I have searched Superset docs and Slack and didn't find a solution to my problem.
[ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
[ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

@dosubot This is only applicable if I can create a new user through superset in the OAuth. In our case, we do not want to create a new user and the option to create a new user is shouldn't be there. We just want the role to be a mandatory field and for existing users, at least one role should be present.

@dosu But does this ensure that role becomes a mandatory field and on editing a user profile the role cannot be left empty?

@dosu Using native login, it is already a mandatory field, why is it when using Oauth this is not a mandatory field. Also, we are not maintaining codebase instead using docker image. So, editing the code suggested as above is not possible

@dosu this is not working

@dosu I can still see that no role is required for a user and it is giving 500 internal server exception

@dosu why users are always bind to public role


Scala Core: add support for Unversioned Schema

com.acme/order/jsonschema/?-?-?

com.acme/order/jsonschema/1-?-?

com.acme/order/jsonschema/1-3-?

Need a good name for this...

Schema placeholder
Schema matcher
Schema basis

Just random thoughts on it.
I think would be a very common to advice users to set it as 1-?-?, because for me if data-producers are changing Schema of data so it become back-and-forward incompatible - they usually announce it publicly and change API path (/api/v2) or similar.

I hear you - but I'm not sure I trust data-producers to understand precisely what changes are backwards compatible... Think how many APIs are generated by dynamically typed languages as a starting point!

What about PartialVersion instead of UnversionedSchema?

I like PartialVersion more (functions, right), but problem is that ?-?-? not even partial, it is really unversioned.

OpenVersion?

I really like OpenVersion. It says that we are staying open-minded about what schema version validates this schema...

So, let it be.
sealed trait SchemaVer
case class OpenVersion() extends SchemaVer
case class FinalVersion() extends SchemaVer

What about final?

ClosedVersion - meaning fully defined?

Yes. May be if we're going to consider ClosedVersion... what about SealedVersion?

Thinking about adjective. "Open-versioned data" sounds really good. Close-versioned... not so good.

FullVersion
ExplicitVersion
...

Pushing back as not related to r5 main scope (igluctl).

Where did we end up on the naming of these?

Good question. Let's go with PartialVersion and FullVersion maybe, if you don't have strong opinion against it?

Okay, sounds good

Bringing this into R7 as it seems we have enough time and it's already partly implemented.


"sails new project" does not work for me

Hi When I try to use "sails" command , the sails.js file is open
I add C:\Users\XXXX\AppData\Roaming\npm\node_modules\sails\bin; to my PATH variable

"does not work for me" is not an error message. Describe what is actually happening


New Electrobob Update Crash

https://paste.dimdev.org/wogetocawe.mccrash
It seems that Iblis and the new wizardry update are clashing.

Fixed.


amrfinderplus support & conda installation

Hi there,
Thank you very much for this package, it looks really promising.
I have seen in the documentation here in github that amrfinderplus is supporte but when installing package via pip latest version available to be installed version (0.0.1rc3).
I wonder if you could update the pypi package with the current version of your code.
Also, I wonder if you are planning in generating a conda installation available for this package.
Best regards
Jose

Happy to hear you find it useful.
We will eventually do a conda package, but we feel that this would imply a higher level of polish and trust than we think the package has attained so far.
On the other hand, updating the pyPI package is something we can do, but in the meantime, you can also install directly from github:
pip install https://github.com/BigDataBiology/argNorm/archive/refs/heads/main.zip

Great!
Thanks for the tip for installation from the github directory.
Looking forward to Conda/pip package availability!
Best regards

We just released version 0.1.0 on PyPI.
We're probably holding off on doing a conda package for now, though.


Update device-manager-problem-codes.md

CM_FAILED_ADD is not correct. It should be CM_PROB_FAILED_ADD.

Thanks Akihisa, good catch.  Those span tags don't appear on the rendered page, so I added a change to the actual text that appears on that line.


Changing the input display coordinate type causes the add-in to register coordinates as invalid

Background
Changing the Display Coordinates to something other than MGRS is causing Coordinate Conversion to think that MGRS coordinates are invalid.
Repro steps

In the Coordinate Conversion pane paste in 42S XD 45583 55343
Press Enter
coordinates are located
Click the Edit Properties button
Change the Display Coordinate to DD
Click Clear All
Paste in 42S XD 45583 55343 or 42SXD4558355343
7 Press Enter
Invalid Coordinate message is displayed
follow-on steps
Paste in 34.82997N 70.592043E
Press Enter
coordinates are located
Paste in 42S XD 45583 55343 or 42SXD4558355343
Press Enter
Invalid Coordinate message is displayed
Click the Edit Properties button
Change the Display Coordinate to MGRS
Paste in 42S XD 45583 55343
Press Enter
coordinates are located
Change the Display Coordinate to DMS
Paste in 42S XD 45583 55343 or 42SXD4558355343
Press Enter
Invalid Coordinate message is displayed

Could not get the same behavior from other coordinate types

Change the Display Coordinate to MGRS
Paste in 42S XD 45583 55343
Press Enter
coordinate is located
Paste in 34.82997N 70.592043E
Press Enter
coordinate is located
Change the Display Coordinate to DD
Paste in 34 49 47.89N 070 35 31.35E
coordinate is located
Change the Display Coordinate to DMS
Paste in 34.82997N 70.592043E
coordinate is located
Paste in 42S XD 45583 55343
Press Enter
Invalid Coordinate message is displayed

@lfunkhouser


RuntimeError: derivative for dequantize is not implemented

Hi, first of all, thanks for creating this package. This is awesome, so much better than the one pytorch officially offers.
I want to try QAT, but I'm getting the following error:
[rank3]: Traceback (most recent call last):
[rank3]:   File "/home/euku/aoboResearch/long_range_GNN/pointnet/cleaned_pointNET/Quantization.py", line 171, in <module>
[rank3]:     loss.backward()
[rank3]:   File "/home/euku/stable/lib/python3.9/site-packages/torch/_tensor.py", line 525, in backward
[rank3]:     torch.autograd.backward(
[rank3]:   File "/home/euku/stable/lib/python3.9/site-packages/torch/autograd/__init__.py", line 267, in backward
[rank3]:     _engine_run_backward(
[rank3]:   File "/home/euku/stable/lib/python3.9/site-packages/torch/autograd/graph.py", line 744, in _engine_run_backward
[rank3]:     return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
[rank3]: RuntimeError: derivative for dequantize is not implemented

My code looks like this:
from optimum.quanto import qint8
from optimum.quanto.calibrate import Calibration
from optimum.quanto.quantize import quantize, freeze
quantize(model, weights=qint8, activations=qint8)
with torch.no_grad():
    with Calibration(momentum=0.9):
        for batch in train_loader:
            X, y = batch
            model(X)
if args.QAT:
    model.train()
    current_lr = optimizer.param_groups[0]['lr']
    print(current_lr)
    for epoch, batch in enumerate(train_loader):
        optimizer.zero_grad()
        X, y = batch
        out = model(X).dequantize()

        ## loss
        xyz_loss = F.mse_loss(out[:, :-1], y[:, :-1])
        energy_loss = args.scale_energy_loss * F.l1_loss(out[:, -1], y[:, -1])
        loss = xyz_loss + energy_loss

        ## gradient
        loss.backward()
        optimizer.step()

freeze(model)

I think your model output is already dequantized, and thus a regular torch.Tensor.
As a consequence:

when calling dequantize(), you actually call the pytorch no-op for non-quantized tensors,
you add the no-op to the graph but it does not implement backward() (which explains the message about no derivative being defined).
If you look at the MNIST example, there is an extra check before calling dequantize:

if isinstance(output, QTensor):
    output = output.dequantize()

Note that I think this is now useless in the MNIST example, as when the loss for that QTensor is evaluated it should be dequantized implicitly.
Aniyway, I think it will work if you remove the call to dequantize.

Oh Thanks! Your explanation makes total sense and getting rid of dequantize fixed the problem.


Cannot override json response of authenticate_user!

Hi,
I am making a REST API server. I have overridden all devise_token_auth resposes.
The only one I can't override is from authenticate_user!
Looking the code, I saw that this function come from devise_token_auth/lib/devise_token_auth/controllers/helpers.rb (Line 32) and the response is did direcly from that code.
Is there any way to override this?
Thanks

Hi, any progress on this? thanks

+1

+1

You can overwrite this method for selected resource in controller that includes DeviseTokenAuth::Concerns::SetUserByToken.
For example:
class ApplicationController < ActionController::API
  include DeviseTokenAuth::Concerns::SetUserByToken

  private

  def authenticate_user!
    unless current_user
      return render json: {
        errors: ["You shall not pass!"]
      }, status: 401
    end
  end
end

nice! that do the work
Many thanks :)


Update DoT Create Object with new paths

CreateObject should:

send POST requests to create objects that don't have a UID
send PUT requests for objects that do have UIDs
return a string to put in SuppMsg that concats all the returned UID strings with ','s in between (commas)

Duplicate of #173


Avoid resizing large struct arrays

Before:
 Method |     Mean |    Error |   StdDev |  Op/s |    Gen 0 |    Gen 1 |    Gen 2 | Allocated |
------- |---------:|---------:|---------:|------:|---------:|---------:|---------:|----------:|
    Dfa | 337.8 ms | 4.650 ms | 4.122 ms | 2.960 | 812.5000 | 750.0000 | 687.5000 | 138.66 MB |

After:
 Method |     Mean |    Error |   StdDev |  Op/s |    Gen 0 |    Gen 1 |    Gen 2 | Allocated |
------- |---------:|---------:|---------:|------:|---------:|---------:|---------:|----------:|
    Dfa | 324.0 ms | 6.375 ms | 14.52 ms | 3.087 | 562.5000 | 500.0000 | 375.0000 | 130.02 MB |

Not related to this PR, but:

There's no need to keep _policies around as it's only used in the .ctor.
Would it be possible to receive policies "pre-sorted" instead of having to sort them each time a DfaMatcherBuilder is instantiated?
Would it be better to construct the _nodeBuilders and _comparer arrays (or use Lists instead) manually instead of iterating over the policies twice (using Linq)?
This allows us to eliminate the ToArray() on the ordered policies.

If you want, I'll test whether this actually reduces allocations and/or improves performance later this week.


Change request to Friends list

Description of a Change request:
Friends list view should be changed.
Reason for Change:
It is inconvenient that friends’ cards are not separated by their borders when the system shows more than one friend in the list.
Acceptance Criteria
When more than one friend card is shown to me, then those friends’ cards should be visually separated by their cards borders.
The user should be able to use the infinite scroll to view all friends' cards.
Each friend's card should also contain the total number of person's friends.
User is able to search for a friend by typing in a search field:

One of the following information:
a. friend’s name;
b. username.
User should type a minimum 1 character and a maximum 30 characters (alphabetic characters, a dot, and a space character) and press “Enter”;
a. system is able to search results from 1 character typed;

Search results are represented by a person's card which contains the following attributes:
a. avatar
b. name
c. personal rate
d. city
e. all friends
f. mutual friends

By clicking on the “friend’s card” the user is redirected to the friend’s page.
When system didn’t find any results user sees a message “We didn't find any results matching this search”.

Impact of Change
All stories related to the Friend List feature are to be affected.
Labels to be added
[change request: UI improvement]
Epic: “My habit” #938
As-Is state

Current state

https://www.figma.com/file/092AhCGadc1Hq8nAysxmYB/ITA-Greencity-%D0%A3%D0%91%D0%A1?node-id=12201%3A3


Fix future Apple toolchain build failure

More detail at https://github.com/firebase/firebase-ios-sdk/pull/7563

@yulin-liang

Sorry this slipped through the cracks. If you could reapply the change to latest master, I'll merge it. There's now one other condition added to avoid the redefinition of iovec in this file.

@drfloob Thanks and done.

Thanks Paul! I'll merge this. Regarding the test failures, the python test is a simple timeout, which is not uncommon. The kokoro build failure was a manual run that I set up incorrectly, and can be disregarded.


@Options should be applied before task is configured

currently it very easy to add CLI options to a task, but very difficult to use them to affect the task configuration.
Expected Behavior
the code in the user's gradle scripts should never see different values for the @Options.
Current Behavior
the task configuration code sees the @Option default values before they are set from the command line, while the task execution code sees the values after having been set.
Context
fixing this would (among other things) make it possible for a task to dependOn different stuff according to the command line.
Steps to Reproduce
my fork of the gradle-issue-reproducer: https://github.com/jeremy-im/gradle-task-option
Your Environment
(this is specified by the reproducer action, which I did not change)

This is expected. You can only read the task options during execution time, not at configuration time. You'd need to use something like a project property to affect project configuration.

if this is expected, then it should be mentioned prominently in the documentation:
https://docs.gradle.org/current/userguide/custom_tasks.html#limitations


Update README.md with Acknowlegements and Policies Section

Added sections to comply with CMS Open Source Policy and Repo-scaffolder standards for README.me.

LGTM


Add multilanguage sexes

Add support for sexes (Male, Female) in multiple langauges.
example in Poland: (Mężczyzna, Kobieta)

hey @cieslarmichal , could you please assign this issue to me? Thank you

Hello! sure, assigned, please give a star to the repository to make this project more popular 🔥


Added timeline events for post frame callbacks

Before this change, long-running post-frame callbacks wouldn't show up in the timeline at all.  This adds a timeline event for post-frame callbacks, with a debug flag that will add timeline events for each individual callback.
#testexempt -- we have no way to test calls to the timeline.
Pre-launch Checklist

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[x] I read the [Tree Hygiene] wiki page, which explains my responsibilities.
[x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
[x] I signed the [CLA].
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is [test-exempt].
[x] All existing and new tests are passing.

Assets are added in https://github.com/flutter/assets-for-api-docs/pull/227

Test failure looks like https://github.com/flutter/flutter/issues/136329


Workaround for NoMethodError by specifying selenium-webdriver version

It looks like updated versions of selenium-webdriver don't have the assume_untrusted_certificate_issuer method, leading to errors like the following when trying to run spiders from the openregs repo, which currently uses the feature/add-cuprite-driver branch.
From the openregs Gemfile:
gem "kimurai", git: "https://github.com/kindsys/kimuraframework", branch: "feature/add-cuprite-driver"
Error when trying to run a spider, e.g. be thor spider:crawl NM-STAT
1: from /Users/gracexu/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/capybara-3.35.3/lib/capybara/session.rb:104:in `driver'
/Users/gracexu/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/bundler/gems/kimuraframework-e1994babff19/lib/kimurai/browser_builder/selenium_firefox_builder.rb:71:in `block in build': undefined method `assume_untrusted_certificate_issuer=' for #<Selenium::WebDriver::Firefox::Profile:0x0000000103b32f50> (NoMethodError)

Specifying the 3.142.7 version of selenium-webdriver as in this commit resolves the issue (you will need to delete your Gemfile.lock in the openregs repo and re-run bundle install and/or gem pristine --all).

hmm, we'll have to remember to come back around and either fix kimurai to not rely on that method or to bump the selenium-webdriver when they fix it. should add a comment on the dependency line so we know why it's locked to such a specific version. you should also be able to specify and restrict the selenium-webdriver version in the openregs Gemfile, which is a bit more visible than kimuraiframework.

Makes sense, I added a comment for now -- thanks Stan!


QNAP armv7 docker after 2022.6.7

Please could you check this ? https://github.com/home-assistant/core/issues/86589

No need to create duplicate issues.
Closing this one.
../Frenck


depthai::initialize() not thread or exception safe; recommend using singleton

depthai::initialize() does not handle simultaneous threads attempting to initialize() with blocking, and does not handle scenarios of exceptions. Easy to fix (see below)
Setup

all OS
all compilers
depthai-core at main

Repro
By code review.
https://github.com/luxonis/depthai-core/blob/8ab5bf954e3b65f6ec0933e372777aac10b81900/src/utility/Initialization.cpp#L48-L55
Imagine multiple thread running on a multicore CPU which can concurrently run at the same time.
Scenario 1 - thread unsafe

time index
thread 1
thread 2

1
initialize()
initialize()

2

std::atomic initialized{false}

3
initialized.exchange(true) returns false
initialized.exchange(true) returns true, therefore returns back to caller

4
OS chooses to pause this thread
💥other code runs and access/uses the variables and structs which are not   yet initialized

5
OS chooses to pause this thread

The existing code does not distinguish between start/completed initialization. It also does not block other threads while initialization is in progress.
Scenario 2 - exceptions

time index
thread 1
thread 2

1
initialize()
initialize()

2
std::atomic initialized{false}

3
initialized.exchange(true) returns false
initialized.exchange(true) returns true, therefore returns back to caller

4
…
…

5
std::make_uniquebackward::SignalHandling() throws an exception   -or- any code in initialize() throws
…

6

💥other code runs and access/uses the variables and structs which are not   yet initialized

The existing code does not distinguish between start/completed initialization. If something throws during initialization, then all threads are locked out and can not try to initialize again.
Expected
Thread safe, exception safe.
Threads are blocked until initialization successfully completes.
Fix
This is easily fixed with a singleton approach. Use a normal static const bool in the function and put all the code in a lambda.
This guarantees threads will block while another thread is attempting to initialize. It allows any thread to attempt init again. And it handles exceptions because c++ guarantees that if an exception occurs in such a local static var initialization, then that init doesn't occur and therefore another thread can attempt again.
For example pseudocode...
bool initialize(std::string additionalInfo, bool installSignalHandler) {
    // singleton for checking whether depthai was already initialized
    static const bool initialized = [&]() {
#ifdef DEPTHAI_ENABLE_BACKWARD
        ...
        return true;
    }();
    return initialized;
}

Good catch @diablodale - this should've been a mutex + bool instead to block other callers in the mean time.
Interesting shorthand approach on the lambda singleton.
Any differences compared to a mutex+bool vs the lambda singleton?
And if you'd like, please open a PR for it, either approach seems fine. (otherwise will address by the end of the week)

I have read the meyer single approach (these are based on c++11 "magic" statics) is the best choice. The C++ runtime and compiler manage the "magic" locks internally allowing for widespread optimization since it is a native language feature. When the same features of a meyer singleton are replicated with manual C++ code by the dev, there are less opportunities for these optimizations and most crucially more opportunity for dev mistakes/poor design.
This has been my learning for years. Here is one article which is on topic https://www.modernescpp.com/index.php/thread-safe-initialization-of-a-singleton showing the Meyer Singleton is almost 3 magnitudes faster than a mutex+lock_guard

I'll push a PR. I applied my fix and it passed all 60/61 of the depthai-core test suite. The one test fails due to my OAK-D not having RGBD cali data in EEPROM.
[ctest]
[ctest] 98% tests passed, 1 tests failed out of 61
[ctest]
[ctest] Total Test time (real) = 1279.47 sec
[ctest]
[ctest] The following tests FAILED:
[ctest] 	 51 - rgb_depth_aligned (Failed)
[ctest] Errors while running CTest
[ctest] CTest finished with return code 8

(Merge defaulted to squash - seems like that's why it didn't close automatically. Closing)


SHIP-0004: Build Environment Variables

Changes

Submitter Checklist

[x] Includes tests if functionality changed/was added
[ ] Includes docs if changes are user-facing
[ ] Set a kind label on this PR
[ ] Release notes block has been filled in, or marked NONE

See the contributor guide
for details on coding conventions, github and prow interactions, and the code review process.
Release Notes

/hold
depends on https://github.com/shipwright-io/build/pull/817 and removing line from go.mod

Changes
Submitter Checklist
* [x]  Includes tests if functionality changed/was added

* [ ]  Includes docs if changes are user-facing

* [ ]  [Set a kind label on this PR](https://prow.k8s.io/command-help#kind)

* [ ]  Release notes block has been filled in, or marked NONE

See the contributor guide for details on coding conventions, github and prow interactions, and the code review process.
Release Notes

release note formattting is not quite right ... nothing is rendoring underneath the heading as I currently view it

otherwise, since I see no one assigned, am I correct @coreydaley that in addition to the hold you have here, this is still a work in progress item, and we should hold off on reviewing?
Or should I add this to the queue and dive in?
thanks

@gabemontero

I have updated the Release Notes section.
Yes, you can go ahead and do a review, keeping in mind the comments that you left on shipwright-io/build and that we are waiting on @adambkaplan to make a decision in regards to the afformentioned.

These tests won't pass until https://github.com/shipwright-io/build/pull/817 merges

/approve

/hold
just in case, as we need to vendor in the associated build PR

@gabemontero
* I have updated the Release Notes section.

* Yes, you can go ahead and do a review, keeping in mind the comments that you left on shipwright-io/build and that we are waiting on @adambkaplan to make a decision in regards to the afformentioned.

some cross referencing with  https://github.com/shipwright-io/build/pull/817#issuecomment-930221460 and https://github.com/shipwright-io/build/pull/817#issuecomment-930228329
summary

@coreydaley thinks he'll be able to add all the k8s corev1.envvar *valuefrom related doc and tests, so his change will continue to use that k8s type
most likely this cli pr will not take on creating env vars from those supplied sources ... we'll track that as a potential add on based on user feedback, with the history that with the analogous support in openshift builds, we never took on cli level support for adding valuefrom env vars

Some comments left on the pull-request, and I'm having issues with the Go modules changed. As per:
$ go mod vendor
go<EMAIL_ADDRESS>(replaced by github.com/coreydaley/shipwright-io-build@v0.0.0-20210927153915-3175d47a83e4): version "v0.0.0-20210927153915-3175d47a83e4" invalid: unknown revision 3175d47a83e4
go: downloading github.com/coreydaley/shipwright-io-build v0.0.0-20210927153915-3175d47a83e4
go: downloading github.com/spf13/cobra v1.2.1
go: downloading k8s.io/apimachinery v0.20.6
go: downloading k8s.io/client-go v0.20.2
go: downloading k8s.io/api v0.20.6
go: downloading github.com/onsi/gomega v1.10.3
go: downloading k8s.io/cli-runtime v0.20.2
go: downloading github.com/texttheater/golang-levenshtein/levenshtein v0.0.0-20200805054039-cae8b0eaed6c
go: downloading k8s.io/utils v0.0.0-20210111153108-fddb29f9d009
go<EMAIL_ADDRESS>(replaced by github.com/coreydaley/shipwright-io-build@v0.0.0-20210927153915-3175d47a83e4): version "v0.0.0-20210927153915-3175d47a83e4" invalid: unknown revision 3175d47a83e4

Same with go mod tidy, for the record.

Correct, since I am pulling in source from my own branch until my other api changes merge in shipwright-io/build.

so @coreydaley do you plan to add any valueFrom secret/configmap related options on the build/buildrun creates you added ... i.e. a --valueFromSecret= sort of thing

No, @adambkaplan and I discussed it yesterday. I also don't think that oc ever added that capability (probably due to complexity)

so @coreydaley do you plan to add any valueFrom secret/configmap related options on the build/buildrun creates you added ... i.e. a --valueFromSecret= sort of thing

No, @adambkaplan and I discussed it yesterday. I also don't think that oc ever added that capability (probably due to complexity)

I'm fine waiting and seeing if users ask for it ... as noted OCP / oc users either never did, or not enough that we bothered to add it
thanks

/hold cancel