less-than-failed-output.eo:28-31: To add several tests...

The puzzle 41-0f42b54c from #41 has to be resolved:
https://github.com/objectionary/eo-hamcrest/blob/b27a1fb3a4550ea5dc1880f988a9d4ffe0f977e6/src/test/eo/org/eolang/hamcrest/failed-output/less-than-failed-output.eo#L28-L31
The puzzle was created by graur on 14-Sep-22.
Estimate: 30 minutes,  role: DEV.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

@includealex please, take this task

The puzzle 41-0f42b54c has disappeared from the source code, that's why I closed this issue. //cc @graur


Dataset is never fully read when caching to disk

Click to expand!
Issue Type
Bug
Have you reproduced the bug with TF nightly?
No
Source
binary
Tensorflow Version
v2.12.0-rc1-12-g0db597d0d75 2.12.0
Custom Code
No
OS Platform and Distribution
Windows 11
Mobile device
No response
Python version
Python 3.10.7
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
When caching a dataset to disk with dataset.cache("some_folder"), the dataset is never fully cached according to the log message. I have reproduced this consistently in my application code and in the MWE below.
When testing this MWE, the message does not appear on subsequent runs unless I delete the cache files, which makes me think Tensorflow is actually able to them, and this might be a logging issue.
When caching to memory (dataset.cache()), no log message is generated.
Standalone code to reproduce the issue
import tensorflow as tf

dataset = tf.data.Dataset.range(1_000_000, dtype=tf.float32)
dataset=dataset.cache("cached")
for _ in dataset:
    pass # Exhaust the dataset to force it to cache

Relevant log output
2023-03-30 12:15:57.441876: W tensorflow/core/kernels/data/cache_dataset_ops.cc:296] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset  will be discarded. This can happen if you have an input pipeline similar to `dataset.cache().take(k).repeat()`. You should use `dataset.take(k).cache().repeat()` instead.

The behavior you are observing is expected, and it is not a logging issue. When we cache a dataset to disk with dataset.cache("some_folder"), TensorFlow only partially caches the dataset to disk. The caching mechanism works by storing the elements of the dataset in memory as they are consumed, and then periodically flushing these elements to disk to avoid consuming too much memory.
If you want to ensure that the dataset is fully cached to disk, you can add a call to dataset.prefetch() after the call to dataset.cache(). This will cause TensorFlow to fully cache the dataset to disk before starting to consume it.
import tensorflow as tf

dataset = tf.data.Dataset.range(1_000_000, dtype=tf.float32)
dataset = dataset.cache("cached")
dataset = dataset.prefetch(tf.data.AUTOTUNE)
for _ in dataset:
    pass # Exhaust the dataset to force it to cache

Sorry, even with the addition of prefetch I am still seeing this log message.

Try dataset = dataset.take(5000).cache("cached")  #Use take() before cache(). the take() function is used to create a new dataset that contains only the first 500 elements of the original dataset. The cache() function is then called on this new dataset to ensure that only the first 500 elements are cached. Finally, the dataset is iterated over to force it to cache.
By using take() before cache(), the warning message should be resolved and the dataset should be properly cached. https://github.com/tensorflow/tensorflow/issues/42773

I have not tested this solution, however I will not use it because I want to iterate over the entire dataset and not just part of it. Additionally, I do not use take or any other function that limits the number of elements used from the dataset.

@Thf772 , I would second to the suggestion given by @bhagirath20.
Cache should be used only when the dataset is small and if the dataset fits into your memory.
The existing behavior is expected.
In general, cache should be used after loading and preprocessing the data but before shuffling, repeating, batching, and prefetching so that each instance will only be read and preprocessed once instead of every epoch in your model.


Create new event grid topic and subscription

Relies on this PR in the shared package: https://github.com/Microsoft/vscode-azuretools/pull/156
The portal supports all of these topic types when creating an event subscription, but I intentionally only support 4 in this first PR:

Fixes #4
FIxes #3

ping @StephenWeatherford @nturinski


Feature Request: PubSubHubbub/WebSub support

Given the nature of email, being able to push articles to clients the moment they are received would be convenient. This can be achieved by implementing PubSubHubbub/WebSub to push to clients that are publicly available (i.e. web-based or client/server feed readers). This would probably reduce bandwidth usage too, as frequent refreshes would turn into a single request per new email.
The PubSubHubbub Github organization has lots of good resources on the protocol.

Thanks for the information. That sounds interesting. I’ll look into it…

As well as real-time updates being great for users this could reduce load on the service. I run a small reader but with a couple of subscriptions I end up polling every few minutes. I'm sure for larger reader services this adds up very quickly.
When WebSub is supported my service switches to only polling each feed once a week (plus a WebSub subscription request). So in exchange for a small amount of storage to remember subscriptions you can save a lot of CPU and network load. I don't know if that is a notably concern right now but using less resources is always nice.

This would be an amazing feature to add. I would be more then happy to donate to a bounty source or something similar as well to get this going.

Hi y’all,
I implemented WebSub support as part of 2.0.4.
Thanks for bringing this to my attention—I didn’t even know WebSub was a thing and it was fun to implement 🤗
Enjoy.

@edsimpsons83:

This would be an amazing feature to add. I would be more then happy to donate to a bounty source or something similar as well to get this going.

Oh, that’s very nice of you. Thanks. Here’s how you can support me:
Patreon · PayPal · GitHub Sponsors

Thanks for adding support! I can see the subscription in the FreshRSS UI and logs, but KTN doesn't appear to be pushing new entries. I've tested with other feeds and the reader does seem to be working properly, though it wasn't before I checked :P
Is it working for anyone else using a different feed reader?

I've tested with https://websub.rocks/hub when I was developing the feature and it worked.

Thanks for implementing this. One odd thing that I noticed is that subscription validation takes a while. I've seen 1h, I've seen 5h I've seen over a day (or maybe there was just an outage with no verification sent in that time).
The WebSub specification Section 5.1.2 says "The hub SHOULD perform the verification and validation of intent as soon as possible". These times don't seem "as soon as possible".
This is just a "SHOULD" but what ends up happening with my reader is that it assumes that the subscription failed for some reason so it creates a new one on the next feed check. Then it ends up getting a handful of subscription validations in a row.
I haven't actually seen any entries posted yet, but that seems to be just that the feeds my readers are using are quiet. I'm adding my own feed and I'll do a test as soon as the subscription validation occurs.

Hi @kevincox,
Thanks for reporting.
The issue that there are too many verifications/updates to send and the server isn’t keeping up.
I’m surprised by how popular WebSub turned out to be. But that’s a great problem to have 🤗
I’ll investigate what I can do about it…

I'd just like to pop in and say that once the subscription is eventually confirmed it does work properly. I sent a message to my test feed and got a pushed feed update quite quickly. 🎉

Seems like @kevincox is right, the confirmation does take some time but afterwards websub works great! I also sent an email to my test feed and it arrived in my feed reader within seconds.

I tweaked some settings and now the background job works seem to be managing the load. Please let me know if things are speedy for y’all now.

As of about 2024-06-23 16:00 all of the subscriptions have been confirmed within a second. But my service only has a few feeds so not a huge sample point. I can check back in a week after they refresh a few times (as they seem to be 24h leases) and confirm that it is stable.


Generation failure involving JsonProperty and JsonConstructor

Original issue: https://github.com/Azure/autorest.typescript/issues/235
Swashbuckle.AspNetCore version: 2.4.0
Issue
I tried to use Microsoft's Autorest to generate the client code for one of my web services where the Swagger definition is generated by this tool. However, since the document is incoherent, Autorest fails and shows this error:
Property 'foo' in Model 'BarModel' is marked readOnly and is also required. This is not allowed.
No errors is shown when we navigate to the web server's Swagger UI route, but it is faulty when I copy the generated Swagger file in Swagger's online editor:

BarModel:
    required:
      - foo
    type: object
        properties:
           foo:
           $ref: '#/definitions/FooModel'
           readOnly: true

The readonly: true property might be wrong since the foo property is set in the constructor decorated with the JsonConstructor attribute, as shown here:
[JsonObject(MemberSerialization.OptIn)]
public sealed class BarModel
{
        #region Properties

        [JsonProperty(PropertyName = "foo", Required = Required.Always)]
        public FooModel Foo { get; }

        // Other properties omitted

       #endregion

        #region Constructors

        [JsonConstructor]
        public AgentModel(FooModel foo)
        {
            Foo = foo;
            // Other constructor parameters omitted
        }

        #endregion
}

Should it really be read-only in this case since it will be set when the JSON is deserialized in the C# object inside the constructor? It will become read-only after the object is constructed, but not before. If we remove the Required = Required.Always in the JsonProperty attribute, the value could be nullable, which is not the case.

How are you using this model - is it being passed as a body parameter (i.e. [FromBody])?
If so, if you submit a JSON request, including a value for the foo field, the serializer WILL NOT populate it as the setter is private. Therefore, from an API perspective - it's readonly

Ahhh - I just saw your use of JsonConstructor for deserialization as opposed to public setters. Swashbuckle doesn't currently handle that approach. Is there any reason why you wouldn't just switch to a public setter?

Is support for the getter-only+JsonConstructor pattern on the roadmap anywhere? Or something you'd be open to contributions for?

@domaindrivendev should we still see those errors when we navigate to the Swagger endpoint like we would see in Swagger's online editor? I could file another issue for this separately.

@RikkiGibson - nothing on the roadmap right now but always open to contributions.
One thing to note if you do take a look at this, Swashbuckle delegates to the JSON.NET ContractResolver for most of the schema metadata. This is by design because it keeps API descriptions closely in line with "actual" serialization behavior.
Ideally, there would be a single field on JsonProperty (provided by the ContractResolver) that automatically takes read-only+JsonConstructor usage into account. It seems to me that JsonProperty.Writable should be doing this but clearly doesn't right now. So, before refactoring the Swashbuckle code, I would ask that you investigate this avenue a little further - perhaps the best place for the change is in here rather than Swashbuckle?

Good idea :) I went ahead and created JamesNK/Newtonsoft.Json#1800. I'm sure it'll be valuable to hear from a maintainer on JSON.NET about the intended meaning of the property.
My suspicion on reading the code is that the burden is going to wind up on the Swashbuckle side-- in addition to checking jsonContract.Properties, it will be necessary to check jsonContract.CreatorParameters.

@felpel @RikkiGibson @kzryzstof - I've finally gotten around to adding this functionality and it should be available as of 5.2.1. One thing to note - it will only work if you're using the Newtonsoft serializer as opposed to System.Text.Json as the latter doesn't support this feature yet either. Furthermore, from v5.0.0 of Swashbuckle and onwards, you need to explicitly tell it to honor Newtonsoft behavior rather than STJ behavior. See this readme section for full details.


Error Crypto.PublicKey

Hello,
I keep getting the following error whenever I try to run the script
Traceback (most recent call last):
  File "./RsaCtfTool.py", line 17, in <module>
    from Crypto.PublicKey import RSA
ImportError: No module named Crypto.PublicKey

I've installed all dependencies like asked in the requirements.txt
Thank you,

is pycrypto correctly installed ?

same issue even with pycrypto installed

got the same issue, this is the output from pip install -r "requirements.txt":
Requirement already satisfied: PyCrypto in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 1)) (2.6.1)
Requirement already satisfied: GMPY2 in /usr/local/lib/python2.7/dist-packages/gmpy2-2.1.0a3.dev0-py2.7-linux-x86_64.egg (from -r requirements.txt (line 2)) (2.1.0a3.dev0)
Requirement already satisfied: SymPy in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 3)) (1.1.1)
Requirement already satisfied: requests in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 4)) (2.18.4)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python2.7/dist-packages (from SymPy->-r requirements.txt (line 3)) (1.0.0)

Thx ivansena35 !


Do you think implement to flutter web?

Support for web can be added with minimal JS interop.
Web provides the web share API which can be accessed with Navigator.share() method.
Please consider implementing it @KasemJaffer

@gdelgado11 , implementing this feature for the web may not be possible as you need an app to receive an intent. All webpages will open in the browser by default

instead of passing as intents, on opening the app, all query params after the host could be passed to the app's receiver listener, So the existing logic written for mobile could work for Web,


Add Amrutha to contributors list

:warning: This is a Pull Request Template :warning:
This will help you complete a pull request. Please follow the instructions below. And also,make sure that you're bound to code of conduct
Submission Checklist :heavy_check_mark:

[ x] I read the CONTRIBUTIONS.md .
[ x] I've forked the repository.
[ x] I've created a branch in the following format  MY- USERNAME - SHORT-NAME-OF-SUBMISSION.
[x ] I've read the Hacktoberfest's official guidelines regarding making pull request.
[x ] I've added my name into CONTRIBUTORS.md
[ x] I've filled in this form.

actually I have kept pull request for hacktoberfest-2018

Could you make a contribution other than just adding your name?


'edgetpu_compiler' does not work (Model not quantized)

I tried to use edgetpu_compiler to convert my quantized model. (with tensorflow 2.4.0)
I followed sample code Retrain a classification model for Edge TPU using post-training quantization (with TF2) and Post-training integer quantization
I quantized following model:
model = tf.keras.Sequential([
  tf.keras.layers.InputLayer(input_shape=(28, 28)),
  tf.keras.layers.Reshape(target_shape=(28, 28, 1)),
  tf.keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation='relu'),
  tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(10)
])

But, when I tried command line edgetpu_compiler my_mnist_quant.tflite I got following message.

Edge TPU Compiler version 14.1.317412892
Invalid model: my_mnist_quant.tflite
Model not quantized

However, when I commented two layers Conv2D and MaxPooling2D, it worked well
model = tf.keras.Sequential([
  tf.keras.layers.InputLayer(input_shape=(28, 28)),
  tf.keras.layers.Reshape(target_shape=(28, 28, 1)),
  #tf.keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation='relu'),
  #tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(10)
])

Edge TPU Compiler version 14.1.317412892

Model compiled successfully in 44 ms.

Input model: my_mnist_quant.tflite
Input size: 9.39KiB
Output model: my_mnist_quant_edgetpu.tflite
Output size: 56.50KiB
On-chip memory used for caching model parameters: 49.25KiB
On-chip memory remaining for caching model parameters: 7.81MiB
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 1
Total number of operations: 4
Operation log: my_mnist_quant_edgetpu.log
See the operation log file for individual operation details.

I don't know why it is not working with Conv2D and MaxPooling2D.
(Actually, it is also not working when only one of them are commented)
My full code is below (tensorflow 2.4.0):
! pip uninstall -y tensorflow
! pip install tf-nightly
import tensorflow as tf
import os
import numpy as np
import matplotlib.pyplot as plt

# Load MNIST dataset
mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

# Normalize the input image so that each pixel value is between 0 to 1.
train_images = train_images / 255.0
test_images = test_images / 255.0

# Define the model architecture.
model = tf.keras.Sequential([
  tf.keras.layers.InputLayer(input_shape=(28, 28)),
  tf.keras.layers.Reshape(target_shape=(28, 28, 1)),
  tf.keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation='relu'),
  tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(10)
])

# Train the digit classification model
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

model.fit(
  train_images,
  train_labels,
  epochs=1,
  validation_data=(test_images, test_labels)
)

def representative_data_gen():
    mnist_train, _ = tf.keras.datasets.mnist.load_data()
    images = tf.cast(mnist_train[0], tf.float32) / 255.0
    mnist_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
    for input_value in mnist_ds.take(100):
    # Model has only one input so each data point has one element.
        yield [input_value]

converter = tf.lite.TFLiteConverter.from_keras_model(model)
# This enables quantization
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.int8]
# This ensures that if any ops can't be quantized, the converter throws an error
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
# These set the input and output tensors to uint8 (added in r2.3)
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
# And this sets the representative dataset so we can quantize the activations
converter.representative_dataset = representative_data_gen
tflite_model = converter.convert()

with open('my_mnist_quant.tflite', 'wb') as f:
    f.write(tflite_model)

! edgetpu_compiler my_mnist_quant.tflite

@kimwj94 could you downgrade to tf2.2 for now? Currently release of the compiler isn't compatible with tf2.4 or nightly yet.

@Namburger Thank you! it works.

Hello,
I also get Model not quantized when trying to quantize a SavedModel, but using experimental_new_converter = False is causing me an Aborted (core dumped):
2020-07-28 13:56:44.148771: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2020-07-28 13:56:44.148797: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2020-07-28 13:56:46.174485: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-07-28 13:56:46.174506: W tensorflow/stream_executor/cuda/cuda_driver.cc:312] failed call to cuInit: UNKNOWN ERROR (303)
2020-07-28 13:56:46.174524: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] no NVIDIA GPU device is present: /dev/nvidia0 does not exist
2020-07-28 13:56:46.174727: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-07-28 13:56:46.198563: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency:<PHONE_NUMBER> Hz
2020-07-28 13:56:46.199123: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1238d320 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-28 13:56:46.199142: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-07-28 13:56:46.229969: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:118] None of the MLIR optimization passes are enabled (registered 1)
WARNING:absl:Please consider switching to the new converter by setting experimental_new_converter=True. The old converter (TOCO) is deprecated.
Traceback (most recent call last):
  File "./converter.py", line 50, in <module>
    tflite_model = converter.convert()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/lite.py", line 752, in convert
    output_tensors)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/lite.py", line 669, in convert
    **converter_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/convert.py", line 574, in toco_convert_impl
    enable_mlir_converter=enable_mlir_converter)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/convert.py", line 279, in toco_convert_protos
    raise ConverterError("See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: See console for info.
2020-07-28 13:56:48.602468: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2020-07-28 13:56:48.602490: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2020-07-28 13:56:49.827525: F tensorflow/lite/toco/import_tensorflow.cc:2707] Check failed: !absl::EndsWith(specified_input_array.name(), ":0") Unsupported explicit zero output index: input:0
Fatal Python error: Aborted

Current thread 0x00007f30c55c6740 (most recent call first):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/toco/python/toco_from_protos.py", line 56 in execute
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250 in _run_main
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299 in run
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 40 in run
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/toco/python/toco_from_protos.py", line 93 in main
  File "/usr/local/bin/toco_from_protos", line 8 in <module>
Aborted (core dumped)

Any idea what could go wrong here? Could it be that the problem is in the model itself?
For completeness, my converter script is pretty basic:
#!/usr/bin/env python3
import tensorflow as tf
import os
import sys
import cv2
import itertools as it
import numpy as np
import pathlib

if len(sys.argv) == 1:
    print("./converter_tf2.py [${PWD}/model_dir] [output filename: default 'model.tflite'] [list_masked.txt list_unmasked.txt]")
    sys.exit(1)
#print(list(zip(range(len(sys.argv)), sys.argv)))

def get_gen_from_file(txt_file):
    with open(txt_file, "r") as f:
        image_paths = [line.strip() for line in f.readlines()]

    for image_path in image_paths:
        image = image = cv2.imread(image_path)
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
        image = cv2.resize(image, (160, 160))
        image = np.expand_dims(image, 0)
        image = image.astype(np.float32)
        image -= image.mean()
        image /= image.std()
        yield image

def dataset_gen(unmasked_txt, masked_txt):
    def f():
        i = 0
        for t in it.chain(get_gen_from_file(unmasked_txt), get_gen_from_file(masked_txt)):
            if t is not None:
                print("Loaded image", i)
                i += 1
                yield [t]
    return f

model_dir = sys.argv[1]

converter = tf.lite.TFLiteConverter.from_saved_model(model_dir)
converter.experimental_new_converter = False
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.target_spec.supported_types = [tf.int8]
converter.inference_input_type = tf.int8  # or tf.int8
converter.inference_output_type = tf.int8  # or tf.int8
if len(sys.argv) > 3:
    converter.representative_dataset = dataset_gen(sys.argv[3], sys.argv[4])
tflite_model = converter.convert()
tflite_name = pathlib.Path(os.getcwd())
tflite_name = tflite_name / ('model.tflite' if len(sys.argv) < 3 else sys.argv[2])
tflite_name.write_bytes(tflite_model)

@InCogNiTo124 It looks like a converter bug, if it's getting a core dump. Unfortunately that's out of our hand :(
This would be the best place to report this bug

@Namburger is the only solution in such a situation to wait for next release of edgetpu_compiler? tf 2.2 does not properly allow int8 quantization. 2.3 has fixed that 2.4 nightlies allow converting more models.

@gan3sh500 can you try turning off MLIR converter with tf2.3 and see if it works?

@Namburger I tried but TOCO was failing and hard to understand the source of the error. Do you know when edgetpu_compiler with support for newer tf will be out?

Sorry, we're still working on supports for MLIR so there isn't a committed date for full compiler compatibility yet :/

Any update on MLIR support?

@msokoloff1 should works now, are you still having issue?

Actually, it looks like I am having an issue with tf 2.4.0. I want to convert a model trained with QAT. I have only been able to run full integer quantization on a QAT model in tf 2.4.0. But then when I go to compile for the edge tpu it I am getting the same issue as above.
Edge TPU Compiler version 15.0.340273435
Invalid model: model_out.tflite
Model not quantized

I installed TF-nightly(2.5.0-dev20201227) to a model with QAT, and used the following configure to convert the model to tflite
converter.optimizations = [tf.lite.Optimize.DEFAULT]
quantized_tflite_model = converter.convert()
converter.experimental_new_converter = False
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8  # or tf.uint8
converter.inference_output_type = tf.int8  # or tf.uint8

I got some warning while converting. The message showed MLIR was turned off.
2020-12-30 02:22:00.056289: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:933] Optimization results for grappler item: graph_to_optimize
function_optimizer: function_optimizer did nothing. time = 0.065ms.
function_optimizer: function_optimizer did nothing. time = 0.001ms.

2020-12-30 02:22:01.718892: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:332] Ignored output_format.
2020-12-30 02:22:01.718972: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:335] Ignored drop_control_dependency.
2020-12-30 02:22:01.790998: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:194] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.`

But I still got the error while compiling
root@tf2:~/git/train_face# edgetpu_compiler -s models/20201228_quantized-efficientnet-lite2.tflite
Edge TPU Compiler version 15.0.340273435
Invalid model: models/20201228_quantized-efficientnet-lite2.tflite
Model not quantized


post the data

i work now with the code and it is sooo beutiful
i understand how to change every thing
but
when i try to post the tags in form the input has no thing like this
<form method=post>
<input name="test" id="test">
<input type=submit>

........
in the result

the view is
array(test=>)

Hi @amerhendy
Do you have a test page I can see? Or try and set up a JSFiddle at http://jsfiddle.net/
Adam

i resolve it but not complete
i add this line

$('input[name=' + elem.attr("name") + ']').after('
<input type="hidden" value="' +     $(this).children('.SO_result_id').html() + '" name="' + elem.attr("name") + 'result[]"›');

after this:

if (elem_parent.children('#tag' + $(this).children('.SO_result_id').html()).length === 0) {


Visual Studio Code won't launch!

VSCode Version: 1.27
OS Version: 10.13.4

Steps to Reproduce:

Downloaded VSCode-darwin-stable.zip to Downloads folder
Unzipped to same directory
Click-and-dragged VSC app to Applications
Double-clicked on VSC to launch
VSC icon shows up on dock, bounces a few times, then disappears

Does this issue occur when all extensions are disabled?: Not sure
My feeling is that this app was never properly installed.
I checked on Console but couldn't even find any reports on VSC, which may sound very strange.
Any suggestions/advice would be highly appreciated!

Please see if you can find any logged output that could help us find the source of the issue:

run code --verbose from the command line and attach the output
in Code select Help | Toggle Developer Tools and select the Console
attach any output you see in the Console you can find

Thanks!

Hello,
I dragged the Electron file into the terminal and ended up with this message:
[main 4:53:25 PM] Startup error: ,Error: EACCES: permission denied, mkdir '/Users/TeresaLeong 1/.vscode'

Even when I try to open VSC from terminal, I get this message:
Teresas-MacBook-Pro:Applications TeresaLeong$ open -a visual studio code
The files /Applications/studio and /Applications/code do not exist.
But the app is clearly there and installed on my computer!

Sorry I closed the issue by accident yesterday. I'm still unable to open the app.

Hi, same issue here (on fedora 29 beta).
/usr/bin/code : ligne 35 :  5840 Erreur de segmentation  (core dumped)ELECTRON_RUN_AS_NODE=1 "$ELECTRON" "$CLI" "$@"

Please see if you can find any logged output that could help us find the source of the issue:

run code --verbose from the command line and attach the output
in Code select Help | Toggle Developer Tools and select the Console
attach any output you see in the Console you can find

Thanks!

I waited for the newest release to download and tried again. Now it works on my computer. Thanks!

I occasionally have the same problem.
@bpasero here's what I see on the console
> code . --verbose
[main 4:14:42 PM] Sending env to running instance...
[main 4:14:42 PM] Sent env to running instance. Terminating...
[main 4:14:42 PM] Lifecycle#kill()


CriticalTemperatureGauge: New dependencies.

@formicant I've opened a issue in your repo, because the zip of the <IP_ADDRESS> release is corrupted: https://github.com/formicant/CriticalTemperatureGauge/issues/5
Can you try to re-zip it and release it again?

Yep, looks fine now.


add font modifier cell options: bold, italic, underline.

This PR adds cell options for the font modifiers bold, italic, and underline.
tcell was upgraded 1.4.0.
Strikethrough cell option was added as a no-op until tcell is updated to v2.0.0. I figured it wasn't a good idea to upgrade tcell by a whole major version without testing.
Closes #251

I thought I did sign the cla... I'll try it again

@googlebot I signed it!

can you please respond indicating that you are giving me permission to merge this into the master branch?

I give you permission to merge this into the master branch.

Thank you for the permission. We will merge this into the devel branch first and it will go into the next release together with a few minor changes.

Pull Request Test Coverage Report for Build 1852

14 of 44 (31.82%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.2%) to 90.975%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

terminal/termbox/cell_options.go
12
16
75.0%

terminal/termbox/termbox.go
0
10
0.0%

cell/cell.go
0
16
0.0%

Totals

Change from base Build 1843:
-0.2%

Covered Lines:
7873

Relevant Lines:
8654

💛  - Coveralls

Pull Request Test Coverage Report for Build 1852

14 of 44 (31.82%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.2%) to 90.975%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

terminal/termbox/cell_options.go
12
16
75.0%

terminal/termbox/termbox.go
0
10
0.0%

cell/cell.go
0
16
0.0%

Totals

Change from base Build 1843:
-0.2%

Covered Lines:
7873

Relevant Lines:
8654

💛  - Coveralls

@dyc3 please let me know if you are still working on this, or if it is ready for merging. There is still some missing coverage on some of the added code, but I will be happy to add those tests as a follow-up if you prefer that.

I'm currently done working on this. I looked at the missing coverage, and I thought the behavior that wasn't being covered was pretty trivial and wouldn't benefit too much from tests.
Although looking at it again, should the error paths be covered in tests?

I would cover the error paths that we can cover easily. But there certainly are some not worth the work, because they depend on the termbox object being instantiated and we don't have a pre-established way of doing that so far.
I think this is good, I propose to merge it as it is and I can try to iterate on the test coverage a bit more before the merge to master.
Thank you very much for improving Termdash. I will aim towards upgrading tcell to 2.x next and then we can add support for the additional font modifiers. Please feel free to let me know if you would like to help with either of the two.

Agree, lets merge it as-is.
I'd like to help out with both. Feel free to @ me for whatever needs to get done.

Sounds good and will do.


Implement BoneMapper and BoneRenamer in Importer

Implement BoneMapper and BoneRenamer in Importer, as I said in https://github.com/godotengine/godot-proposals/issues/4510.

Contained fixes:

~Fix with #61596 is prerequisite.~ No longer needed.
Fixed a bug where PostImportPlugin::get_internal_import_options() was not properly passing added parameters to PostImportPlugin::internal_process().
Added NodePath::get_concatenated_names(). This is often needed when processing animation track paths.

Preview:
Add a BoneMap as a resource, which generates a map based on a BoneMapProfile.
There are two types of BoneMapProfile: BoneMapProfile, which can be customized by the user, and GDHumanoid, which is a built-in preset that imitates a humanoid. GDHumanoid is a class that extends BoneMapProfile, but all parameters are set to readonly.
https://www.youtube.com/watch?v=dgduRK0xj2M
The buttons on the GUI map editor change color depending on the state of the map elements.

Gray: Bone not set.
Green: Bone is set.
Red: Set bone is conflicted with other elements.

Todo in the future:

Need to do the rest of the https://github.com/godotengine/godot-proposals/issues/4510 tasks (3, 4, 5) as another PR.
Bone mapping all manually is a pain, so an auto-mapping implementation will need to be added future.
It may be necessary to add a validation process to foolproof for the following cases:

Currently, renaming is done even when there is a warning with Red button.
We may need to consider forbidden characters in profile bone name, It will be used in Animation track path and Skeleton bone name.
There may be a conflict between the unmapped bone names and the mapped-renamed bone name.

Since the property path changes depending on whether bool Rename Bones is enabled or disabled, and considering the impact on other PostImportPlugin, one of the following three methods must be taken:

make rename the first in the list and then enforce it
always make it the last in the PostImportPlugin list
change the path in other PostImportPlugin depending on bool Rename Bones

2 is not recommended; it is consistent inside core, but it is difficult to insert an external PostImportPlugin in the middle of a PostImportPlugin list.
1 is worth considering. But, considering that there may be cases where you want to do only T-posing without renaming, it may be a bad way.
3 is a bit redundant, but is the best in view of consistency.

I have two concerns:

Regarding the bone name, it might be better to define GDHips instead of Hips, which would not conflict with outside definitions. Context: https://github.com/godotengine/godot/pull/61596#issuecomment-1164633639
There is no property "parent" in Profile yet. I suppose it could be added later as an enhancement to validation, but I am not sure if that is necessary at this stage

After discussion with @reduz:
Skin will be renamed directly. For BoneAttachment, which may be imported as a glTF Extention, we call the method _notify_skeleton_bones_renamed and rename it in that method.
For example, if you make a GDExtention such as DynamicBone that may be imported from within glTF, it is recommended that you define _notify_skeleton_bones_renamed in the same way as for BoneAttachment::_notify_skeleton_bones_renamed. These will be added later in the documentation along with the usage of BoneMap.

Okay, I modified pointed out part. I think we might going to add parent and basis to Profile as standard pose when we implement T-poser, but at the time of this PR that has not yet been done and implementation is minimal.

@fire I can add parent and basis items, but I think we will have to decide again when we implement RestPoseFixer what value is actually appropriate as a standard.
I remember discussing that value with you before, but I will have to try it out to see if it actually works. After this PR is merged, I will start working on the RestPoseFixer and will implement it in that.

@reduz Okay, I think it is ready.

@TokageItLab
Will there be a video tutorial? I would like to try and feedback once it is merged?

@GeorgeS2019 There is no information beyond the images, even if there was a video.
This is just a renamer and is only the base of the retargeting feature, there are still many things missing such as T-Poser and bone name estimation and etc. so we will add them as plugins and enhancements around this after this is merged.

@TokageItLab Thanks for providing what is yet to come. Very excited about the progress. Good Job!

Thanks!


What to do with the data

I have downloaded the brainData in NIFTI format from IXI and put them in /raid/dongnie/brainData, but it reports an error:
FileNotFoundError: [WinError 3] The system could not find the specified path: '/raid/dongnie/brainData'
Do I need to process these data further?


NIFI-3879: Allow null Avro default values

Avro uses their own class for null values, so a standard check against
null isn't picking them up.

Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

@champagst - was able to verify the issue with your template. Then applied the PR and verified that it addressed the problem. Many thanks again! Merged this to master. Excited to see this getting used already!


GPII-3707: Move keyring from global to infra_region

Rendered docs: https://github.com/mrtyler/gpii-infra/blob/gpii-3707-regionalism/gcp/README.md
Requires https://github.com/gpii-ops/exekube/pull/49, which can be safely merged on its own.

[x] Merged and deployed

I'm finalizing documentation and the rollout plan but I think I'm done with the implementation, hence no WIP label.
Changes
I tried to make targeted commits but there was a hard-to-remove detour where I thought I needed to change the Keyring name based on Region. Anyway, the main themes, ordered roughly by batches of commits:

Small cleanups, including removing Stepan's hack for GPII-3568. I think Stepan is removing that in one of his in-flight branches, but it was causing me trouble so I removed it here as well.
Refactor secrets.rb and its call sites in rake land. Use Keyrings in infra_region instead of global. Add some basic unit tests. I'm not a huge fan of classes but:

This is a classic use case for a class: data bundled with methods for working with that data
Dealing with environment variables in unit tests is kind of annoying (see vars_spec.rb).
I think instance variables (set via constructer parameters) are a more natural fit than environment variables, and they declutter the rake file a bit compared with passing things like project_id in every time.

Add some helpers to facilitate migration from the global keyring, and for moving between regions.

Alternative approach
My main goal was to enable us to run GPII Cloud environments in any region. I have been able to do that in my tests, but the current approach has some downsides:

It requires some extra terraform import steps because KMS Keyrings are indestructible and Keys have an unusual lifecycle

I think moving from region to region will be rare in practice
I've added some rudimentary automation for this, rake import_keyring

Each environment has a single tfstate bucket in Multi-Region US, no matter in which Region the environment really runs

I have considered, but not yet tested, an alternative approach with a tfstate bucket in each region where we might wish to run an environment. This avoids the terraform import steps because tfstate in us-east1 can always point to the keyring in us-east1. I haven't pursued this avenue because:

We need to migrate off of global Keyrings regardless, so something like this PR and its rollout plan will be necessary
tfstate buckets are managed by common, so we'd need to create them ahead of time for each environment X region we want to have available
Multiple tfstate buckets would allow (in fact, require) apply_infra resources (e.g. VPCs, public IPs) in each Region. This is a small waste and a small amount of extra clutter
Multiple tfstate buckets would allow users to accidentally run two environments at once (e.g. one in us-east1, one in us-central1)
DNS is still a singleton. The default behavior with multiple tfstate buckets would be "last one wins", which is not so bad but nonetheless adds a potential source of confusion.

Exported logs
We have a decision to make about existing exported logs for stg and prd:

Delete them - how much value do they have?
Leave them alone - it is theoretically possible to decrypt and read the logs later as long as we are careful not to destroy the global Key used to write them
Re-encrypt them in place - this is the safest, but most expensive option

I think we can use a modified version of rotate_tfstate_key to accomplish this, but I haven't tested it yet.

[x] Decided to keep prd logs

Rollback plan
The basic idea is to undo the rollout by re-importing the global Keyring and Keys and re-encrypting data using the Old Keys. The steps here are for pointing the environment back to the Old Keyring. Other steps may be needed as well, depending on when the rollout was abandoned.

Revert this PR (i.e. go back to Old version of code)

The associated Exekube PR and resulting image may remain

Update gpii-infra to Old version
cd gpii-infra/gcp/live/dev
rake sh
unset GOOGLE_ENCRYPTION_KEY
cd /project/live/$ENV/secret-mgmt
Replace the New Keyring and Keys with the Old Keyring and Keys:

terragrunt state rm module.gcp-secret-mgmt.google_kms_key_ring.key_ring
terragrunt import module.gcp-secret-mgmt.google_kms_key_ring.key_ring projects/$TF_VAR_project_id/locations/global/keyRings/keyring
terragrunt state rm module.gcp-secret-mgmt.google_kms_crypto_key.encryption_keys[0]
terragrunt import module.gcp-secret-mgmt.google_kms_crypto_key.encryption_keys[0] projects/$TF_VAR_project_id/locations/global/keyRings/keyring/cryptoKeys/default
terragrunt state rm module.gcp-secret-mgmt.google_kms_crypto_key.encryption_keys[1]
terragrunt import module.gcp-secret-mgmt.google_kms_crypto_key.encryption_keys[1] projects/$TF_VAR_project_id/locations/global/keyRings/keyring/cryptoKeys/gcp-stackdriver-export

Rollout plan

[ ] Complete migration steps (below) for dev-doe (skipping some steps since dev-doe is an ephemeral cluster), which will be deployed into a new region (us-east4) as a test case
[ ] Notify cloud-developers of required actions (see below)
[ ] Merge this PR, which will eventually run CI for dev-doe.
[ ] Verify everything looks correct. In particular, look out for changes to Keyrings and changes to the tfstate bucket as part of common-stg
[ ] Complete migration steps (below) for stg
[ ] Promote common-prd
[ ] Verify everything looks correct in stg and in plan-all for prd
[ ] Complete migration steps (below) for prd
[ ] Promote prd
[ ] Re-encrypt prd's exported logs (see below)

Migration steps
"Old" refers to the tip of master / the latest deployed infrastructure from before this PR (i.e. with Keyrings and Keys in the "global" Region). "New" refers to code / infrastructure after this PR is merged.

Update to latest version of gpii-infra
cd gpii-infra/gcp/live/dev
rake update_exekube
Replace the Old Keyring and Keys with the New Keyring and Keys:

TF_VAR_decrypt_with_key_from_region=global rake import_keyring

Verify that terragrunt plan doesn't want to make any changes

TF_VAR_decrypt_with_key_from_region=global rake sh and then inside shell:
unset GOOGLE_ENCRYPTION_KEY
cd /project/live/$ENV/secret-mgmt
terragrunt plan
Exit shell

TF_VAR_decrypt_with_key_from_region=global rake rotate_secrets_key
rake rotate_secrets_key"[gcp-stackdriver-export]"
rake (perhaps via CI)

Should be mostly a no-op.

Fix encryption for exported logs bucket (see below)
Verify logs exported both before and after the migration are readable

Go to bucket
Navigate to some Old logs. Verify that you can view them (no KMS error)
Navigate to some New logs -- you may have to wait up to an hour for new logs to be exported. Verify that you can view them (no KMS error)

Disable (do NOT destroy!) Keys in global Keyring

Email to cloud-developers
Subject: New regions available for dev envs (action required)

Hi cloud developers,

Good news: It is now possible to create a dev environment in any Google Cloud region![1][2] Instructions are here:
https://github.com/gpii-ops/gpii-infra/blob/master/gcp/README.md#i-want-to-spin-up-my-dev-environment-in-a-different-region

Bad news: You must take action -- even if you are happy to stay in your current region -- to keep your cluster up-to-date. (Please do this migration separately, before trying out a new region. :))

If you are able to destroy your environment, the process is pretty standard:
* `cd gpii-infra/gcp/live/dev`
* `rake destroy`
* Update gpii-infra to latest master
* `rake update_exekube`
* `rake destroy_hard`[3]
* `rake`

If you are not able to destroy your environment, you can safely migrate an existing environment. It's a little more complex, so you may want to consult Ops before pursuing this route:
* See "Rollout plan: Migration steps" in https://github.com/gpii-ops/gpii-infra/pull/329.

Please ping us in #ops if you have questions or any trouble!

Thanks,
the ops team

[1] I haven't tested anything outside of the US. Who will spin up the first GPII Cloud in Europe??

[2] Some infrastructure (the GCS Bucket where Terraform keeps its state) remains in the `US` multi-region.

[3] `destroy_hard` is a new helper command for destroying even more stuff related to your environment, like Terraform state and secrets.

Re-encrypt exported logs
TBD

Thanks for thorough description 👍, it helps ;). I'm commenting on the text of PR only so far:

an alternative approach with a tfstate bucket in each region where we might wish to run an environment

What about creating the bucket in only the desired region (this would be parametrized in the common code) and then creating the cluster and all the resources in the corresponding region? (info about the region can be retrieved for example by using TF remote_state) That would avoid having unused resources and solve most/all the issues mentioned.
In general I would definitely prefer to have tfstate bucket in the same region as the rest of resources.

We have a decision to make about existing exported logs for stg and prd:

As discussed, I'm ok to loose log in non-prod env. For production I would try to preserve those - i think it should be achieveavble using gsutil - unless the old key material is destroyed. gsutil shoul be able to read those logs and by running cp save & reencrypt them with the new key/keyring (gsutil cp between 2 GCS buckets should work directly without actually pulling/pushing data via the node running the command).

Stepan, I'm afraid I don't understand your suggestion:

What about creating the bucket in only the desired region (this would be parametrized in the common code)

There is no single "desired region". The goal of this feature is to make it possible to run infrastructure in any region. So AFAICT we must either "migrate" back and forth, updating tfstate each time, or we must have dedicated tfstate in each region.

and then creating the cluster and all the resources in the corresponding region? (info about the region can be retrieved for example by using TF remote_state)

We do (with this PR) create the cluster and all resources in the specified region (i.e. the value of TF_VAR_infra_region. The only thing that is hardcoded to us-central1 is the tfstate bucket itself.

That would avoid having unused resources and solve most/all the issues mentioned.

I don't see how.

Re: exported logs:
Due to an incident, we have already destroyed the exported stg logs.
For prd logs, we agreed to rewrite them and keep them. I am figuring out how best to do this.

There is no single "desired region". The goal of this feature is to make it possible to run infrastructure in any region. So AFAICT we must either "migrate" back and forth, updating tfstate each time, or we must have dedicated tfstate in each region.

So where I was going with this was that the tfstate region, and therefore the region for the cluster, would be specified in common code. That way you can create the bucket in the region where the cluster will land.
So you would still be able to have cluster in any region, but you would want to change the region, you would have to rerun common.

Passing the desired region to all the places that need it is not the problem. The problem is keeping tfstate files in sync with cloud resources. I also don't like forcing developers to ask for help if they want to change regions.
I'd like to move forward with the current design. No matter what we do with the tfstate bucket, we will do it in a separate PR as I think this PR already has enough moving parts.

LGTM, for now.
I can not see very clear the advantages of having a bucket per region, and I can see some pain every time that we need to move from one to other. I would leave this PR as is, and see how we go. New approaches can be implemented in the future if we need them.

I would still consider unique names for keyrings (I think it'll make our lives easier).

This would avoid the need to import existing Keyrings when starting fresh. But Keyrings are indestructible, so we would accrue many discarded Keyrings over time. I think Sergey and I discussed this option and abandoned it for that reason.

Ok, I've done more testing, fixed a couple bugs, and updated the docs and rollout plan. My plan is to deploy this tomorrow morning unless there are objections.
The last bit I need to work out is re-encrypting the exported logs. I don't think it will be a big deal, but I keep forgetting to test it before deleting my environment and all the exported logs in it :). This step is not in the critical path, so it can be delayed a bit while I get the rest of the changes out.

Latest changes LGTM


cannot read property 'configureProvider' of undefined

Hi, as I made reference to here https://github.com/fullstackreact/react-native-oauth/issues/14
I'm having an issue with the oath on Android. Do I need to configure the manager differently based on platform? At present my code looks like this
const manager = new OAuthManager('MyAPp');
const config = {
  facebook: {
    client_id: 'MY_CLIENT_ID',
    client_secret: 'MY_CLIENT_SECRET',
  },
};
manager.configure(config);

It works amazingly (Thanks @auser !!) on ios, but in android everything about the manager is then coming up as undefined. I removed all my promises handling etc to just strip it back to console logging the manager variable but I was getting 'cannot read property 'configureProvider' of undefined and it wouldn't go any further.
Is this a bug or is this just user error? I (think) I;ve read the docs quite extensively and everything seems as it's supposed to be? Cheers

I just realised I hadn't configured anything here. Does this need to be setup or is that only to handle the ios side? If so what values should I put in, is it similar to how the ios is setup? I can't find anything about it in the docs sorry :/

Interesting... it should be handled in Android... I just confirmed it's working on android here. What version of rn-oauth do you have?

I just pushed an update. Also, the method name is authorize, not authorise

I just updated to version 2.19, yeah sorry 'authorise' was a typo here it was authorize.
I'm still getting the error. I've moved the function call and setup to a componentWillMount just so there's no additional factors to consider and it works fine on ios but on android I get this:
TypeError: Cannot read property 'authorize' of undefined
    at promisify.js:11
    at tryCallTwo (core.js:45)
    at doResolve (core.js:200)
    at new Promise (core.js:66)
    at promisify.js:6
    at OAuthManager.authorize (react-native-oauth.js:38)
    at AppNavigator.componentWillMount (appNavigator.js:48)
    at ReactCompositeComponent.js:347
    at measureLifeCyclePerf (ReactCompositeComponent.js:74)
    at ReactCompositeComponentWrapper.performInitialMount

What's the difference between your implementation and this? Is there any? https://github.com/fullstackreact/FirestackApp/blob/master/app/views/Authentication/Demos/Providers.js#L38

Here is the implementation along with the accompanying console logs when just fired in a componentWillMount lifecycle method:
  componentWillMount() {
    const manager = new OAuthManager('MY_APP');
    const config = {
      facebook: {
        client_id: 'MY_ID',
        client_secret: 'MY_SECRET',
      },
    };
    manager.configure(config);
    console.log('manager:', manager);
    manager.authorize('facebook')
    .then(x => console.log(x))
    .catch(err => console.log('error', err));
  }

And the logs:

As I said it works fine on ios that's the strange thing, but with android it's not even getting past the fire the function stage

Also you're passing in provider as a variable to authorize but it equals the string  'facebook' in one case so I can't see any difference there

Can you relink the library? Rnpm unlink and react-native link? It sounds like the android side didn't get linked

On trying to do that it seems my folder structure isn't quite what link is expecting, I've pmed you on gitter with the details, not sure if it's expected behaviour or my project is set up wrong?

Responded

I am also encountering this same issue, and react-native unlink then react-native link didn't solve the issue. Any idea?

Hi @signorekai does your android package name end in com.android by any chance? This was causing a problem with link for me

No it didn't, unfortunately.
On Thu, 19 Jan 2017, 13:41 Sam Matthews<EMAIL_ADDRESS>wrote:

Hi @signorekai https://github.com/signorekai does your android package
name end in com.android by any chance? This was causing a problem with link
for me
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fullstackreact/react-native-oauth/issues/37#issuecomment-273687090,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALms1-miDGjGnJgBVLx0s_V-cGLKn5O4ks5rTvdzgaJpZM4LgRVH
.

Happy to hop on ScreenHero to help debug.

Hi @auser , I'm having the same issue on iOS. I tried the react-native unlink and react-native link, as well and that didn't resolve the issue. I took a look at the set up you have in the link and nothing glaringly different. Here's how I'm
`
componentWillMount(){
const manager = new OAuthManager('my_app_name');
    const config =  {
        twitter: {
            consumer_key: 'my_consumer_key',
            consumer_secret: 'my_consumer_secret'
        },
        facebook: {
            client_id: 'my_client_id',
            client_secret: 'my_client_secret'
        }
    };
    manager.configure(config);
this.manager = manager;

}
`

Also having this issue.. @tendolukwago/ @SamMatthewsIsACommonName   did you manage to resolve this ?

@jsbranco  Not yet. The functionality wasn't pertinent at the time of posting this message, but is more pressing now. I'll be taking a look at it again this weekend, and will follow up.

@tendolukwago sure let me know if there is anything I can help with

I have the same problem in iOS.

Is there any progress ?

I have resolved my issue by doing the following changing the key sharing group to : io.fullstack.oauth.AUTH_MANAGER as described in README, i assumed it was my apps's details... but seems like after updating to  this and rebuilding resolved the configureProvider issue.

I have same issue like @ozsirma

Did anyone managed to get this running? I've strictly followed the README file, I've added io.fullstack.oauth.AUTH_MANAGER as Keychain Sharing group. I've run react-native link and I still get this error, even in a new project.

Ok I've figured it out. For me I haven't noticed Next, navigate to the neighboring "Build Phases" section of project settings, find the "Link Binary with Library" drop down, expand it, and click the "+" to add libOAuthManager.a to the list. in README.
I'm relatively new to react-native, but is there any reason why react-native link can't do this for us @auser ? Maybe if we discuss this I could help out and learn how automatic linking works.

I have the same error. Have anyone found a solution?

I'm having the same problem iOS. Has this been resolved?

i am getting the same issue in android. can you please help. @kodie looks like your setup worked fine in android as you have mentioned. i am new to this and looking please help. i need this to work as soon as possible. Thank you.


rapidjson v.s. nlohmannjson

For the same purpose, it is very wired to use two different json libraries in a single repo.
@zhanglei1949, as the one introducing the second json lib, you are obligated to compare the two libs and:

either give strong support to unified to nlohmann json, and change all rapidjson usage.
or unified to use rapidjson.

P.S. I will add a CI or post-commit hook to ask every PR explicitly to declare when introducing new third-party dependencies.

I find nlohmann json easy to use, but rapidjson offers better performance. We should consider switching to rapidjson.

Closed since resolved.


problem with pip install

when I type:

pip install -r requirements.txt

I got this message:

Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'

and I have already install all the requirements.
What's wrong with that?

@channimol first make sure that you have clone/downloaded the complete repo, then open terminal in the same directory where udemy-dl.py is, check if requirements.txt exists then use pip command to install.
Alternative
You can also install the requirements by using separate  commands for both dependencies.
Example (Linux if running as other than root) make sure python-pip is installed in linux:
sudo pip install requests
sudo pip install colorama
Example (windows) make sure pip is in you env, or change to directory under Python2**/3** "Scripts" and use pip:
`pip install requests`
`pip install colorama`


Adds default values & additional checks for database

This PR adds default values to the main config.
databaseEnabled() now also checks for doingExperimentalFeatures(), this reduces some code duplication.
Might fix #217 #182

The problem is the same.

The problem is the same.

Add me on discord and send me your config.yml + fish.yml (my discord is sarhatabaot)


Create suhuixin.py

suhuixin

view it on GitHub
---- Replied Message ----
| From | @.> |
| Date | 10/20/2024 19:48 |
| To | @.> |
| Cc |<EMAIL_ADDRESS>|
| Subject | Re: [USTB-SkyCode/2024Task] Create suhuixin.py (PR #8) |
Merged #8 into main.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>


feat: [DOC-3323]: documentation for step group variables

Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the maintainers.
What Type of PR is This?

[ ] Issue
[x] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[x] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

@shalini-agr this is open for a long time. Let me know when to merge.


Update PPP and DG

Codecov Report

Merging #202 (e5b5081) into master (6b17f0c) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #202   +/-   ##
=========================================
  Coverage     74.12%   74.12%
  Complexity      736      736
=========================================
  Files           108      108
  Lines          2095     2095
  Branches        249      249
=========================================
  Hits           1553     1553
  Misses          467      467
  Partials         75       75

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Add support for Ollama, Palm, Claude-2, Cohere, Replicate Llama2, CodeLlama, Hugging Face (100+LLMs) - using LiteLLM

This PR adds support for the above mentioned LLMs using LiteLLM https://github.com/BerriAI/litellm/
LiteLLM is a lightweight package to simplify LLM API calls - use any llm as a drop in replacement for gpt-3.5-turbo.
Example
from litellm import completion

## set ENV variables
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages)

# anthropic call
response = completion(model="claude-instant-1", messages=messages)

I noticed this repo also calls text-davinci-003, you can call it using litellm.completion
response = litellm.completion(model="text-davinci-003", messages)

@dakinggg @vchiley @mvpatel2000 can I get a review on this PR ?

Thanks @ishaan-jaff for the PR! After some discussion, we decided to keep the OpenAIChatAPIEvalWrapper class self-consistent by staying with the openai package.


Help with orientation to aggregation code

As luck would have it, I need to do some serious aggregation stuff over the next couple weeks -- exactly when @ggdhines is gone. He set up the "aggregation" button on the project builder, but I never get any results back if I push it. So I'm diving into the https://github.com/zooniverse/aggregation/ code. There's a lot there, though, and not much top-level documentation. I'm assuming the code I want to look at is in the engine/ subdirectory. But I'm getting a bit lost as to the top-level code (web_api.py? job_runner.py?).
I'm sure this code is all set to pull directly from Panoptes and create the directories and files that would be sent if the aggregation button were working. I'd like to modify a copy of it run the aggregation as stand-alone code, where I download all the classifications for a project (via the project builder) and then run aggregation on my machine using those classifications. But I'm having trouble figuring out what files do what.
Is anyone other than Greg familiar with these files? @edpaget maybe?

Greg is on vacation for the next couple weeks. I am only familiar with the api and job running code I setup.

@mkosmala the web_api.py and job_runner.py is to link the aggregation engine to the panoptes API (and not surprisingly run the jobs). Looks like something is up with the aggregation engine not reporting errors, i'll have a look asap.
We wanted to get this feature out asap but realized it will have bugs (unfortunately Greg is away). I'm not sure where @ggdhines got to with csv exports being an input for the aggregation engine, though it was discussed at length to do this. Also this code is setup to run on our systems with access to internal databases (postgres and cassandra) so i'm not sure how useful it will be. That said i'd start here and see if you can't shoehorn the csv data as an input and remove the dependency on postgres / cassandra.

@camallen Thanks. Yeah, looks like the main guts are in aggregation_api.py. I'm starting there. Thanks again.


Profile pages layout missing a container

The text goes from side to side of the screen
Steps:

Go to Team
Open any profile

Cool!


Clarify service providers can submit usage of non-CF resources (e.g. IaaS resources like VMs)

The readme on github described Abacus as providing "usage metering and aggregation for Cloud Foundry (CF) services". I believe it also allows submitting usage for non-CF resources (like IaaS VMs), which is an important requirement for IBM Bluemix. Could you update the readme to explain that this use case is indeed envisioned and supported, and add documentation (with example) how? (with a suggestion how to map to CF specific orgs)

Any service provider that wants to push usage to Abacus for the services (and more generally resources, including app runtimes, containers, etc) it provides can do that by using the Abacus usage submission API. The resource you submit usage for doesn't necessarily have to be provisioned by a CF service broker, it can be anything you want, provisioned the way you want.
For Abacus to be able to process usage data for a particular type of resource you just need to provide a definition of the types of measurements and metrics relevant to that resource (eg. my resources are CouchDB or MongoDB databases, or Linux containers, and their metrics are GB of storage and memory), and optionally provide a set of Javascript functions or formulas that will be used to meter, accumulate and aggregate those metrics.
Once you've done that you can start submitting usage data for instances of your resources. Usage data needs to include a resource_id (e.g. CouchDB, MongoDB, Linux-container... you can pick these ids yourself), a resource_instance_id (e.g your DB name or some id for an instance of your Linux container) and the measurements representing the usage incurred by that resource instance (e.g. 10GB of DB storage, 2GB of memory). Abacus will then process your usage data, meter and aggregate it for you.
Usage data is also expected to reference things like organization_id, space_id etc allowing you to aggregate your resources under organizations, spaces (or similar concepts that you'd like to map to organizations and spaces, e.g. department or team and dev/test/prod environments). That categorization and mapping are left as an exercise to the integrator of Abacus, but in my experience these two levels of aggregation have proven generic enough to fit how most people organize their Cloud resources.

We should probably update the docs at some point with an overview of how to define resources and configure Abacus to recognize them.

The question was answered. Please re-open if more discussion is needed.


[PE-D] joinG Command works when not provided a person

The command joinG g/1 will go through, with a message like "Person(s) added to group : CS1231S". Since it doesn't make much sense for a command to add no one to a group, I would like to suggest making this an invalid command.

Labels: severity.Low type.FeatureFlaw
original: kaldius/ped#4

duplicate


Intermittent core dumping when instantiating multiple CRS objects

Code Sample, a copy-pastable example if possible
import pyproj, datetime

test_codes = pyproj.get_codes("EPSG", pyproj.enums.PJType.PROJECTED_CRS, False)
start = datetime.datetime.now()
projs = [pyproj.CRS.from_epsg(code) for code in test_codes]
print(
    f"Instantiating {len(projs)} projs took {(datetime.datetime.now() - start).total_seconds()} seconds"
)

Problem description
Does not occur when PYPROJ_GLOBAL_CONTEXT=ON
$ python global_pyproj.py
Instantiating 4500 projs took 85.85579 seconds
$ python global_pyproj.py
Segmentation fault (core dumped)
$ python global_pyproj.py
Segmentation fault (core dumped)

With gdb:
$ gdb python
,,,
(gdb) run global_pyproj.py
Starting program: ~/miniconda/envs/proj71/bin/python global_pyproj.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
__strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:62
62	../sysdeps/x86_64/multiarch/strlen-avx2.S: No such file or directory.

Environment Information

Latest pyproj master version
PROJ 7.1

Installation method

conda & from source

More info:

Cannot re-produce with pyproj 2.6.1post1 & proj 7.1, so it is something changed in pyproj I think
Happened before the global context change

Probably need to bisect this one ...

Definitely reproducible: https://travis-ci.com/github/snowman2/pyproj/builds/174266962

Possibly related to #651

ad290fb8dc1fbf7da77c9014d88f769482bbbe16 is the first bad commit
...
    BUG: Make sure user directory in search paths (#638)


Refactor TextBox

This PR removes most functionality from TextBox.
We create a TextBox for every Timer, but it is only used as a storage class for the related text and potentially for measurements. The Draw() method was never called when rendering timers, and in nearly all cases, the TextBox is only used to access the linked timer.
The only place where TextBox::Draw() was really used is to draw the overlay when selecting a timeslice in the capture window, and this functionality abused TextBox by not passing any valid timer at all, it only required drawing of the green rectangle and some text.
Refactor:

All rendering-related functionality of TextBox was removed, it is purely used to store a pointer to a timer and some additional layouting information
Drawing of the selection rectangle has been implemented in the CaptureWindow without use of TextBox

Maybe we could remove TextBox completely in the future and work with the timer pointers directly.

Thanks. LGTM. I only added some C++-ish comments. But I really don't know much about this part of the code, so someone else should also approve (Gerrit's +1).

SInce Florian commented the same: @pierricgimmig I added you to the reviewer's list since Ronald is OOO - maybe you know of some side effects I was missing, just to be sure.


feature: constrain value of ENVIRONMENT setting

We have a bit of logic that depends on the value of settings.AppSettings.ENVIRONMENT variable, so we probably need to make the allowed values for that more structured, at the moment it is just typed str.
I'm thinking allowed values should be "dev", "stage", "prod", "local", "test", and we can either use Literal or Enum for this.

I use an Enum internally with these values as well, but can they be adapted to all projects?

To clarify, are you asking if all projects can fit into the dev/stage/prod/local/test paradigm?
What values do you use in your enum?
If a downstream app has requirements beyond those they can always create their own local settings object with a variable specific to thier needs. And we can always consider adding more environments if they are common enough.
Another way to look at it is that we are doing the wrong thing by overloading the ENVIRONMENT variable with all of this implicit functionality, and should maybe be using feature flags, or a separate MODE variable instead. But, I suspect that is over-complicating it.

I'd like to make a release soon, so LMK if you think this should be left out. I'm happy to hold off as it's not a critical issue.

are you asking if all projects can fit into the dev/stage/prod/local/test paradigm?

Specifically, I wonder if one need some more values, how can a local enum be used without avoiding lib's app.ENVIRONMENT to fail because it's a known value from the lib env enum?

Another approach could be:
# settings.py
class AppSettings(BaseSettings):
    ENVIRONMENT: str = "prod"
    TEST_ENVIRONMENT: str = "test"
    LOCAL_ENVIRONMENT: str = "local"

app = AppSettings()

IN_TEST_ENVIRONMENT = app.ENVIRONMENT == app.TEST_ENVIRONMENT
IN_LOCAL_ENVIRONMENT = app.ENVIRONMENT == app.LOCAL_ENVIRONMENT

something like that?

I'd like to make a release soon, so LMK if you think this should be left out. I'm happy to hold off as it's not a critical issue.

Actually I think It's part of a bigger issue which is: How can we easily override any setting in downstream projects?
For instance, before #219 was merged I first tried to add this at the end of my settings.py
settings.server.RELOAD = settings.app.ENVIRONMENT == ApiEnv.LOCAL.value

But this wouldn't work as my settings.py was imported after running after the lib's run_app() was called.
So all settings cannot be overriden the same way, and if you don't how the lib use them it's more difficult to meet your own needs.

But maybe It's not something we want to support as the lib is already a bit opiniated design wise, so enforcing env default values is ok.

# settings.py
class AppSettings(BaseSettings):
    ENVIRONMENT: str = "prod"
    TEST_ENVIRONMENT: str = "test"
    LOCAL_ENVIRONMENT: str = "local"

app = AppSettings()

IN_TEST_ENVIRONMENT = app.ENVIRONMENT == app.TEST_ENVIRONMENT
IN_LOCAL_ENVIRONMENT = app.ENVIRONMENT == app.LOCAL_ENVIRONMENT

That looks good! I find it a bit cleaner to write if IN_TEST_ENVIRONMENT: than if settings.app.ENVIRONEMNT.value == ENV.TEST.value: and adding IN_CUSTOM_ENVIRONMENT downstream is easy too.

settings.server.RELOAD = settings.app.ENVIRONMENT == ApiEnv.LOCAL.value

isn't this just the same as:
# .env
ENVIRONMENT=local
SERVER_RELOAD=true

But maybe It's not something we want to support as the lib is already a bit opiniated design wise, so enforcing env default values is ok.

Yes, this. Configuration by environment is pretty fundamental to the pattern here.

That looks good!

 I think it adds enough structure:)

isn't this just the same as:

# .env
ENVIRONMENT=local
SERVER_RELOAD=true

I still have to set SERVER_RELOAD=false in staging/prod . I prefer to have as much of my settings to be set programmatically when possible so I can write some tests for env behavior, and I deal with a minimal set of env vars when deploying ;)

Interesting! I've always gone the other way - in my production apps I don't have any "default" environment to force explicitly setting all values. I backed off from that stance with this library as I figured it wouldn't be a very popular design choice.
In practice, it just means that the default environment values are set in terraform config instead of the application, but that means that they can all be easily modified without having to modify the application source.

Just a matter of preference I guess ;)


Make snapshots work even if the task DAG changes

Currently snapshots are associated with tasks and node snapshot data is keyed by the node ID from the DAG. These node IDs can easily change. We need a way to make saving/loading a snapshot work even if the DAG changes.

The solution to this is proposed to make use of a new mechanism called the blob store. See https://github.com/influxdata/kapacitor/blob/master/BLOB_STORE_DESIGN.md

I am interested in the use case mentioned in the design document, to load and store trained model data. In particular, there are families of filters built from time-series models (the many variations of AR and MA, for example) in which the runtime execution is a relatively simple computation of weights and a moving window of values.
There is a class of problems of this type that lend themselves well to representation in InfluxDB. The input data arrives from the outside and is written to the database. The points themselves can be streamed through a filter UDF running in Kapacitor, providing realtime analytics. And then the base input data can be collected in batch and re-estimated, allowing a runtime system to easily perform its own rolling updates.
The features to implement this missing from the TICK stack right now are:

a deterministic method to feed the current set of weights (state) to a UDF as it starts up
a way to signal to a UDF to reload its weights from an updated estimate

The Blob Store proposal appears intended to provide opaque storage/retrieval for a UDF. In terms of providing a robust snapshot store to aid in system startup, and thus avoid a crippling amount of startup-querying as a bunch of big filters queue up their history data, it seems like a good approach.
However, as a mechanism for supporting the filtering models mentioned above, it seems like a step away from the easy transparency of the time series data stored in InfluxDB. While it is easy to imagine an external program performing a model estimation and storing the new weights into a blob for the UDF to read, if one wants to store the weights as-computed and as-used it seems more in-character to store the data as Points.


TypeScript types appear to be broken in 4.1.5

When upgrading from 4.1.4 to 4.1.5, we started seeing a bunch of TypeScript complaints:
spec/javascript/responses/selectors_spec.ts:120:29 - error TS2322: Type '{ responses: { is_selected: boolean; }[]; are_all_selected_across_pages: boolean; meta: { pagination: { total_count: number; }; }; campaign_data: { doc_pages_count: number; }; }' is not assignable to type 'never'.

120     expect(canGeneratePdf({ responses })).toBe(true);
                                ~~~~~~~~~

const canGeneratePdf = createSelector([getSelectedResponsesCount], (selected_responses_count) => {
  const can_bulk_download_pdf = Feature.isActive('responses/bulk_download_pdf');

  return can_bulk_download_pdf || selected_responses_count <= default_page_size;
});

Not sure why the type never is being inferred from the arguments.

Can you please show the entire source code for the selector and its inputs, and preferably as a CodeSandbox or repo that demonstrates this problem happening?  Also, do you have strict: true or strictNullChecks: true enabled in your TS config?

@markerikson hmm, I'll see what I can do. It's a pretty big private codebase. We don't have strict or strictNullChecks enabled at the moment.

Okay, yeah, that's the first issue.  Per several other recent issue threads, the current Reselect types expect that TS strict checks are turned on, and will very likely produce incorrect results if that's turned off.

Oof, well we've got several thousand violations still in our codebase, so we're not going to be able to switch that on for a bit. I think we might need to keep reselect locked down to 4.1.4 for now.

Okay, I think I've got a bare-bones repro. It looks like the issue seems to have something to do with chained selectors.

@mockdeep the first issue with that example is that none of the inputs are actually typed correctly. You have to give their arguments meaningful types for createSelector to be able to infer anything.

@markerikson that's the dream, but when we're trying to introduce TypeScript to an existing codebase it's an ongoing process. I'll add types to those and see what happens.

Okay, adding some more types to the selectors got it to pass type checking. I can understand it might be pretty difficult for you to sort out typing on your end, but it's also not quite ideal that a patch release would cause our build to break.

That is unfortunately the nature of TS in general :(  You can see in the recent release history how we've been iterating and fixing things up after the Reselect 4.1 release.
When I published 4.1, it passed all the existing type tests, and it had a fallback to the old types for anyone using TS 4.1 and earlier.  So, I assumed everything was good to go.
Since then, people have reported various breaking issues: the parameters types were getting unioned instead of intersected, the intersecting wasn't working right, etc, etc.
We're doing our best to make the types as correct as possible. Trust me, I don't want to have anyone's builds breaking :)  but we also can't predict exactly how everyone is going to use the library, and as you've seen, what gets fed in to the API can greatly change how the types behave.  So, there's a big distinction between "good-faith attempt to improve things that causes someone's build to break due to unforeseen complications", and "intentional major change to how the API is designed that will cause existing code to break".  All of the Reselect 4.1.x releases have been published with the intent of keeping existing code working as-is, or at most needing minor code changes that remove incorrect usages, rather than a deliberate breaking change to the API.

@markerikson generic types are broken too

import { createSelector } from 'reselect';

const getValue = <T>(value: T): T => value;

const makeGetValue = () =>
  createSelector(
    ({ data: { value } }: { data: { value: string } }) => value,
    // (v) => v,
    getValue,
  );

const value: string = makeGetValue()({ data: { value: 'value' } });

export default value;

@ahce : is this a real example, or one that's just trying to illustrate generic usage? Because that selector as written is not the right way to use Reselect conceptually. You should never write an "output selector" that is just x => x - there's no actual memoization happening.
Additionally, you should never continually call createSelector() every time like that, because it always generates a new selector instance and that can never memoize properly.
In general, the way the 4.1.x types work is you must provide actual types for all of the "input selectors", and Reselect will infer the resulting generated selector parameters and the types of the arguments for the "output selector".  You shouldn't have to provide any types for that output selector yourself.

@markerikson is an example with generic types

Yeah, like I said, that's not how you're supposed to use Reselect at all in the first place, TS or otherwise :(
If you can put together a different example that actually matches how Reselect is supposed to be used and demonstrates a problem, I'd definitely be interested.

Updated to 4.1.5 and also have types broken that worked fine before.
Some parts of my code:
// types.ts
export type Animal = { ... }
export type Elephant = Animal & { ... }

// reducers.ts
import { createReducer } from 'typesafe-actions';

const entitiesReducer = createReducer<Record<string, Elephant>>({});
const idsReducer = createReducer<Array<string>>([]);
const selectedReducer = createReducer<Array<string>>([]);

// selectors.ts
const getElephantState = (state: State) => state.elephants;

const getElephantEntities = createSelector(
  getElephantState,
  state => state.entities,
);

const getElephantIds = createSelector(
  getElephantState,
  state => state.ids
);

const getElephantSelection = createSelector(
  getElephantState,
  state => state.selected,
);

const getElephants = createSelector(
  getElephantEntities,
  getElephantIds,
  (animals, ids) => filterMapByKeys(animals, ids).filter(isNotEmpty),
);

export const getSelectedElephantList = createSelector(
  getElephants,
  getElephantSelection,
  filterAnimalsByIds,
);

// utils.ts
const filterListByKeys = <TEntity>(
  entities: Array<TEntity>,
  keys: Array<string>,
  getKey: (entity: TEntity) => string,
) => {
  const set = new Set(keys);
  return entities.filter(entity => set.has(getKey(entity)));
};

export const filterMapByKeys = <TEntity>(
  entities: Record<string, TEntity>,
  keys: Array<string>
) => keys.map(key => entities[key]).filter(isNotEmpty);

export const filterAnimalsByIds = <TAnimal extends Animal>(
  animals: Array<TAnimal>,
  ids: Array<string>,
) => filterListByKeys(animals, ids, getAnimalId);

export const isNotEmpty = <TValue>(value: TValue | null | undefined): value is TValue =>
  typeof value !== 'undefined' && value !== null;

// component

const selectedElephantList = useSelector(getSelectedElephantList);

Previously selectedElephantList would result in Elephant[], now it results in Animal[]. 4.1.2
seems to work whereas 4.1.3 breaks, so it must be something in that release.
Now that I've reverted back to 4.1.2 a different typing issue occurs:
// types.ts
export type SortDirection = 'asc' | 'desc';

export interface SelectorSortDirectionProps {
  sortDir?: SortDirection;
}

export interface SelectorSortFieldProps {
  sortKey?: string;
}

// utils.ts
export const getSelectorSortDirection = (state: State, props: SelectorSortDirectionProps) =>
  props.sortDir;

export const getSelectorSortKey = (state: State, props: SelectorSortFieldProps) =>
  props.sortKey;

// selectors.ts
export const getZebraEntityList = createSelector(
  getZebraEntityMap,
  getZebraIds,
  getSelectorSortDirection,
  getSelectorSortKey,
  (entityMap, ids, sortDir, sortKey) =>
    getSortedEntityList(filterMapByKeys(entityMap, ids), sortKey, sortDir, ['name']),
);

This is causing the following error, which I didn't have before:

Reverting back to 4.0.0 fixes it but even going up one release to 4.1.0-alpha.0 it breaks.

@codeaid : could you put together a repo or CodeSandbox that shows this behavior?

I have the same issue similar usage as @codeaid .
Revert back to 4.0 works.

@MingyaoLiu : I'll ask the same thing: please provide a CodeSandbox or repo that demonstrates the problem.
Just saying "I have the same problem" does not give me any information I can use to try to fix things.

@MingyaoLiu : I'll ask the same thing: please provide a CodeSandbox or repo that demonstrates the problem.
Just saying "I have the same problem" does not give me any information I can use to try to fix things.

Hi sorry I can't help with that in the moment, as I'm not very knowledgable on reselect. I'm working on doing a full upgrade on our build process, and I run into issues around this. I did downgrade all reselect in our packages to 4.0.0, and the build process is fixed.

@MingyaoLiu fwiw I'm not asking you to fix the actual problem :)
I'm asking you to:

Look at your app code that is showing an error
Try to create a very small standalone project that uses the same code patterns and also has the same error

FWIW, it's very possible that there may actually be a problem with the app code.  Reselect 4.1.x does a much better job of handling the TS types of all of the input selectors than 4.0 did.
But it could also be a bug in Reselect.  And I can't know whether it's our problem or your problem if you can't provide me any actual info.  For that matter, so far you haven't even said what actual error message you're seeing :)
The only way I can investigate is to see what some actual code that produces the error, and the best way to do that is to create a minimal standalone example project (preferably using CodeSandbox) so that I can try it out myself.

yah that's true, if I have time I will try to create something.

@markerikson Hey, sorry for the late reply. Here's a repo reproducing both of the issues:
https://codesandbox.io/s/reselect-issue-558-o3n52?file=/src/index.tsx
The way to test it is:

Ensure reselect is at v4.0.0 in the left dependencies panel and notice how the results variable in index.tsx is Elephant[] if you Ctrl/Cmd hover over it.
Also notice how there are no errors in the getSortedElephantEntityList call on line 25.
Change the version of Reselect to 4.1.5 in the left panel
Notice how results is now Animal[] and the call on line 25 errors out not understanding the merge of two parameter selectors getSelectorSortDirection and getSelectorSortKey in the selectors.ts file.

Hope this helps and thanks for looking into it!

any update?

Thanks just checking in trying to learn more about this issue.

@codeaid : I'm finally taking a look at this, and tbh I have two thoughts:

The example still feels overly complicated and would be easier to follow if you could shrink this down further
That said, looking at the very last step, I'm not convinced this is a "bug" per se.

If we skip the rest of the app and logic, what it seems to boil down to is:
  const getElephants = createSelector(
    getElephantEntities,
    getElephantIds,
    (animals, ids) => filterMapByKeys(animals, ids).filter(isNotEmpty)
  )

  const getElephantSelection = createSelector(
    getElephantState,
    state => state.selected
  )

  const filterAnimalsByIds = <TAnimal extends Animal>(
    animals: Array<TAnimal>,
    ids: Array<string>
  ) => filterListByKeys(animals, ids, getAnimalId)

  const getSelectedElephantList = createSelector(
    getElephants,
    getElephantSelection,
    filterAnimalsByIds
  )

  const results = getSelectedElephantList(store.getState())

and while both getElephants and getElephantSelection are typed as dealing with Elephant values, filterAnimalsByIds is already typed as dealing with Animal values, so that's what the final return type is.
At this point, I've seen a couple different "reproductions" added to this thread, but neither of them appear to be real actual bugs.  The first sandbox at https://github.com/reduxjs/reselect/issues/558#issuecomment-982209763 was invalid use of Reselect overall, and the one at https://github.com/reduxjs/reselect/issues/558#issuecomment-1020426657 seems like the code is working correctly based on the provided types.  Additionally, the very first examples provided appeared to be broken due to lack of strict null checks.
Given that, I'm going to close this.
If someone can put together a better reproduction that shows real problems happening with Reselect 4.1.5+, please file a new issue.

I ran into this issue recently when trying to consume a library that used reselect and was targeting ^4.0.0, after updating the application to target the latest version ^4.1.7. Once I updated the library the issue went away.
@markerikson do you have any advice in the case that I would not be able to update the library? Stick to targeting the old version?

@jacobroufa what "issue" are you seeing, specifically?

Apologies @markerikson. I was seeing that my input type is not assignable to type 'never' when I expected it to point to the interface of my state object.
After not seeing the error when creating the exact same selector in my own application codebase (targeting reselect ^4.1.7), I consumed the library (internal to my company) which targeted the older version of reselect (^4.0.0) and the error came up. I do not know how to provide this as a reproduction within a codesandbox -- can that emulate consuming a second sandbox as a library?
The selector I was using is very simple, e.g.:
import { selectIsLoading as librarySelectIsLoading } from 'myLocalLibrary'; // library targets 4.0.0 reselect
import { createSelector } from 'reselect'; // 4.1.7 reselect

interface IState {
  isLoading: boolean;
}

const selectState = (state: IState): IState => state || {} as IState;

const selectIsLoading = createSelector(
  selectState,
  (state: IState): boolean => state.isLoading || false
);

const state: IState = { isLoading: true };

// if using local selector as above, no error
selectIsLoading(state);

// error TS2345: Argument of type 'IState' is not assignable to parameter of type 'never'.
librarySelectIsLoading(state);

My immediate assumption would be that trying to mix and match selectors created with 4.1.x and 4.0 would be incompatible, because of the completely different TS type definitions.  I would strongly recommend trying to get this internal library updated to use 4.1.x, which has much better TS inference overall.

Sounds good. I intend to update this library; it's internal so thankfully I have a bit of say here. I am more asking for the benefit of others who may not have that capability, if you can point to a specific incompatibility between type definitions. No worries if not!
By "targets 4.0" I mean that the library I am consuming has 4.0 as a dependency. If I git grep reselect I see several references including that 4.0 dependency in my package-lock.json, and even though as you can see it resolves in both cases (locally and within the consumed module) to 4.1.7, I still get the aforementioned type error when I run tsc on my application:
package-lock.json:        "reselect": "^4.1.7"
package-lock.json:        "reselect": "^4.0.0",
package-lock.json:    "node_modules/reselect": {
package-lock.json:      "resolved": "https://registry.npmjs.org/reselect/-/reselect-4.1.7.tgz",
package-lock.json:        "reselect": "^4.0.0",
package-lock.json:    "reselect": {
package-lock.json:      "resolved": "https://registry.npmjs.org/reselect/-/reselect-4.1.7.tgz",

Yeah, I'd really need to see a repo or sandbox that demonstrates an issue happening at this point so I can poke at it myself.


Add MongoDB Atlas vector store auto-configuration

It also provides a starter.
Avoid using spring-ai-openai-spring-boot-starter in spring-ai-mongodb-atlas-store.
Fixes #455

This was a very quick response ;)
Thanks @eddumelendez


[CLOUD-3762] Restructure java imagestreams

Following the model being implemented by the SCLs, imagestreams are now
defined by both runtime and underlying OS version, with an emphasis on
making UBI 8 available and default.
In the case of Java, we also have both OpenJ9 alongside OpenJDK on some
architectures as an additional option.  The naming scheme used here
should cover any expected permutations going forward.

/assign @jmtd

Hi @yselkowitz thanks for this, it looks good. I think as soon as I merge this OKD will pick it up so we could see what impact this has on the generation scripts at openshift/library for OKD, and/or in a cluster setup too, before cherry-picking over to release. Depending on urgency.
One question, the form of the imagestream names you've chosen is e.g. "openjdk-8-ubi8", whereas our image names are the other way around, ubi8/openjdk-8 or ubi8-openjdk-8.yaml. Is the form you've chosen to be consistent with what SCL are doing?

One question, the form of the imagestream names you've chosen is e.g. "openjdk-8-ubi8", whereas our image names are the other way around, ubi8/openjdk-8 or ubi8-openjdk-8.yaml. Is the form you've chosen to be consistent with what SCL are doing?
I'm merging (to develop) ahead of hearing back from you on this

Thanks @jmtd this is the impact on OKD:
https://github.com/openshift/library/commit/ad744b03994a7f056782f8e2800353baac9eb86a
The SCLs are moving to a naming scheme of e.g. nodejs:12-ubi8.  Java is in a unique position where multiple implementations exist on some architectures.  To keep a common set of imagestream tags across architectures, I chose to prefix those with the implementation name, e.g. java:openjdk-8-ubi8.  Hope that helps make sense of this renaming.


Make it easier to extract values from the view elements for tests

The ViewElement created but the view function can be unit tested, but it's not always easy to do so. It would db nice to have some helper functions to extract aitributes/children etc.

I've been having a play around with this on my streams and come up with a hacky solution where the ViewElements have create called on them first and therefore become their Xamarin Forms equivalent:
https://github.com/garethhubball/TwitchDeck
There's also a helper method or two in there that will help me traverse the DOM. As the project grows I'm intending to do more for it and break out the test helper functions.
Given that there seems to be a mapping from string -> Xamarin Forms property would it be worth doing that kind of mapping in reverse in order to create a strongly-typed object to assert against?

Closed by #325


animation not worked properly when i put recyclerview inside nested scrollview

animation not worked properly when i put recyclerview inside nested scrollview
i think in nested scrollview all the items bind immediatley how to solve it

any solutions?


⚠ Bofrost.at has degraded performance

In 7d48e23, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 947 ms

Resolved: Bofrost.at performance has improved in 00537a0.