ByteBuffer is not a valid flatbuffer model

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04.5 LTS [Google Colab]
TensorFlow installed from (source or binary): nightly
TensorFlow version (or github SHA if from source): 2.5.0-dev20201029

Command used to run the converter or code if you’re using the Python API
I am trying to covert custom efficientdet d2 object detection model that I trained for Android.
I successfully trained it and also was able to convert it to tflite by using various github issue threads.
I followed this colab file to convert and was able to convert also used tf_convert and was able to convert
# did ran this after running export_tflite_graph_tf2.py
tflite_convert --saved_model_dir /content/model_nightly/saved_model --output_file /content/tflite-conversion.tflite

and also
import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_saved_model('/content/model_nightly/saved_model/',signature_keys=['serving_default'])
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.experimental_new_converter = True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]

tflite_model = converter.convert()

with tf.io.gfile.GFile('/content/model.tflite', 'wb') as f:
  f.write(tflite_model)

The output from the converter invocation
from command 1
W1030 22:40:38.686931<PHONE_NUMBER>34784 function_deserialization.py:416] Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
W1030 22:40:38.687373<PHONE_NUMBER>34784 function_deserialization.py:416] Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
W1030 22:40:38.687821<PHONE_NUMBER>34784 function_deserialization.py:416] Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
2020-10-30 22:40:56.738889: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:316] Ignored output_format.
2020-10-30 22:40:56.738961: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:319] Ignored drop_control_dependency.
2020-10-30 22:40:56.738973: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:325] Ignored change_concat_input_ranges.
2020-10-30 22:40:56.740050: I tensorflow/cc/saved_model/reader.cc:32] Reading SavedModel from: /content/model_graph_100K_nightly/saved_model
2020-10-30 22:40:57.034144: I tensorflow/cc/saved_model/reader.cc:55] Reading meta graph with tags { serve }
2020-10-30 22:40:57.034227: I tensorflow/cc/saved_model/reader.cc:93] Reading SavedModel debug info (if present) from: /content/model_graph_100K_nightly/saved_model
2020-10-30 22:40:57.034311: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-10-30 22:40:57.901184: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2020-10-30 22:40:58.067181: I tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.
2020-10-30 22:40:58.215865: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency:<PHONE_NUMBER> Hz
2020-10-30 22:41:00.181931: I tensorflow/cc/saved_model/loader.cc:190] Running initialization op on SavedModel bundle at path: /content/model_graph_100K_nightly/saved_model
2020-10-30 22:41:01.017615: I tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: success: OK. Took 4277564 microseconds.
2020-10-30 22:41:04.164837: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:194] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2020-10-30 22:41:06.474280: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
Exception ignored in: <bound method Buckets.__del__ of <tensorflow.python.eager.monitoring.ExponentialBuckets object at 0x7f037e83e798>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/monitoring.py", line 407, in __del__
AttributeError: 'NoneType' object has no attribute 'TFE_MonitoringDeleteBuckets'

from command 2
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.
WARNING:absl:Importing a function (__inference_EfficientDet-D2_layer_call_and_return_conditional_losses_131625) with ops with custom gradients. Will likely fail if a gradient is requested.

Also, please include a link to the saved model or GraphDef
# Put link here or attach to the issue.

Failure details

While using it in the android application from the repository here, I get the following error.

E/tensorflow: CameraActivity: Exception!
    java.lang.RuntimeException: java.lang.IllegalArgumentException: ByteBuffer is not a valid flatbuffer model
        at org.tensorflow.lite.examples.detection.tflite.TFLiteObjectDetectionAPIModel.create(TFLiteObjectDetectionAPIModel.java:153)
        at org.tensorflow.lite.examples.detection.DetectorActivity.onPreviewSizeChosen(DetectorActivity.java:99)
        at org.tensorflow.lite.examples.detection.CameraActivity.onPreviewFrame(CameraActivity.java:200)
        at android.hardware.Camera$EventHandler.handleMessage(Camera.java:1221)
        at android.os.Handler.dispatchMessage(Handler.java:107)
        at android.os.Looper.loop(Looper.java:214)
        at android.app.ActivityThread.main(ActivityThread.java:7356)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)
     Caused by: java.lang.IllegalArgumentException: ByteBuffer is not a valid flatbuffer model
        at org.tensorflow.lite.NativeInterpreterWrapper.createModelWithBuffer(Native Method)
        at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:62)
        at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:277)
        at org.tensorflow.lite.examples.detection.tflite.TFLiteObjectDetectionAPIModel.create(TFLiteObjectDetectionAPIModel.java:148)
        at org.tensorflow.lite.examples.detection.DetectorActivity.onPreviewSizeChosen(DetectorActivity.java:99)
        at org.tensorflow.lite.examples.detection.CameraActivity.onPreviewFrame(CameraActivity.java:200)
        at android.hardware.Camera$EventHandler.handleMessage(Camera.java:1221)
        at android.os.Handler.dispatchMessage(Handler.java:107)
        at android.os.Looper.loop(Looper.java:214)
        at android.app.ActivityThread.main(ActivityThread.java:7356)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)

I am trying to do this for my college project.

Hi Santosh,
You used SELECT_TF_OPS in the conversion, so in the app, it is necessary to add the dependency to gradle as well. Please check the instructions here.
dependencies {
    implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly'
    // This dependency adds the necessary TF op support.
    implementation 'org.tensorflow:tensorflow-lite-select-tf-ops:0.0.0-nightly'
}

BTW, only for EfficientNet-Lite version, they do not require SELECT_TF_OPS.

I also have the dependency added and am getting this error.

Hi Santosh and MCR,
Looks like there were failures of EfficientDet model in conversion, and then the interpreter cannot load the model file correctly.
I am not sure about the detail of EfficientDet you used. Can you provide more information? BTW, is there any EfficientDet lite model that you can find from the repo?
Thanks!

@ssuwal
Please provide complete code or if possible share a colab gist with the issue faced.

Sorry for delayed reply here. To give more info on what I am doing is, I have trained object detection model with Efficient Det D2 from Model Zoo. The model seems to be exported successfully. Only TFLite export doesn't work on current stable version so I tried beta version as someone was successful in doing that, I have linked the file in the main issue.
I have tried different ways of exporting as mentioned above and here is the link to the gist.
Gist TFlite Export
Also, linking the page in example android tflite object detection example where this is consumed
Function Loading TFLite
Let me know if you require more info.

I don't think this is related to the converter.
I think this is similar to
https://github.com/tensorflow/tensorflow/issues/44192
which should be fixed already

@karimnosseir
I am still having the same problem here is the github gist, please ignore the last code box.
Github Gist
I did pull latest nightly available

TensorFlow version (or github SHA if from source): 2.5.0-dev20201203

@ssuwal That doesn't look the same problem earlier. I see
"
failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
tf.Size {device = ""}
"
TFLite doesn't support tf.Size op natively, you can use TF SELECT which enables fallback to TF for unsupported Op.
See the guide
https://www.tensorflow.org/lite/guide/ops_select

@karimnosseir
Thanks for the quick reply here,
I was never able to get that last code block in the gist get to work, if you see the previous gist, it also fails with same error.
sorry if this mislead the issue.
My main concern is still the same, I have a custom model trained and able to convert to tflite using current nightly version, however the android sample application give the Bytebuffer is not valid flatbuffer error. I hope this clear things up.
Is there a way to check what is wrong with the tflite, or why is the bytebuffer not a valid flatbuffer?

@ssuwal I see, the gist confused me. Can you please paste the Java sample code ? Also, can you run it in python for example or fails also ?

  /** Memory-map the model file in Assets. */
  private static MappedByteBuffer loadModelFile(AssetManager assets, String modelFilename)
      throws IOException {
    AssetFileDescriptor fileDescriptor = assets.openFd(modelFilename);
    FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
    FileChannel fileChannel = inputStream.getChannel();
    long startOffset = fileDescriptor.getStartOffset();
    long declaredLength = fileDescriptor.getDeclaredLength();
    return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
  }

this is the code in the sample example, I haven't tried to load tflite in python, I will try that post update as soon as I can.
Also here is the file with the java code Load Model

# tf.__version__ = 2.5.0-dev20201028
# Load the TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path="./tflite-models/noops-model.tflite")

# interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
interpreter.resize_tensor_input(input_details[0]['index'], (1, 1008, 756, 3))
interpreter.allocate_tensors()

img = cv2.imread('./assets/dataset_test_old/JPEGImages/IMG_0644.jpg')
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# img_rgb = cv2.resize(img_rgb, (300, 300), cv2.INTER_AREA)
img_rgb = img_rgb.reshape([1, 1008, 756, 3])
interpreter.set_tensor(input_details[0]['index'], img_rgb)
interpreter.invoke()
de_boxes = interpreter.get_tensor(output_details[0]['index'])[0]
det_classes = interpreter.get_tensor(output_details[1]['index'])[0]
det_scores = interpreter.get_tensor(output_details[2]['index'])[0]
num_det = interpreter.get_tensor(output_details[3]['index'])[0]

I loaded the tflite model using above code and worked without any errors. Let me know if you need anything else from me.

This issue has be stagnant for 3 months now, any feed back or updates would be great.
Let me know if you anything is needed from my side to resolve this issue or assist in any way.

Am also having this issue as well

Am also having this issue as well

It seems that I get that error only when I load the module on a separate thread and only for some devices.

Hi @ssuwal !
We are checking to see whether you still need help in this issue . Could you save the model as mentioned in this comment before the lite conversion and let us know.
Thank you!


[Bug Report]: Bid auction problem with offline player.

Spark link
No response
Server Version
1.20.4
Server Platform
Purpur
Describe the bug
CrazyAuctions try call winner of auction but when he is unlogged from server, items from seller is expired and not added to winner.
Crate Type
None
Steps to reproduce

Added item to bid auction from first account.
Bid to item from second account / winner leave server.
Wait to time left.
After when time is left is not added to winner.

Additional context
CrazyAuction have in if statement, getBalance but with static function getPlayer() excepted online player on server, Plugin with Economy return 0.0 balance because player is null, when player leave server.
if (data.getBoolean("Items." + i + ".Biddable") && !data.getString("Items." + i + ".TopBidder").equalsIgnoreCase("None") && plugin.getSupport().getMoney(getPlayer(data.getString("Items." + i + ".TopBidder"))) >= data.getInt("Items." + i + ".Price")) {

    public static Player getPlayer(String name) {
        try {
            return Bukkit.getServer().getPlayer(name);
        } catch (Exception e) {
            return null;
        }
    }

Related to this issue: https://github.com/EncryptSL/LiteEco/issues/80
Agreements

[X] I have searched for and ensured there isn't already an open issue regarding this.
[ ] I ticked all the boxes without actually reading them
[X] My version of Minecraft is supported by CrazyCrew.

Fixed in newer versions since we no longer use player names for data


An error ocurred while starting the kernel(tensorflow-gpu)

While using the keras to proceed the code suddenly，the warining came out which are as follow：
An error ocurred while starting the kernel
2019󈚩󈚧 16:08:03.433226: I c:\users\user\source\repos\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1405] Found device 0 with properties:
name: GeForce RTX 2060 major: 7 minor: 5 memoryClockRate(GHz): 1.2
pciBusID: 0000:01:00.0
totalMemory: 6.00GiB freeMemory: 4.89GiB
2019󈚩󈚧 16:08:03.435266: I c:\users\user\source\repos\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1484] Adding visible gpu devices: 0
2019󈚩󈚧 16:08:05.728803: I c:\users\user\source\repos\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
2019󈚩󈚧 16:08:05.730056: I c:\users\user\source\repos\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:971] 0
2019󈚩󈚧 16:08:05.730321: I c:\users\user\source\repos\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:984] 0: N
2019󈚩󈚧 16:08:05.730723: I c:\users\user\source\repos\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4644 MB memory) ‑> physical GPU (device: 0, name: GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5
my system information

win10
cuda9.2
cdunn7.2
python3.65
TensorFlow version :1.10.0
GPU/CPU:RTX2060,i7 9750h
memory:16g

but when i used tensorflow directly,without processing code by keras,there was no warning and competely successful.
Despite all this,I still prefer keras rather than directly using tensorflow，somebody help me please XD.

The messages posted above all look like informational messages and not warnings.

The messages posted above all look like informational messages and not warnings.

Thank you for your reply，Although it can not be considered as error or warning,it stuck in epoch1 and came out this message.

@Oldhuntor This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.If you think we've misinterpreted a bug, please comment again with a clear explanation, as well as all of the information requested in the issue template. Thanks!

@Oldhuntor This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.If you think we've misinterpreted a bug, please comment again with a clear explanation, as well as all of the information requested in the issue template. Thanks!

Thank you for your reply,i'm going to ask question in that platform.


Only wrap exceptions in CosmosException - 400 for custom serializers

Description
Previous change started wrapping all Exceptions in all CosmosItemSerializers in a CosmosExceptions. This was intended for custom serializers - but not for the "default" path - we should not change exception types there.
All SDK Contribution checklist:

[X] The pull request does not introduce [breaking changes]
[X] CHANGELOG is updated for new features, bug fixes or other significant changes.
[X] I have read the contribution guidelines.

General Guidelines and Best Practices

[X] Title of the pull request is clear and informative.
[X] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[X] Pull request includes test coverage for the included changes.

/azp run java - cosmos - spark

/azp run java - cosmos - spark

/azp run java - cosmos - tests

/azp run java - cosmos - spark

/azp run java - cosmos - tests

/azp run java - cosmos - ci

/azp run java - cosmos - tests

/azp run java - cosmos - spark

/check-enforcer override

All tests including spark and live tests passed. Could not merge because of permanent status "Checking for ability to merge automatically" - tried pushing a random change (and reverting it) - it restarted tests but did not allow the auto-merge :-(

/check-enforcer override


flex-flow property doesn't get prefixed in Chrome (25)

Looking at this example on Codepen in Chrome 25, you would expect the results to look identical, but they don't.
However, looking through the result of getComputedStyle(document.documentElement, null) in Chrome's console, I'm not seeing -webkit-flex-flow anywhere on the list (despite the fact that the property exists/works).
So this may very well be a Chrome bug, but I thought I'd open an issue here in case it isn't, or in case anyone smarter than me cares to explain what's going on. :p  Thanks for PrefixFree, it's awesome!

+1, this is the only issue i have found. otherwise this is the best script of all time.


Invalid UTF-8 characters

Not all UTF-8 characters are accepted in an XML document. Here is the range of possible characters.
If you insert a character outside of this range, you get an invalid XML document that would give you "PCDATA invalid Char value .." error when you try to open it in chrome.
Do you think it would be a good idea to add an input sanitizer that strips away all invalid characters?
function utfForXml (input) {
    return input.replace(/[^\x09\x0A\x0D\x20-\xFF\x85\xA0-\uD7FF\uE000-\uFDCF\uFDE0-\uFFFD]/gm, '')
}

Sounds good to me. Maybe in combination with dropping a warning that characters have been removed or something similar.

Can you give an example of a character that is not supported? we've switched to an xml library since this code was created and this ticket may no longer be true.


Spike in latency when deleting plugin/route via Admin API

Is there an existing issue for this?

[X] I have searched the existing issues

Kong version ($ kong version)
2.5.1
Current Behavior
Hello we are having following issue in our production environment : Every time when we  DELETE a route or a plugin from our kong configuration we are seeing a spike in the latencies times. Since some of our services have a strict timeouts of 150ms this leads to a bunch of 499s.

We did the following test :
Initial configuration : 100 services with 2 routes  and 3 plugins , each service is pointed to an upstream with 5 targets.
Every 20-30 min we double this configuration ( see the image below).
Every 3 mins we delete one random plugin and one random route ( red lines on the graps)

Outcome : We are constantly seeing spikes in latency when we delete entities and these spikes are getting larger based on the amount of configuration entities. This leads to customers complaining about latency or rejecting the queries with 499 error code
Our production setup is 10 kong servers and one used for admin queries handling around 6000 QPS.
The KONG setup is as follows AWS c5.2xlarge machine (green line) and c5.xlarge ( blue line)  :
c5.2xlarge setup is :
proxy_listen = <IP_ADDRESS>:80,<IP_ADDRESS>:443 ssl
mem_cache_size = 6500m
headers =off
database = postgres
pg_host = DB_HOST
pg_port = 5432
pg_user = DB_USER
pg_password = DB_PASS
pg_database = DB_NAME

admin_listen = off
admin_access_log = /mnt/kong/logs/admin.log
admin_error_log = /mnt/kong/logs/admin.log

status_listen = <IP_ADDRESS>:8100
status_access_log = /mnt/kong/logs/status.log
status_error_log = /mnt/kong/logs/status.log

nginx_http_proxy_intercept_errors=off
nginx_http_keepalive_timeout=0
nginx_proxy_proxy_buffers = 1000 1m
nginx_proxy_proxy_buffer_size = 1m
nginx_proxy_large_client_header_buffers=4 8k

nginx_http_ssl_protocols = TLSv1 TLSv1.1 TLSv1.2
ssl_cipher_suite = custom
ssl_ciphers = ALL:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK

proxy_access_log = off
proxy_error_log = /mnt/kong/logs/nginx_error.log

We are also using a custom NGINX template mostly to adjust lua running and pending  timers
events {
    worker_connections 65535;
    multi_accept on;
}

http {

lua_max_running_timers 32768;
lua_max_pending_timers 32768;
lua_shared_dict kong                        10m;
lua_shared_dict kong_locks                  20m;
lua_shared_dict kong_healthchecks           50m;
lua_shared_dict kong_process_events         50m;
lua_shared_dict kong_cluster_events         10m;
lua_shared_dict kong_rate_limiting_counters 12m;
lua_shared_dict kong_core_db_cache          ${{MEM_CACHE_SIZE}};
lua_shared_dict kong_core_db_cache_miss     20m;
lua_shared_dict kong_db_cache               ${{MEM_CACHE_SIZE}};
lua_shared_dict kong_db_cache_miss          20m;

Kong is running as root with following params :
root		 hard	 nofile		 131072`

Expected Behavior
When configuration is added/deleted in kong via Admin API existing traffic do not see a spike in latency. When  adding/deleting entities  we should not see latency spikes on routes/services that are not modified.
Steps To Reproduce
I am using wrk2 https://github.com/giltene/wrk2 to simulate traffic against these 2 servers ( each getting 500QPS )
wrk -c 100 -t 50 -d 18000s -s miniperf.wrk.lua --latency -R 500 --timeout 1  http://kong-external-ip

I am creating services,routes, plugins and consumers using following scripts based on miniperf.sh
#!/usr/bin/env bash

export PATH=./bin:/usr/local/openresty/bin:$PATH
export KONG_PG_PASSWORD=<DB_PASS>
export KONG_PG_HOST=<DB_HOST>
export KONG_PG_DATABASE=<DB_NAME>
export KONG_PG_USER=<DB_USER>

KONG=localhost
KONG_PROXY=<LB_EXTERNAL_ADDRESS>
KONG_ADMIN=http://${KONG}:8001
KONG_DEMO_UPSTREAM_PORT=12345
KONG_DEMO_UPSTREAM_URL=http://<UPSTREAM_IP>:${KONG_DEMO_UPSTREAM_PORT}
KONG_DEMO_UPSTREAM_IP=<UPSTREAM_IP>

curl_output=/dev/null

declare scenario
declare noise=false
declare n_services
declare n_routes
declare n_consumers
declare n_targets
declare prefix
plugins=()
declare extra_services=0
extra_plugins=()

function get_longOpt {
  local -a opt_list=( $@ )
  local -A opt_map
  local -i index=0
  local next_item
  for item in ${opt_list[@]}; do
    # Convert arg list to map.
    let index++
    next_item="${opt_list[$index]}"
    if   [[ "${item}" == --* ]] \
      && [[ "${next_item}" != --* ]] \
      && [[ ! -z "${next_item}" ]]
    then
      item="$(printf '%s' "${item##*-}")"
      opt_map[${item}]="${next_item}"
    elif [[ "${item}" == --* ]] \
      && { [[ "${next_item}" == --* ]] \
      || [[ -z "${next_item}" ]]; }
    then
      item="$(printf '%s' "${item##*-}")"
      opt_map[${item}]=true
    fi
  done
  for item in ${!opt_map[@]}; do
    # Convert map keys to shell vars.
    value="${opt_map[$item]}"
    [[ ! -z "${value}" ]] && \
    printf -v "$item" '%s' "$value"
  done
}

function die() {
   echo -e "\033[1;31m *** $@\033[0m"
   echo
   exit 1
}

function add_plugin() {
   local service_name=$1
   case "$plugin" in
   udp-log)
      curl $KONG_ADMIN/services/$service_name/plugins -d "name=$plugin" -d "config.host=<IP_ADDRESS>" -d "config.port=10001" &> $curl_output
      ;;
   *)
      curl $KONG_ADMIN/services/$service_name/plugins -d "name=$plugin" &> $curl_output
      ;;
   esac
}

get_longOpt $@

echo "Values of our variables after parsing options:"
echo "scenario = $scenario"
echo "noise = $noise"
echo "n_services = $n_services"
echo "n_routes = $n_routes"
echo "n_consumers = $n_consumers"
echo "n_targets = $n_targets"
echo "prefix = $prefix"

###############################################################################

kong version &> /dev/null || die "kong is not in PATH."
openresty -V &> /dev/null || die "openresty is not in PATH."
curl --version &> /dev/null || die "curl is not in PATH."
wrk --version | grep -q wrk &> /dev/null || die "wrk is not in PATH."

   for (( i = 1; i <= n_services; i++ ))
   do

      service_name="${prefix}-service-$i"

      curl $KONG_ADMIN/upstreams -d "name=$service_name" -d "slots=1000" -d "healthchecks.active.unhealthy.tcp_failures=2"  -d "healthchecks.active.unhealthy.http_failures=2" -d "healthchecks.active.unhealthy.interval=10" -d "healthchecks.active.healthy.interval=10" -d "healthchecks.active.healthy.successes=2" -d "healthchecks.active.concurrency=5" -d "healthchecks.active.type=http" &> $curl_output

      for (( target = 0; target < n_targets; target++ ))
      do
       curl $KONG_ADMIN/upstreams/$service_name/targets -d "target=${KONG_DEMO_UPSTREAM_IP}:$((KONG_DEMO_UPSTREAM_PORT + target))" &> $curl_output
      done

      curl $KONG_ADMIN/services -d "name=$service_name" -d "protocol=http" -d "port=12345" -d "host=$service_name" &> $curl_output

      for plugin in "${plugins[@]}"
      do
         add_plugin "$service_name"
      done
      for (( j = 1; j <= n_routes; j++ ))
      do
         route_name=$prefix-s$i-r$j
         curl $KONG_ADMIN/services/$service_name/routes -d "name=$route_name" -d "paths[]=/$route_name" -d "request_buffering=false" -d "response_buffering=false"&> $curl_output
      done

   done

   echo "Creating $n_consumers consumers..."
   for (( i = 1; i <= n_consumers; i++ ))
   do
      consumer_name="$prefix-consumer-$i"
      curl $KONG_ADMIN/consumers -d "username=$consumer_name" &> $curl_output
      curl $KONG_ADMIN/consumers/$consumer_name/key-auth -d "key=$consumer_name" &> $curl_output
   done

Backend is started again using sample from miniperf.sh
#!/usr/bin/env bash

n_targets=5
KONG_DEMO_UPSTREAM_PORT=12345
export PATH=./bin:/usr/local/openresty/bin:$PATH

###############################################################################

for (( backend = 0; backend < n_targets; backend++ ))
do
  rm -rf miniperf.nginx.prefix_${backend}
  mkdir -p miniperf.nginx.prefix_${backend}
  mkdir -p miniperf.nginx.prefix_${backend}/logs
  mkdir -p miniperf.nginx.prefix_${backend}/pids
  mkdir -p miniperf.nginx.prefix_${backend}/conf
  cat <<EOF > miniperf.nginx.prefix_${backend}/conf/nginx.conf
  worker_processes auto;

  pid pids/nginx.pid;

  worker_rlimit_nofile 1024;

  events {
      worker_connections 1024;
      multi_accept on;
  }

  http {
      server {
          access_log /dev/null;
          error_log /dev/null;
          listen  $((KONG_DEMO_UPSTREAM_PORT + backend));
          location ~ /.* {
              return 200;
          }
      }
  }
EOF

  openresty -p miniperf.nginx.prefix_${backend}
done

Following scripts were used to delete routes and plugins every 3 mins :
for i in `curl http://localhost:8001/routes?size=100 |jq --raw-output '.data | .[]| "\(.id)"'`;do echo "DELETE : $i"; curl -X DELETE http://localhost:8001/routes/${i}; sleep 180; done

for i in `curl http://localhost:8001/plugins?size=100 |jq --raw-output '.data | .[]| "\(.id)"'`;do echo "DELETE : $i"; curl -X DELETE http://localhost:8001/plugins/${i}; sleep 180; done`

Anything else?
No response

Adding some more graphs with one more increase of the config size :

worker_consistency=eventual
worker_state_update_frequency=<secs>

More  less impact on proxying, it defaults to 5, but try 10. Changes will be applied with delay.
this helps. We need to perhaps add some yielding on plugin iterator rebuilds. For router we already added some in 2.6.
Are you running kong in traditional mode?
In future we look for incremental rebuilds, but you can try the above for now.

See:
https://konghq.com/blog/kong-1-2-released/
For the effect on latency with:
worker_consistency=eventual

(it was called router_consistency originally)

Btw. miniperf has noise option that does changes while running tests.

Yes, afaik it only happened in hybrid dataplanes/dbless nodes. There on config changes a lot of stuff gets rebuild. On traditional it is more granular and event based. We have only kept the strict mode for backward compatibility. I think wr should drop it completely and just have the eventual. That might change with 3.0.

https://github.com/Kong/kong/pull/8888 This might help.


fix(blesta registrar module): manual renewals fix via manage domain section in admin panel

:tada: This PR is included in version 3.1.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add comments to spec.go

spec.go lacks lot of explanation of fields and structs. This commit
adds comments to the file.

@containscafeine can you PTAL at this ?


Feature request: primitive/unsafe/internal API which doesn't require MonadResource

I'm considering changing to things like this:
query
  :: (ToRow q, FromRow r, MonadIO m, MonadCatch m)
  => Connection
  -> Query
  -> q
  -> (Stream (Of r) m () -> m a)
  -> m a

Would that be a workable API? That is, are you ok with continuation passing? I'm open to other ideas.


Update vss-extension.json

adding "supportsTasks" property key to make the Trivy task visible in build result summary only when the task is actually being used. We are using Trivy task in few of our pipelines and the tab gives error [Timeline record(s) missing: cannot load results. Is Trivy configured to run on this build?] when trivy is task is not part of the pipeline.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Would be great to get this merged, currently we're always seeing Trivy in the pipeline build tabs. When you have a lot of tasks you use this could clog up the UI. It'd definitely be sweet to only show this when it is actually being used

Thanks, this is great :+1:

Thanks. Our pipeline looks better now!!


Restrict prod linux builders to Debian machines

In preparation for builder image switch from debian to ubuntu, restrict builders to only build on Debian VMs.

Fixed in #102253


tests/hassh Add test cases for hassh

This commit adds test cases for the ssh.hassh, ssh.hassh.string, ssh.hassh.server, ssh.hassh.server.string keywords.
Modifies https://github.com/OISF/suricata-verify/pull/259 by

Fixing comments in ssh-banner-only test.

Can you also add checks to one of the tests that looks for the various hassh fields and confirms they are correct?

Can you also add checks to one of the tests that looks for the various hassh fields and confirms they are correct?

I am not sure I understand what you mean @victorjulien
Do you mean using alerts as checks against the values is not good ? cf
https://github.com/OISF/suricata-verify/pull/262/files#diff-9dabb85edfb1a42602d72e3013624b99R3

https://github.com/OISF/suricata-verify/pull/263


No clean separation of OER and PER

It looks like that one cannot cleanly build PER-only - if OER wasn't specified, the code won't compile:
$ asn1c -Werror -fwide-types -fincludes-quoted -pdu=Test2c -fline-refs -fbless-SIZE -gen-PER Test2c.asn1
. . . . .
clang -maes -mpclmul -mrdrnd -msse2 -mssse3 -msse4.2 -mtune=native -Os -Ofast -DASN_DISABLE_OER_SUPPORT  -DPDU=Test2c -I.  -o Test2c.o -c Test2c.c
Test2c.c:82:30: error: variable has incomplete type 'asn_oer_constraints_t' (aka 'void')
static asn_oer_constraints_t asn_OER_memb_count_constr_2 CC_NOTUSED = {
                             ^
. . . . .
$ head -84 Test2c.c
. . . . .
static asn_oer_constraints_t asn_OER_memb_count_constr_2 CC_NOTUSED = {  <=== line 82
	{ 2, 1 }	/* (40..20000) */,
	-1};

Add -no-gen-OER flag to asn1c. The ChangeLog specifies:
* asn1c -gen-XXX flags are deprecated. Use -no-gen-XXX to disable codecs.


update character limit of routesubdomain to 54

Signed-off-by: Leena Jawale<EMAIL_ADDRESS>issue: https://github.com/open-cluster-management/backlog/issues/17836

/cc @itdove

/lgtm

Thanks @leena-jawale . Next time you're in here, consider adding a comment explaining where the 54 comes from, or even using the length of the prefix to calculate the size. Thanks!


Incorrect Imagenet dataset

I downloaded the Imagenet dataset linked in this repo, and I think the dataset for it (50000 test images with labels, box downsampling) doesn't match the official Imagenet 32x32/64x64 versions used for NLL benchmarks (https://github.com/openai/vdvae/blob/main/setup_imagenet.sh, 49999 test images with no labels, can download from https://academictorrents.com/details/96816a530ee002254d29bf7a61c0c158d3dedc3b). Difference in downsampling method used during pre-processing will make the NLL's not comparable.

Hello @prafullasd
Thank you for the interest you show in this work and thanks for reaching out about his issue!
You bring up a very good point. We have had our skepticism about the results achieved on Imagenet datasets in our work when the NLL results were very different from the VDVAE baseline. The confusing part about all of this is that the Imagenet version used in NLL benchmarks used to be hosted on this website (as you pointed out by the VDVAE download script), and it seems that since the update of that website, the downsampled imagenet is now downsampled in a different method (this is the part we missed). To add to that confusion even more, some prior work also seems to use the incorrect Imagenet version.
For completeness and future reference:

Original Imagenet 32x32 used in NLL benchmarks: https://academictorrents.com/details/bf62f5051ef878b9c357e6221e879629a9b4b172
Original Imagenet 64x64 used in NLL benchmarks:
https://academictorrents.com/details/96816a530ee002254d29bf7a61c0c158d3dedc3b

NLL reported results on Imagenet will probably change when we use these two versions of the dataset. We will update these metrics as we re-do the experiments (will take some time). The general expectation is that our reported NLL should get closer to the NLL reported by VDVAE, which would make sense and match our findings on FFHQ 5-bits.
While noting this mistake is very important and will ensure full research correctness, it doesn't affect the main contribution of the paper much: "Efficient-VDVAE keeps comparable or better NLL performance with less memory and faster training". Nevertheless, being precise in reporting the results is very much desirable.
Thank you very much for pointing this out and helping improve the quality of our work!
I will keep the issue open until we make our updates.
If you find any other problems, please let us know, we appreciate this a lot!
Rayhane.


TLS version for SFB Edge 2019 on Win2019

the security note in this article says "Security note: To ensure the strongest cryptographic protocol is used, Skype for Business Server 2019 will offer TLS encryption protocols in the following order to clients: TLS 1.3, TLS 1.2. TLS is a critical aspect of Skype for Business Server 2019 and thus it is required in order to maintain a supported environment."
I have SFB 2019 Edge server deployed on Win2019. When I run TLS checks for Access Edge public IP on port 443 (for client SIP/TLS), the https://www.ssllabs.com/ssltest/ tool shows SFB 2019 Edge server supports the following TLS versions:
TLS 1.3	No
TLS 1.2	Yes
TLS 1.1	Yes
TLS 1.0	Yes
SSL 3	No
SSL 2	No
So, TLS 1.3 is not supported and TLS 1.0 and TLS 1.1 are supported!?!?
Does it mean the documentation is incorrect? Or it is only applicable to HTTPS traffic? Or only applicable to internal SFB servers such as Front Ends?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a7679160-fb98-6e52-07f3-b844ba979065
Version Independent ID: a137b9d1-7721-c4ba-b2bd-0122795e80b6
Content: Encryption for Skype for Business Server
Content Source: Skype/SfbServer/plan-your-deployment/security/encryption.md
Product: skype-for-business-itpro
GitHub Login: @LanaChin
Microsoft Alias: v-lanac

@shivtorov  Thank you for submitting feedback and contributing to the docs. We are currently investigating this.

Hello @LanaChin @MicrosoftHeidi I see that 1.3 was added to the article not so long, can you please confirm if SfB 2019 supports TLS 1.3? could not find another official statement and I could repro what @shivtorov is saying here. Thank you

Hello @LanaChin @MicrosoftHeidi did you have chance to review my comments? thanks in advance

@scanum , I have just noticed that all doc pages for on-premises SFB Server don't show GitHub integration any more. Including the question raised in this github issue is not shown on the content page.
Has it been done on-purpose or simply something is broken?
Does it mean Microsoft will not update SFB Server 2019 documentation any more?

@shivtorov Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.

Regarding this- We are investigating at the moment.
@scanum , I have just noticed that all doc pages for on-premises SFB Server don't show GitHub integration any more. Including the question raised in this github issue is not shown on the content page.Has it been done on-purpose or simply something is broken?Does it mean Microsoft will not update SFB Server 2019 documentation any more?

@shivtorov Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.

@scanum, there were no changes to the status of this issue with documentation. The information in the article still does not meet the reality. @get-itips has confirmed above that it is indeed the case. Either the article is providing false info about TLS versions supported by SFB Server 2019, or it is missing a note that some manual configuration steps are required to achieve the TLS settings described in the article.


Tiles metadata fix

Overview
Related Issue / Discussion
Additional Information
Contributions and Licensing
(as per https://github.com/geopython/pygeoapi/blob/master/CONTRIBUTING.md#contributions-and-licensing)

[x] I'd like to contribute [feature X|bugfix Y|docs|something else] to pygeoapi. I confirm that my contributions to pygeoapi will be compatible with the pygeoapi license guidelines at the time of contribution.
[x] I have already previously agreed to the pygeoapi Contributions and Licensing Guidelines

@PascalLike rebasing with master and fixing the tilesets template, if you can merge this fix then I believe https://github.com/geopython/pygeoapi/pull/1051 is ready to merge as well.


Getting content when there's an exception

For validation, etc. scenarios, it's useful to be able to access the response content in case of failure. I can see that the following is used:
 if (!this.IgnoreResponseStatusCode)
              response.EnsureSuccessStatusCode();

And that we can set Ignore... to false to get the IRestResponse with raw bytes, status code, etc. However, it's a bit tedious to then check whether it's successful or not. Specially since HttpClient provides am IsSuccessStatus helper. I think it would be useful to have this available in the response. I can think of three ways that may deal with this scenario (i.e. get content in case of failure):

Add a boolean to IRestResponse that basically takes the value of httpclient's IsSuccess flag.
Add an EnsureSuccessCode method on IRestResponse that throws an exception that has the RestResponse contents inside of it.
Instead of bubbling HttpClient's exception raised in Ensure..(), trap it and throw a different exception that has the payload, etc. in it. Guessing this would be a breaking change.

Having easy access to the content would allow clients to use validation messages, etc. without forcing them to do weird Message parsing (if errors aren't ignored), and without having to do validation around the IRestResponse status code (in case of ignoring errors).
Thoughts?

This is a good idea. I think that the best solution is to add an "IsSuccess" flag to the RestResponse.

Yeah... that should meet the criteria and should be easy to do. Option 2 could be an extension method that clients can implement themselves if they're interested. Option 3 doesn't look attractive as it breaks clients.

Version 2.3.0 is available now.


NormalModule.needRebuild cache calculations seem to be wrong for filesystems that don't support millisecond precision mtimes

I was writing some integrations tests for broccoli-webpack-cached (trying to test that broccoli and webpack caching were working well together) and I was getting pretty unexpected, seemingly non-deterministic behavior. That lead me to digging into how Webpack caches things, namely how:

NormalModule saves a buildTimestamp every time a file is built
At the start of a new compile, Webpack gathers the mtimes from the filesystem for of every file in the dependency tree via CachePlugin
And then during the next compile, compares the new mtime against the previously saved buildTimestamp for each file in needsRebuilt

That all makes sense, but the problem I see is that buildTimestamp is saved with millisecond-level precision. However, many filesystems out there (in my case, HFS on OSX) only support second-level precision. So this can happen:

A compile starts, let's say at at 13:50:00 and 100ms (hour, min, second, and ms). That is saved to this.buildTimestamp.
The compile ends, the file is modified immediately at 13:50:00 and 500ms. But the timestamp saved in the compiler.fileTimestamps object has no millisecond precision, so is only 13:50:00 and 0ms
Another compile is kicked off soon after and checks if that file needs to be rebuilt. It compares to see if the new filesystem timestamp (13:50:00 and 0ms) is >= the previous buildTimestamp (13:50:00 and 500ms) and decides there has not been a file modification. Even though there definitely was a change to the file, it just happened within the same second and was ignored.

I verified this problem goes away when I force NormalModule to truncate milliseconds from saved the buildTimestamp, like so:
this.buildTimestamp = Math.floor(new Date().getTime() / 1000) * 1000;

But that likely isn't a good change. Ideally, I think that truncation would only happen if we can detect that the input filesystem only has second-level precision (but I didn't look deep how hard/easy that is).
I realize that I am much more likely to run into this issue when writing test code, and that is pretty different than a typical local dev server/watch/build script use-case. But I gotta think that this bug even rears its head in normal scenarios from time to time... well that is unless I've missed something.
ps: Above Webpack github links are to 1.x not 2.x-beta source.
pps: Node didn't always, but now does support millisecond precision for filesystems that support it.

This is pretty critical...

okey, I tried to fix it... but unsure if it's working correctly. Could you run your test agains the webpack master branch with updated watchpack dependency.

Awesome, thanks.
To test I first backported your fix on top of v1.12.13, since there were several other things I'd need to change to get on webpack 2.x.
After that, things were almost working, except for a couple small things. I needed this this fix to ensure the timestamps were actual timestamps and summed into ints rather than a concatenated string.
And also, I needed to make sure that the timestamp adjustment happened after the async.forEach loop was finished.
After that tiny fix, things are golden :smile_cat:. Though I thought I could make the FS_ACCURENCY heuristic a bit better:

Tweak calculation of FS_ACCURENCY
Before this accuracy detection code would end up leaving FS_ACCURENCY equal to 10000 or 2000 on my machine (depending on whether the time stamp was something like<PHONE_NUMBER>000 or<PHONE_NUMBER>000). Which was pretty far away from the actual precision of one second… and concerning to me if that would make us cache too few things.
So my attempt here was to assume that FS accuracy was always an order of magnitude—1, 10, 100, or 1000 milliseconds—and be a little more aggressive to set the FS_ACCURENCY lower. For example, in the new code, any time a timestamp is seen with 3 trailing zeros (mtime % 10000), I assume that the precision is 1000ms.

Those changes look good to you? Let me know if you'd like to me make an actual PR.

@sokra, I continued to run into more issues†, so I tried a different approach. Instead of accounting for filesystem accuracy, I tried to instead improve webpack to only refer to and compare actual file mtimes and content hashes‡. I spiked on that, with a couple tests to confirm, over in my fork's mtime-for-caching branch: https://github.com/webpack/webpack/compare/v1.12.13...timmfin:mtime-for-caching (note it is on Webpack 1.x and also requires these webpack-core changes.
Additionally, to more concretely illustrate this issue (and another I found with CachePlugin), I created a webpack-cache-issues example repo with scripts to replicate the cache issues.
Note, my proposed code changes are far from perfect. Not only do I not have a full understanding of the whole Webpack code base (for example, are NormalModules the only thing I need to worry about?), I also ended up needing to make the needRebuild function async and refactor some core methods. And I'm reading the file from the inputFileSystem in two separate spots. So consider my code a prototype, but not recommendation of the exact implementation. Fortunately, I do have four new tests to test these issues and I've made sure my prototype passes the rest of the test suite. Plus, I'd be very glad to work on this more given some pointers and recommendations.
†: The other issue I was running into was that Webpack's previous caching logic (in particular, the logic when directly using the Webpack node API) assumed that the there was little-to-no time elapsed between when he original file modification occurs and when Webpack starts to build the file. Since the buildTimestamp is merely a new Date it can be problematic to compare it to the file's mtime. Because if Webpack doesn't actually start to build (and save a new buildTimestamp) immediately after the file is modified, another modification could sneak in and never get built.
Here's one (but not the only) example of how this goes wrong, even if the filesystem has millisecond-level precision:

I run broccoli server to start my build  (let's say it started at 12:00pm and 0 seconds)
Various things happen, maybe taking a few seconds to do other broccoli build tasks.
(Sometime during this build, I modify X.js again—new mtime of 12:00pm and 2 seconds—but since the broccoli server/watcher is blocking, that change only queues up another build for when the current one finishes.)
Then my Brocfile.js delegates to the Webpack compiler (via broccoli-webpack-cached)
A new buildTimestamp is created for X.js (let's say at 12:00pm and 5 seconds) and Webpack builds against the original content (aka, those modifications made at 12:00pm and 2 seconds were not passed on by broccoli yet)
The Webpack compile finishes, and then the broccoli build finishes.
Broccoli kicks off the queued rebuild from X.js's earlier change.
Broccoli again does other build steps, which may take a few seconds before getting to the Webpack step.
The Webpack compile is kicked off and the CachePlugin runs before the compile, saving the mtime from X.js for later use (it saves 12:00pm and 2 seconds)
Webpack then sees if X.js needs to rebuild, and to do so compares the files's mtime, 12:00pm and 2 seconds, and the buildTimestamp, which was set to 12:00pm and 5 seconds. Since the buildTimestamp is greater than that saved mtime, needRebuild returns false. Which is incorrect because the file has been modified.

‡: The main reason why a file mtime is not sufficient for a needRebuild check, is because when the filesystem only has second-level precision you can pretty easily modify & build a file multiple times in the same second. So without anything else, needRuild has to return newMtime >= oldTime—in other words if the newMtime == oldMime, you have to assume that there might be a change. When you check the file's content hashes too (only if the newMtime >= oldMtime) you can be sure that you are not unnecessarily rebuilding file. (I created these tests to check for this sort of unnecessary rebuilding)

Thanks for the testcases. I added your test case and also added my own tests to the webpack test suite. But they pass without any change to the code (on my computer and on travis). (master branch)
So either this was already fixed, or is something specify to your platform.
I want to note here that while developing watchpack I already considered file system accurency. I tries to detect accurency automatically and fires more event if it is unsure if a change could have occured.
Could you run the webpack tests on your computer to check if this is a platform specific issue? (you need to npm link webpack into webpack, see package.json scripts for the CI)

K, here is an example of those master tests failing on my macbook pro (I think that one of these problems is specific to filesystems with second level precision which is very often/always the case on macs these days). Note, I did a fresh, separate clone and clean npm install:
tfinley at BOSMM2N693QG3QD in ~/src/temp/webpack-master on master
± mocha --harmony --full-trace --check-leaks test/Compiler-caching.test.js

  Compiler (caching)
    ✓ should cache single file (with manual 1s wait)  (1171ms)
    ✓ should cache single file (even with no timeout)  (1123ms)
    1) should only build when modified (with manual 1s wait)
    2) should only build when modified (even with no timeout)

  2 passing (2s)
  2 failing

  1) Compiler (caching) should only build when modified (with manual 1s wait):

      Uncaught AssertionError: a.js should have been built
      + expected - actual

      -false
      +true

      at Assertion.fail (/Users/tfinley/src/temp/webpack-master/node_modules/should/lib/assertion.js:92:17)
      at Assertion.Object.defineProperty.value (/Users/tfinley/src/temp/webpack-master/node_modules/should/lib/assertion.js:164:19)
      at /Users/tfinley/src/temp/webpack-master/test/Compiler-caching.test.js:242:39
      at /Users/tfinley/src/temp/webpack-master/test/Compiler-caching.test.js:63:5
      at Compiler.<anonymous> (/Users/tfinley/src/temp/webpack-master/lib/Compiler.js:224:14)
      ...

  2) Compiler (caching) should only build when modified (even with no timeout):

      Uncaught AssertionError: a.js should have been built
      + expected - actual

      -false
      +true

      at Assertion.fail (/Users/tfinley/src/temp/webpack-master/node_modules/should/lib/assertion.js:92:17)
      at Assertion.Object.defineProperty.value (/Users/tfinley/src/temp/webpack-master/node_modules/should/lib/assertion.js:164:19)
      at /Users/tfinley/src/temp/webpack-master/test/Compiler-caching.test.js:284:39
      at /Users/tfinley/src/temp/webpack-master/test/Compiler-caching.test.js:63:5
      at Compiler.<anonymous> (/Users/tfinley/src/temp/webpack-master/lib/Compiler.js:224:14)
      ...

Also, a couple other things:

Do you follow my logic at all regarding the issue with using webpack via the API instead of the cli tools? I very well may be missing something, but one of the problems I've noticed is that a module's buildTimestamp can become "disconnected" from the actual file modification time when webpack is merely a part of an existing build toolchain (and not actually doing the filesystem watching itself).
Did you take a peek at the separate test scripts I made in the webpack-cache-issues repo? There are a little different than the unit tests in my fork (which I think you based your new master tests off of).

Thanks again for looking into this. And @sokra please do not hesitate to let me know how I and my team at HubSpot can help Webpack out! We are incredibly appreciative of all your efforts, and want to do our part to help push community/project forward.

Ok I was able to reproduce the caching issue by formating a usb stick with FAT32 (2s fs). I also enabled osx build on travis (seem to be new :+1: ), which also shows this issue.
Now investigating.

If using Compiler.run instead of Compiler.watch, webpack will read timestamps of all files of the last build before starting the build to determine what can be reused. When watching webpack get these timestamps from the watcher, so the build is faster.

Nice. I'm going to go ahead and try and get travis to build my webpack-cache-issues repo too.

ok I fixed some issues (purge before cache, typos in CachePlugin).
Some stuff was expected to fail (module rebuilds more often on inaccurate file system, because we rebuild if we are unsure if the file was changed).
Some stuff is still not working (2 watchpack tests fail, but I need to grab a Mac to fix it (don't own one)).
The webpack tests now pass on OSX too and CI runs on OXS too.

@timmfin Can you still see this issue?

I'm no longer seeing this specific issue. But that's because we (HubSpot internal build tools) have moved away from a broccoli-based build that calls webpack and are using the webpack cli and/or dev middleware directly.
So while this is not specifically an issue for me anymore I'm pretty sure that there is still a core problem with webpack if you are manually starting a compilation via the API (when there is a significant delay between the file's modification time and when some build code manually kicks off a compiler.run(...)). And I don't think there is a good way to solve that other than tracking file content hashes for invalidation (which I tried to implement in a fork).
I leave it up to the maintainers if this issue should be closed or not. Even though I do think there is a problem, I don't know how many people are dealing with that case. And I certainly won't be offended, I can't offer up as much time trying to debug/validate an issue that isn't pressing for me anymore.
ps: Though, I'm still often running into awkwardness with second-level filesystem precision. However I'm pretty sure that is illustrated and well described by this watchpack issue (this comment in particular).

see watchpack issue


Bug in /ndn/transport/stream_socket.py

I've installed python-ndn through source code. However when I ran the example programs, it said that sock file '/run/nfd.sock' was not found while running stream_socket.py.
Here 're the logs:
Traceback (most recent call last):
File "consumer.py", line 58, in
app.run_forever(after_start=main())
File "/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/app.py", line 293, in run_forever
File "/usr/python/lib/python3.6/asyncio/base_events.py", line 467, in run_until_complete
return future.result()
File "/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/app.py", line 249, in main_loop
File "/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/transport/stream_socket.py", line 84, in open
File "/usr/python/lib/python3.6/asyncio/streams.py", line 128, in open_unix_connection
lambda: protocol, path, **kwds)
File "/usr/python/lib/python3.6/asyncio/unix_events.py", line 236, in create_unix_connection
yield from self.sock_connect(sock, path)
File "/usr/python/lib/python3.6/asyncio/selector_events.py", line 451, in sock_connect
return (yield from fut)
File "/usr/python/lib/python3.6/asyncio/selector_events.py", line 456, in _sock_connect
sock.connect(address)
FileNotFoundError: [Errno 2] No such file or directory

Did you run NFD in the background?
On Tue, Dec 22, 2020 at 09:09 ChessNineeee<EMAIL_ADDRESS>wrote:

I've installed python-ndn through source code. However when I ran the
example programs, it said that sock file '/run/nfd.sock' was not found
while running stream_socket.py.
Here 're the logs:
Traceback (most recent call last):
File "consumer.py", line 58, in
app.run_forever(after_start=main())
File
"/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/app.py",
line 293, in run_forever
File "/usr/python/lib/python3.6/asyncio/base_events.py", line 467, in
run_until_complete
return future.result()
File
"/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/app.py",
line 249, in main_loop
File
"/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/transport/stream_socket.py",
line 84, in open
File "/usr/python/lib/python3.6/asyncio/streams.py", line 128, in
open_unix_connection
lambda: protocol, path, **kwds)
File "/usr/python/lib/python3.6/asyncio/unix_events.py", line 236, in
create_unix_connection
yield from self.sock_connect(sock, path)
File "/usr/python/lib/python3.6/asyncio/selector_events.py", line 451, in
sock_connect
return (yield from fut)
File "/usr/python/lib/python3.6/asyncio/selector_events.py", line 456, in
_sock_connect
sock.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/named-data/python-ndn/issues/21, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADMVK47PDF5SESBFPEGPIT3SWCSCBANCNFSM4VFV3QUA
.

Did you run NFD in the background?
…
On Tue, Dec 22, 2020 at 09:09 ChessNineeee @.***> wrote: I've installed python-ndn through source code. However when I ran the example programs, it said that sock file '/run/nfd.sock' was not found while running stream_socket.py. Here 're the logs: Traceback (most recent call last): File "consumer.py", line 58, in app.run_forever(after_start=main()) File "/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/app.py", line 293, in run_forever File "/usr/python/lib/python3.6/asyncio/base_events.py", line 467, in run_until_complete return future.result() File "/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/app.py", line 249, in main_loop File "/usr/python/lib/python3.6/site-packages/python_ndn-0.3a1-py3.6.egg/ndn/transport/stream_socket.py", line 84, in open File "/usr/python/lib/python3.6/asyncio/streams.py", line 128, in open_unix_connection lambda: protocol, path, **kwds) File "/usr/python/lib/python3.6/asyncio/unix_events.py", line 236, in create_unix_connection yield from self.sock_connect(sock, path) File "/usr/python/lib/python3.6/asyncio/selector_events.py", line 451, in sock_connect return (yield from fut) File "/usr/python/lib/python3.6/asyncio/selector_events.py", line 456, in _sock_connect sock.connect(address) FileNotFoundError: [Errno 2] No such file or directory — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#21>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADMVK47PDF5SESBFPEGPIT3SWCSCBANCNFSM4VFV3QUA .

Thanks! I forgot to restart nfd-service after rebooting. It works now.


JUCE plugin ports result in audio glitches and not giving great output

There is most likely some intensive operation that prevents realtime processing. Needs some investigation.

One possible improvement is that other plugin API wrappers are based on AudioPlayHead (e.g. VST3Wrapper and LV2Wrapper) while our current implementation is quite simple and minimum.

moved the issue to aap-juce repo.


SignaturePadView Composable references old callbacks

Describe the bug
Using onSigned callback to override some data object by .copy() I enountered, that an old version of onSigned callback is triggered and therefore overriding other values set in that data object.
To Reproduce
Steps to reproduce the behavior:

Set some value as state
Use SignaturePadView and set state of some data class object

Expected behavior
New instance of data object is used in callbacks
Additional context
Code, that triggers the bug
data class DataContainer(val signed: Boolean = false, val buttonCounter: Int = 0)

@Preview
@Composable
fun BugPreview() {
    val (dataState, setDataState) = remember { mutableStateOf(DataContainer()) }

    Column {
        Text(text = "Counter: ${dataState.buttonCounter}")
        Text(text = "Signed:  ${dataState.signed}")

        Button(onClick = {
            setDataState(dataState.copy(buttonCounter = dataState.buttonCounter + 1))
        }) { Text(text = "+1") }
        Box(
            modifier = Modifier.border(BorderStroke(width = Dimens.gs2, color = Color.Black))
        ) {
            SignaturePadView(
                onSigned = { setDataState(dataState.copy(signed = true)) },
                onClear = { setDataState(dataState.copy(signed = false)) },
                clearOnDoubleClick = true
            )
        }
    }
}

Try this:
@Preview
@Composable
fun BugPreview() {
    var dataState by remember { mutableStateOf(DataContainer()) }

    Column {
        Text(text = "Counter: ${dataState.buttonCounter}")
        Text(text = "Signed:  ${dataState.signed}")

        Button(onClick = {
            dataState = dataState.copy(buttonCounter = dataState.buttonCounter + 1)
        }) { Text(text = "+1") }
        Box(
            modifier = Modifier.border(BorderStroke(width = 2.dp, color = Color.Black))
        ) {
            SignaturePadView(
                onSigned = {
                    dataState = dataState.copy(signed = true)
                },
                onClear = {
                    dataState = dataState.copy(signed = false)
                },
                clearOnDoubleClick = true
            )
        }
    }
}

please reopen this ticket if problem persists.


Adds support for Kibana 7.9

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Test Suites: 27 passed, 27 total
Tests:       129 passed, 129 total
Snapshots:   24 passed, 24 total
Time:        16.655s
Ran all test suites.
✨  Done in 18.33s.

Cannot merge until the backend plugin PR is merged first: https://github.com/opendistro-for-elasticsearch/index-management/pull/283
As this would fail the CI tests after pushing


Obsolete PayPal documentation

Docs » PayPal Payments Standard » IPN/PDT variables
You provide link to obsolete PayPal documentation: https://developer.paypal.com/webapps/developer/docs/classic/ipn/integration-guide/IPNandPDTVariables/
In Django 1.10.5 errors related to obsolete data variables that are no longer returned in the IPN object are not raised or printed in the console by default. To do this I had to add this code in my_app.__int__.py
from django.conf import settings

class ExceptionLoggingMiddleware(object):
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        response = self.get_response(request)
        return response

    def process_exception(self, request, exception):
        if settings.DEBUG:
            print(exception.__class__.__name__)
            return None

and also add to project's settings.py:
MIDDLEWARE = [
    ...
    'my_project.my_app.ExceptionLoggingMiddleware',
]

I'll happily update the docs if you can provide a link to the correct docs.
I do not understand your second point at all.

if an exception occurs, it should crash the app, and normally this will be shown as a full stack trace in the returned page in development, and as a 500 error with a stack trace in the console. This happens in Django 1.10.5. If you are not seeing that, something strange is happening in your project. Do you have some other middleware that is silencing the errors?

I don't understand what you mean about obsolete data variables that are no longer returned in the IPN object.

For future reference, it is helpful if you can file distinct issues separately, thanks.

In paypal.standard.ipn.models.py you use send() function valid_ipn_received.send(sender=self)

send() does not catch any exceptions raised by receivers; it simply allows errors to propagate. Thus not all receivers may be notified of a signal in the face of an error. https://docs.djangoproject.com/en/1.10/topics/signals/
Unfortunately errors related to invalid (obsolete) ipn_obj attributes are not raised / propagated

Unfortunately, I am not aware of any up-to-date PayPal documentation. First I followed the one that is referred in docs and therefore, I expected ipn_obj to return amount value but instead it returns mc_gross that is not documented in referred PayPal documentation.

Here is my MIDDLEWARE list. I added ExceptionLoggingMiddleware at the end so it should be run first as far I can understand, right?
MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'my_project.my_app.ExceptionLoggingMiddleware',
]

I'm afraid I still can't work out what you mean.
On the linked PayPal docs page, both mc_gross field and  amount field ARE present and documented - the latter being labelled as for recurring payments. Neither are listed as obsolete, though they have different purposes, and both are available on the PayPalIPN object, as documented here - http://django-paypal.readthedocs.io/en/stable/standard/variables.html which leads to the page you linked to.
Regarding the suppression of the exception, you are going to have to post the code you are using which is generating the exception, and how you are triggering the exception, including the GET/POST data from the request object (and how you are generating that request).

My error, I am sorry. First I was reading this page Docs » PayPal Payments Standard » Using PayPal Standard IPN. In step number 3. there is a link to PayPal website that should provide "a full list of variables that can be used in paypal_dict".

For a full list of variables that can be used in paypal_dict, see PayPal HTML variables documentation.

...but there is no mc_gross field listed on that PayPal page. I guess this link should point to the page I mentioned in the first post (the same link you have just mentioned above). I can see now how I ended up using amount value when I first wrote my signal receiver function, as the best candidate reflecting amount paid by the buyer.

In my valid_ipn_received signal receiver function I was using ipn_obj.amount (instead of ipn_obj.mc_gross), and because PayPal sent only mc_gross value it was generating exception that was not raised or printed in the console. To replicate this behaviour just use variable that is not returned by IPN. After I changed ipn_ob.amount to ipn_ob.mc_gross value everything works fine.

On number 2, I can't replicate this.
If, in the signal handler function I attach, I access a real attribute on the IPN object for which no value was sent by PayPal, I get the default value as expected e.g.:
print(ipn_obj.amount)

returns
0

In some cases this default value could be None, I haven't checked them all.
If I access a genuinely non-existent field, then I get a stack trace on the console, as expected:
print(ipn_obj.non_existent_field)

produces a stack trace that results in the following on the console:
AttributeError: 'PayPalIPN' object has no attribute 'non_existent_field'

I tested this using the PayPal IPN simulator + ngrok to test in my local development environment. In addition everything else I know about how this works makes it very unlikely that django-paypal works incorrectly here. It's possible you have some strange setup that is silencing errors some how, which I'd recommend you investigate, because otherwise you could have a hard time debugging other things.
The only other thing I can think of is that you are talking about using the console when DEBUG=False In that case no stack trace is printed - this is expected behaviour of Django if DEBUG is False, nothing to do with django-paypal. For development, it's most helpful if you have DEBUG = True, and you shouldn't be using the Django development server at all in production, it's not safe.
I'm going to close this now, I think there must be some confusion. If you can provide enough information for me to pinpoint an error in django-paypal, please re-open.

I added a note about the PayPal docs seeing as they easily confused, and also badly named (e.g. you pass in amount and end up with mc_gross, while the returned amount means something different).


[QA] Upgraded to django 2.0

Coverage remained the same at 100.0% when pulling fea1947152639160736b6c5c6c8f20dfc85af977 on AZtheAsian:master into a799630501cc024b6ca4ffdb48876bb53d3469ac on openwisp:master.

Coverage remained the same at 100.0% when pulling add17f62852924db34a568b8123fb6c8d28f7d23 on AZtheAsian:master into a799630501cc024b6ca4ffdb48876bb53d3469ac on openwisp:master.

Coverage remained the same at 100.0% when pulling 1cfdfcfc8406ae2c96056104b598e45547836567 on AZtheAsian:master into a799630501cc024b6ca4ffdb48876bb53d3469ac on openwisp:master.

Coverage remained the same at 100.0% when pulling 70dbb4520f78197a0a6360ac517d598c71a87d8f on AZtheAsian:master into a799630501cc024b6ca4ffdb48876bb53d3469ac on openwisp:master.

Oh man I forgot to merge this one.


Filtering fails on  without class

I have set up a list with list items that contain the following structure:

	John Doe
	posted an update in the team
	The Substantial Turtles 🐢

In addition, I have a filter <input> that I use to pass a filter string.
I have found that text filtering does not work on the second  element as long as it does not have a class attribute set. Thus, during filtering, the above code will not include the text in the second  element, but if I change the element like so,
posted an update in the team

then the input filter does work on the element. What's going on?

Hi,
Was this problem solved?

Not to my knowledge. For what it's worth, I'd adivise you not to use list.js anymore, at least not in a production environment, because it is not maintained anymore.


Interface initialization change

I don't like this.  While it is simple to do in Python and probably about the same for node.js, it is significantly more complex to handle this in 'C'
self.pg3init = json.loads(
base64.b64decode(os.environ.get('PG3INIT')))
Grabbing the information from STDIN is reasonably easy to do in any language.
What is the quantified change in reliability that prompted this change?

I incorporated both. ENV will be preferred however I added back in STDIN

Thanks!  That will make implementing a 'C' interface library a lot easier.    Still curious why the bas64 encode? Are there plans to have binary data in there?


Run dart_code_metrics on CI and fix current warnings

https://pub.dev/packages/dart_code_metrics
Similar to https://github.com/flutter/flutter/pull/104550

lib/src/client_reports/client_report.dart:
⚠ constructor ClientReport has unnecessary nullable parameters
(this.timestamp)
at /Users/.../Github/sentry-dart/dart/lib/src/client_reports/client_report.dart:8:3
lib/src/enricher/io_enricher_event_processor.dart:
⚠ method _getRuntimes has unnecessary nullable parameters
(List? runtimes)
at /Users/.../Github/sentry-dart/dart/lib/src/enricher/io_enricher_event_processor.dart:54:3
lib/src/http_client/breadcrumb_client.dart:
⚠ constructor BreadcrumbClient has unnecessary nullable parameters
(Client? client, Hub? hub)
at /Users/.../Github/sentry-dart/dart/lib/src/http_client/breadcrumb_client.dart:45:3
lib/src/http_client/failed_request_client.dart:
⚠ constructor FailedRequestClient has unnecessary nullable parameters
(Client? client, Hub? hub)
at /Users/.../Github/sentry-dart/dart/lib/src/http_client/failed_request_client.dart:67:3
⚠ method _captureEvent has unnecessary nullable parameters
(required Object? exception)
at /Users/.../Github/sentry-dart/dart/lib/src/http_client/failed_request_client.dart:149:3
lib/src/http_client/tracing_client.dart:
⚠ constructor TracingClient has unnecessary nullable parameters
(Client? client, Hub? hub)
at /Users/.../Github/sentry-dart/dart/lib/src/http_client/tracing_client.dart:11:3
lib/src/sentry_envelope_header.dart:
⚠ constructor SentryEnvelopeHeader has unnecessary nullable parameters
(this.eventId, this.sdkVersion)
at /Users/.../Github/sentry-dart/dart/lib/src/sentry_envelope_header.dart:7:3
✖ total declarations (functions, methods and constructors) with unnecessary nullable parameters - 7

this package is DISCONTINUED


Freeze on VimResized

Hi, I notice that Snap might cause nvim to freeze for a brief of time (maybe up to 1s) upon terminal resize event, even if Snap is not opened
Disable Snap and the problem is resolved
Hope anyone can shed some light on this

Hmm, definitely a bug. Thanks

Fixed in https://github.com/camspiers/snap/commit/1fbd674dc53401fd643a2bb8bf07dd3e7125d430


Document how to run the benchmarks

It could pretty much be copied from the description on #496 and this comment.
I wrote the code, so it should probably be me.

Already done: https://github.com/sindresorhus/ava/commit/c22eabd717451dbeaeeff1520f399f5a5fd85764 ;)


Please add a screen shot

I'd like to add this to awesome-zsh-plugins, but I prefer to add themes with screen shots so that users can see what they look like without having to install them.
Thanks!

@unixorn Done, ty.


Downcasting AnyPromise to Promise using .toPromise(T)

The motivation behind this addition is described here: https://github.com/mxcl/PromiseKit/issues/388
When working with legacy ObjC parts returning AnyPromises, it's nice to have an option to convert to a typed Promise<T>. This PR enables just this:
LegacyBackend.fetchUser()   // AnyPromise
.toPromise(User)            // Promise<User>
.then { user in
    // Now we can work with user: User
}

I found this very useful when working with large chunks of ObjC code that is otherwise fine and isn't worthy of rewriting to Swift just to get the typed promises.
When the downcasting fails, an error is thrown.

Sorry for the delay on getting to this. I've been reviewing it off and on since submission, and have been trying different approaches to make this better, but I can't find any I like. This seems to be the best implementation we can provide at this time (at least that I have found).
I would however like to make some adjustments to the documentation and maybe add some tests. I'll simply amend/squash them onto your commit before merging if you approve.

Cool. I think the only reason this was not there previously was that it crashed Swift.
However we must have tests verifying this stuff.

I'm glad you found that useful. Feel free to modify it any way you like it, including adding some tests :) I should've added them myself.
I agree that calling .toPromise(T) on top of every "legacy" promise is a bit awkward, but we couldn't find any better (less weird) way either.


Gibberish Instead of Text After Extension Translation

Hello, Pedro Gregorio!
I have encountered the following problem. I wanted to translate the extension into my language for easier use (Russian). I translated the contents of the file \public\popup.html. However, when I imported the modified extension following your instructions, I encountered the following issue:

Hey, @Valeron-bit ! Thank you for reaching out!
Well, now this is an interesting issue. I have no clue how this happened, but I'll definitely take a look, as soon as I can.

@Valeron-bit Yes, that was the case, now it is fixed, I have already made the PR on #71.

@Perseus333 It worked. I'm very grateful to you!

Thank you, @Perseus333 for the snappy fix, and @Valeron-bit for reporting the issue!
Closing this now, will be adding the PR to the upcoming version :)


Feature request: Support configuring project platform version in project config editor

Similar requests

[X] I have searched and found no existing similar requests

How are you using Studio?
Legend Studio
What problems are you trying to solve?
https://github.com/finos/legend-sdlc/pull/492 allows users to specify the platform versions users could use in their pipeline, we should allow users to configure this in Studio project config editor
Describe the solution you would like to see implemented

[ ] Implement this in a separate tab in config editor. See the specs from https://github.com/finos/legend-sdlc/pull/492 to understand the feature and also follow the details below:
[ ] SDLC server defines platform dependencies and versions - http://localhost:7070/api/swagger#/Server/getPlatformsInfo - example result looks like this (NOTE: this is also what specified in legend-sdlc config file)

[
  {
    "name": "legend-engine",
    "groupId": "org.finos.legend.engine",
    "platformVersion": "3.14.1"
  },
  {
    "name": "legend-sdlc",
    "groupId": "org.finos.legend.sdlc",
    "platformVersion": "0.89.1"
  }
]

[ ] When platformConfigurations: null, it means the platform versions are not specified, SDLC will not specify a version for platform dependencies in the POM, but in pipeline, we will use the versions defined by the SDLC server (see above)
[ ] When platformConfigurations is specified, it will be used to update the POM, if let's say we don't specify a version for a particular platform dep, SDLC server will make sure whichever platform dep does not have a version, will be specified a version that the base project structure has (which is quite old), but regardless, the versions specified in platformConfigurations is honored even if it's not valid (i.e. a random string like 12315asdasd or empty)

Describe alternatives you have considered
No response
Documentation, Design, Adoption, Migration Strategy
No response
Contribution

[ ] I would like to work on this feature

When platformConfigurations is specified, it will be used to update the POM, if let's say we don't specify a version for a particular platform dep, SDLC server will make sure whichever platform dep does not have a version, will be specified a version that the base project structure has (which is quite old), but regardless, the versions specified in platformConfigurations is honored even if it's not valid (i.e. a random string like 12315asdasd or empty)

Ideally, in the form, we would like this to be a dropdown (the range/option of versions are shown to users), but if that's too ambitious, we could fallback to just do soft stuff like suggesting the kind of values to be expected via the placeholder for now. I feel that it's too strict to enforce a pattern like x.y.z or ****-SNAPSHOT or so, I'm not sure what the regexp pattern we can use here.
My reasoning is that this is meant for advanced users, they should have some rough idea of what they do here. Now, since they are advanced users, we try our best to suggest to them what we think is the best, we should not error out something like 1.34.5-123123123-asdasd-123asd-dev-snapshot because that could totally be valid version tag for some platform dependency they have in their SDLC instance.


chore(main): release 1.6.3

:robot: I have created a release beep boop
1.6.3 (2023-10-16)
Bug Fixes

🐛 修复打包后图标丢失的问题 (58a66f1)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/viarotel-org/escrcpy/releases/tag/v1.6.3 :sunflower:


Metrics support

I was about to start working on a small crate to export procfs via prometheus using metrics until I saw your crate. Would you be interested in supporting metrics? And if so, how can I help?

Hey, @jean-airoldie! I was experimenting with the metrics crate and Prometheus export too, you can find the sources here, it is broken a little bit (has this issue with counters and absolute values), but fits okay as a proof of concept.
I'm inclined not to integrate metrics into the heim for many reasons:

it would increase code complexity drastically, adding to maintenance burden
it will lead to extra memory allocations; metrics requires either owned Strings or static &str for label pairs, but heim API usually returns references or types, which should be converted into a String manually, ex. &Path or &OsStr
should it expose platform-specific data too?
exposed metric names might not suit everyone needs

On the other hand, we could take this heim-node-exporter, re-brand it maybe and export everything what heim can provide, if you want to join the idea. How does that sound to you?

Yep, that sounds good. The general ideas I had in my mind:

Make a node exporter builder used to configure what metrics are exposed, etc.
We run the exporter as a metrics proxy. This means that we will only run heim when a snapshot is requested (e.g. every 15sec with prometheus).
We could also add a feature that register the proxy automatically with the default settings,
so that a user could simply depend on heim-node-exporter to get access to these metrics.


i18n(fr): Update reference/error-reference.mdx from #7646

Description (required)
Update reference/error-reference.mdx from #7646

Related issues & labels (optional)

Closes #
Suggested label: i18n

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

fr
reference/error-reference.mdx
Localization changed, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.


Default.nix references non-existent version of binary package

@kuznero Can this be closed now?

@rahulmutt yes, that should be ok.


Adds Azure API orchestration support

Codecov Report

Merging #346 (70aeb12) into main (63e0d28) will increase coverage by 0.66%.
The diff coverage is 87.43%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files

@@            Coverage Diff             @@
##             main     #346      +/-   ##
==========================================
+ Coverage   84.23%   84.89%   +0.66%
==========================================
  Files          61       62       +1
  Lines        3502     3589      +87
==========================================
+ Hits         2950     3047      +97
+ Misses        426      416      -10
  Partials      126      126

Impacted Files
Coverage Δ

pkg/testsupport/test_httpclient.go
0.00% <0.00%> (ø)

...chestratorproviders/microsoftazure/azure_client.go
95.85% <95.23%> (+13.39%)
:arrow_up:

...estratorproviders/microsoftazure/azure_provider.go
91.89% <95.23%> (+7.51%)
:arrow_up:

cmd/orchestrator/orchestrator.go
81.25% <100.00%> (ø)

...torproviders/microsoftazure/azure_policy_mapper.go
100.00% <100.00%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 63e0d28...70aeb12. Read the comment docs.


Bug in linked inserts

For linked inserts, it seems like Space Cloud simply uses the schema of the root table for linked documents as well. Need to fix this

Done


Can't Load Pepper Flash on any Electron Version 1.2.5

I had an application using Electron 0.36.8 running Flash perfectly using the code below:
app.commandLine.appendSwitch('--enable-npapi');

var ppapi_flash_path = null;
if(process.platform  == 'win32'){
  ppapi_flash_path = path.join(__dirname, 'pepflashplayer64_22_0_0_192.dll');
} else if (process.platform == 'linux') {
  ppapi_flash_path = path.join(__dirname, 'libpepflashplayer.so');
} else if (process.platform == 'darwin') {
  ppapi_flash_path = path.join(__dirname, 'PepperFlashPlayer.plugin');
}

app.commandLine.appendSwitch('ppapi-flash-path', ppapi_flash_path);
app.commandLine.appendSwitch('ppapi-flash-version', '<IP_ADDRESS>');

I was running version x64 of Electron and, therefore using the x64 version of the pepperflash dll.
Since most of my users still uses x32 computers, I downloaded the newest version of Electron (1.2.5), x32, changed the dll to pepflashplayer32_22_0_0_192.dll and BOOM, no Flash =(
I wasted 2 days looking for informations, forums, sample codes and other Issues and didn't get any solution to the newer version not running Flash. At the end, I downloaded Electron version 1.2.5 x64 and kept my code with pepflashplayer64_22_0_0_192 and it also didn't work.
Same code, same DLL.
Electron 0.35.8 x64 runs it, Electron 1.2.5 x64 doesn't.

OOps, wrong Repository =(
Sorry


I have to re-input the key when I re-open NEXTCHAT every time .

Bug Description
Every time when I open the client, it always raises:
检测到无效 API Key，请前往[设置](tauri://localhost/#/settings)页检查 API Key 是否配置正确

Then I have to re-input the key. But when I re-input the key, it still requires me to re-input it next time when I open it.
It BOTHERS!!
Steps to Reproduce
re-opening it is enough.
Expected Behavior
I wanna it can remember my key.
Screenshots
No response
Deployment Method

[ ] Docker
[ ] Vercel
[ ] Server

Desktop OS
macOS
Desktop Browser
No response
Desktop Browser Version
No response
Smartphone Device
No response
Smartphone OS
No response
Smartphone Browser
No response
Smartphone Browser Version
No response
Additional Logs
No response

Why not try embedding the key in nextchat using environment variables?

I've tried it but can't reproduce it. Can you provide me with specific steps to reproduce it?


golint: comment on exported (method|type) ... should be of the form ...

Fix warnings from golint regarding comments on exported types or methods
$ golint ./... | grep 'comment on exported'
lru_cacher.go:95:1: comment on exported method LRUCacher.GetIds should be of the form "GetIds ..."
lru_cacher.go:124:1: comment on exported method LRUCacher.GetBean should be of the form "GetBean ..."
memory_store.go:15:1: comment on exported type MemoryStore should be of the form "MemoryStore ..." (with optional leading article)
processors.go:7:1: comment on exported type BeforeInsertProcessor should be of the form "BeforeInsertProcessor ..." (with optional leading article)
processors.go:12:1: comment on exported type BeforeUpdateProcessor should be of the form "BeforeUpdateProcessor ..." (with optional leading article)
processors.go:17:1: comment on exported type BeforeDeleteProcessor should be of the form "BeforeDeleteProcessor ..." (with optional leading article)
processors.go:37:1: comment on exported type AfterInsertProcessor should be of the form "AfterInsertProcessor ..." (with optional leading article)
processors.go:42:1: comment on exported type AfterUpdateProcessor should be of the form "AfterUpdateProcessor ..." (with optional leading article)
processors.go:47:1: comment on exported type AfterDeleteProcessor should be of the form "AfterDeleteProcessor ..." (with optional leading article)
session.go:374:1: comment on exported method Session.Conds should be of the form "Conds ..."
statement.go:131:1: comment on exported method Statement.SQL should be of the form "SQL ..."
statement.go:798:1: comment on exported method Statement.Distinct should be of the form "Distinct ..."
statement.go:805:1: comment on exported method Statement.ForUpdate should be of the form "ForUpdate ..."

Thanks


MD030: Allow dynamic configuration for alignment in multi-digit ordered lists

Problem
There's currently no way to configure MD030 to enforce a consistent spacing after the marker of ordered lists, while still allowing me to have a list like this:
1.  Lorem
2.  Ipsum
3.  Dolor
4.  Sit
5.  Amet
6.  Consectitur
7.  Adipisci
8.  Elit
9.  Foo
10. Bar

I could set
MD030:
  ol_single: 2

...which would cause an error in line 10, or
MD030:
  ol_single: 1

...which would accept line 10, but causes an error in all the other lines.
Proposed solution
I'd like there to be an additional setting for MD030's ol_single that would allow me to align the numbered items once the lists expand to double (or triple) digits.
Perhaps something like this:
MD030:
  ol_single: aligned

which would accept both
1. Lorem
2. Ipsum
3. Dolor

and
8.  Elit
9.  Foo
10. Bar

Yes, this would be really nice to have.
I think it would also be important to specify at which level it should be aligned.
E.g.
MD030:
  align_all_at: 3

or making it dependent on indent as e.g. specified in MD007/ul-indent.


How to implement offline storage?

I am testing the Parse SDK for .NET it is awesome.
I would like to implement a data store similar to the other SDKs so that my application can work seamlessly offline. For instance, the Android SDK has Parse.enableLocalDatastore() ParseObject.pinInBackground() and ParseQuery.fromLocalDatastore()
Can you provide some pointers how to start implementing that?
Maybe it was already done somehow? I could not find any info in the documentation.
Thanks,

I would imagine that @cbaker6 may have some design clues, since he has been recently designing the Swift SDK.

I advise against implementing local storage like the other SDKs. The approach I recommend on taking is protocol based allowing developers to add their own local storage. More information is here: https://github.com/parse-community/Parse-Swift/projects/2#card-53117524
Parse-Swift doesn’t have local storage, but I link to an example implementation that uses Parse-Swift with CoreData (sql-lite) to properly keep data in-sync in on/offline scenarios
Of course, if you want to implement it like the other SDKs, I believe the JS SDK has the latest implementation


rustfmt removes GATs from associated type

I tried this code:
#![feature(generic_associated_types)]

trait GAT {
    type R<T>;
    fn do_something<T>(r: Self::R<T>) -> Self::R<T> {
        r
    }

}
struct SomeStruct;

impl GAT for SomeStruct {
    type R<T> = T;
}

impl SomeStruct {
    fn process<T>(v: T) -> <Self as GAT>::R<T>
    // Issue: rustfmt removes the GAT from the associated R
    where Self: GAT<R<T> = T>
    {
        SomeStruct::do_something(v)
    }
}

fn main() {
  let one: u8 = SomeStruct::process(1);
  assert!(one == 1)
}

I expected rustfmt to: keep <T>
where Self: GAT<R<T> = T>

Instead, this happened: removed <T>
where Self: GAT<R = T>

Meta

rustc --version --verbose:
rustc 1.56.0-nightly (2f07ae408 2021-08-05)
binary: rustc
commit-hash: 2f07ae408fce782bf1058e3de808f1b6f9ab60a4
commit-date: 2021-08-05
host: x86_64-unknown-linux-gnu
release: 1.56.0-nightly
LLVM version: 12.0.1

#4911


feat: add rust api for cudaFreeAsync

Describe the changes
This PR add missing cudaFreeAsync API for rust wrappers
(as I personally need it)

@alxiong Thanks for the contribution! 💪🏻 🚀
Can you run our formatter script (https://github.com/ingonyama-zk/icicle?tab=readme-ov-file#development-contributions) and we will review asap

yup, fixing it now, will push soon. Thanks!

the use case is, I want to "warm up" the stream by allocating and deallocating some bytes. and i want this to be non-blocking.

Right. I think any call to the new function must be followed by std::mem::forget as otherwise double free will happen on drop. And forget cannot be called from inside the function as it requires ownership. So the function looks pretty unsafe to use tbh...
Is it possible that we add a special function for the use-case you've described. We can call it warmup or prealloc, it will just allocate and free memory asynchronously given a DeviceContext object. Would that be ok? If so, I can make a quick PR against your branch

We can call it warmup or prealloc, it will just allocate and free memory asynchronously given a DeviceContext object. Would that be ok? If so, I can make a quick PR against your branch

that sounds perfect, feel free to just edit on PR if more convenient.
btw, I want to ask, I should always carry around a DeviceContext rather than a single CudaStream, right?
cuz if I were to mimic the example/msm/main.rs code it's only creating a stream, but never deal with a device context.

https://github.com/alxiong/icicle/pull/1
About DeviceContext - there's just no straightforward way to determine device id given a stream which is why we created this struct. But for the warmup method, I just put a stream there for simplicity.

sure, feel free to merge whenever!


[BUG] - Outdated link in Alex.js embeds

Description

When the bot fires an Alex.js embed, the link in the embed footer is the outdated EddieJaoudeCommunity link. This needs to be updated.
Screenshots

Additional Context

Join Eddie's discord community here

@nhcarrigan If this issue is still open, I have created a pull request: https://github.com/EddieHubCommunity/EddieBot/pull/522

closed with #522


Ansible playbooks for cluster shared storage

Just creating a placeholder issue, until I can work on it: I would like to add a collection of playbook that would add shared storage options prior to the deployment of the cluster. One of the lower hanging items could be Ceph and NFS. Since Kargo already deploys Docker and etcd, storage options could be configured prior, and used as the shared backend for all other services and container volumes. This would increase reliability and production-quality of Kargo. I should be able to start working on this over the weekend or next week (hopefully). This all came from a conversation I had with @ant31. Let me know if you disagree with any of this, or if I overstated anything.

Hello, does this issue relates to https://github.com/kubespray/kargo/issues/312

Related to too #228
I think such service can be  k8s-managed, then kpm is a candidate (to be discuss) to prepare packages and manage resources.
But kargo will have to configure the kubelet plugins before.
we probably want to create new-roles:

network_storage/ceph
network_storage/nfs
network_storage/torus

yes, @ant31 you're right...those are related. I can clean these issues up later today.

I'm close to submitting a PR to provision glusterfs plus k8s support for that (PV and endpoint).

Related https://github.com/kubernetes-incubator/kargo/pull/681

I'm working on https://github.com/heketi/heketi too.
It will provide kinda self-hosted glusterfs in kubernetes.
Also some other projects in the same area are coming, we should wait for them as it will simplify a lot.

Has there been any movement on this at all? It would be great to be able to use Flexvolume plugins (such as flexrex), but even just to be able to talk to my Ceph cluster using the built-in Ceph driver would be good.
The need for persistent storage is definitely there, I'm surprised that this isn't possible with Kubespray yet (unless I'm missing something).


Link to sessions not as expected

The link of sessions has the name of the speaker instead of the name of the session in the URL, e.g. http://2017.fossasia.org/rooms.html#bunnie-huang2217

Using the link to the session shows the session itself hidden behind the top bar.

I will check it out :)


Document LexisML

On the wiki, maybe_? Or gh-pages_?

Done at https://github.com/literallybenjam/LexisML/tree/master/1.0