Shuffle Buffer Filled

When I run the model_main_tf2.py script to train a centernet_resnet50_v1_fpn_512x512 model with a dataset that is of the size 25GB in the tfrecord format.
Is the dataset the main reason for this issue?
the error is
2021-01-30 19:37:30.641972: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-01-30 19:37:30.642004: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-01-30 19:37:41.087656: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-01-30 19:37:41.087854: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-01-30 19:37:41.087870: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2021-01-30 19:37:41.087898: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (kingsman-desktop): /proc/driver/nvidia/version does not exist
2021-01-30 19:37:41.088899: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
WARNING:tensorflow:There are non-GPU devices in tf.distribute.Strategy, not using nccl allreduce.
W0130 19:37:41.089709<PHONE_NUMBER>05600 cross_device_ops.py:1321] There are non-GPU devices in tf.distribute.Strategy, not using nccl allreduce.
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0',)
I0130 19:37:41.089887<PHONE_NUMBER>05600 mirrored_strategy.py:350] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0',)
INFO:tensorflow:Maybe overwriting train_steps: None
I0130 19:37:41.093331<PHONE_NUMBER>05600 config_util.py:552] Maybe overwriting train_steps: None
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I0130 19:37:41.093431<PHONE_NUMBER>05600 config_util.py:552] Maybe overwriting use_bfloat16: False
WARNING:tensorflow:From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/model_lib_v2.py:523: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
W0130 19:37:43.446819<PHONE_NUMBER>05600 deprecation.py:339] From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/model_lib_v2.py:523: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
INFO:tensorflow:Reading unweighted datasets: ['/home/kingsman/deepl/records/train.tfrecord']
I0130 19:37:43.468255<PHONE_NUMBER>05600 dataset_builder.py:163] Reading unweighted datasets: ['/home/kingsman/deepl/records/train.tfrecord']
INFO:tensorflow:Reading record datasets for input file: ['/home/kingsman/deepl/records/train.tfrecord']
I0130 19:37:43.468415<PHONE_NUMBER>05600 dataset_builder.py:80] Reading record datasets for input file: ['/home/kingsman/deepl/records/train.tfrecord']
INFO:tensorflow:Number of filenames to read: 1
I0130 19:37:43.468508<PHONE_NUMBER>05600 dataset_builder.py:81] Number of filenames to read: 1
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
W0130 19:37:43.468611<PHONE_NUMBER>05600 dataset_builder.py:88] num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE) instead. If sloppy execution is desired, use tf.data.Options.experimental_deterministic.
W0130 19:37:43.471664<PHONE_NUMBER>05600 deprecation.py:339] From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE) instead. If sloppy execution is desired, use tf.data.Options.experimental_deterministic.
WARNING:tensorflow:From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.data.Dataset.map() W0130 19:37:43.492702<PHONE_NUMBER>05600 deprecation.py:339] From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.data.Dataset.map()
WARNING:tensorflow:From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py:201: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a tf.sparse.SparseTensor and use tf.sparse.to_dense instead.
W0130 19:37:51.294010<PHONE_NUMBER>05600 deprecation.py:339] From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py:201: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a tf.sparse.SparseTensor and use tf.sparse.to_dense instead.
WARNING:tensorflow:From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/inputs.py:281: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
W0130 19:37:54.907917<PHONE_NUMBER>05600 deprecation.py:339] From /home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/inputs.py:281: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
2021-01-30 19:37:58.317565: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2021-01-30 19:37:58.341385: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency:<PHONE_NUMBER> Hz
2021-01-30 19:38:09.704606: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:177] Filling up shuffle buffer (this may take a while): 68 of 2048
2021-01-30 19:38:18.681284: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:177] Filling up shuffle buffer (this may take a while): 162 of 2048
2021-01-30 19:38:28.579701: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:177] Filling up shuffle buffer (this may take a while): 314 of 2048
2021-01-30 19:38:34.514916: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:230] Shuffle buffer filled.
Traceback (most recent call last):
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/eager/context.py", line 2113, in execution_mode
yield
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 733, in _next_internal
output_shapes=self._flat_output_shapes)
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 2579, in iterator_get_next
_ops.raise_from_not_ok_status(e, name)
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 6862, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input is empty.
[[{{node case/cond/else/_10/case/cond/cond_jpeg/else/_105/case/cond/cond_jpeg/decode_image/DecodeImage}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]] [Op:IteratorGetNext]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "object_detection/model_main_tf2.py", line 113, in
tf.compat.v1.app.run()
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "object_detection/model_main_tf2.py", line 110, in main
record_summaries=FLAGS.record_summaries)
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/model_lib_v2.py", line 566, in train_loop
unpad_groundtruth_tensors)
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/object_detection/model_lib_v2.py", line 344, in load_fine_tune_checkpoint
features, labels = iter(input_dataset).next()
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/distribute/input_lib.py", line 628, in next
return self.next()
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/distribute/input_lib.py", line 632, in next
return self.get_next()
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/distribute/input_lib.py", line 663, in get_next
self._iterators[i].get_next_as_list_static_shapes(new_name))
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/distribute/input_lib.py", line 1619, in get_next_as_list_static_shapes
return self._format_data_list_with_options(self._iterator.get_next())
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/data/ops/multi_device_iterator_ops.py", line 585, in get_next
result.append(self._device_iterators[i].get_next())
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 800, in get_next
return self._next_internal()
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 739, in _next_internal
return structure.from_compatible_tensor_list(self._element_spec, ret)
File "/usr/lib/python3.6/contextlib.py", line 99, in exit
self.gen.throw(type, value, traceback)
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/eager/context.py", line 2116, in execution_mode
executor_new.wait()
File "/home/kingsman/.virtualenvs/deepl/lib/python3.6/site-packages/tensorflow/python/eager/executor.py", line 69, in wait
pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input is empty.
[[{{node case/cond/else/_10/case/cond/cond_jpeg/else/_105/case/cond/cond_jpeg/decode_image/DecodeImage}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]]

@2vin2vin
Please, fill issue template.
Please, share colab link or simple standalone code to reproduce the issue in our environment.It helps us in localizing the issue faster. Thanks!


Windows Executable: PyInstaLive must be properly installed when using the -df argument.

Fill in this template completely. Issues not following this template will be closed and ignored.
Check the boxes below by filling [ ] with an x so it looks like [x].
Use the Preview button to ensure the template is filled in correctly.

[x] I am using the latest version of PyInstaLive: 3.2.0.
[x] I have installed either Python 2.7.x or 3.5+: YOUR VERSION HERE
[x] I have read the README.
[x] I have read the FAQ.
[x] I have checked all existing issues, none of which could solve my issue.

I am using:

[x] Windows 10
[ ] Windows 8.1
[ ] Windows 8
[ ] Windows 7
[ ] Linux (distribution: )
[ ] macOS (version: )
[ ] Other (device & OS name: )

To report a bug, fill in the information below.
Required files
Please attach the log file of the user you were trying to download (if applicable) and the pyinstalive.default.log log file.
If your issue is related to assembling segment files please also include the JSON file and a zipped segment files directory.
If any of these files exceed the Github upload size limit in size please use WeTransfer or a similar service to upload these files.
PyInstaLive information
Run pyinstalive --info and paste its output below. Don't forget to omit your username and password.
Example:
$ pyinstalive --info
---------------------------------------------------------------------------
[I] PYINSTALIVE (SCRIPT V3.2.0 - PYTHON V3.6.3) - 01-02-2019 07:00:17 PM
---------------------------------------------------------------------------
[I] To see all the available arguments, use the -h argument.

[I] PyInstaLive version:        3.2.0
[I] Python version:             3.6.3
[I] FFmpeg framework:           Available
[I] Cookie files:               2 (johndoe.json matches config user)
[I] CLI supports color:         Yes
[I] Command to run at start:    None
[I] Command to run at finish:   None
[I] Config file contents:

    [pyinstalive]
    username = *removed*
    password = *removed*
    download_path = \path\to\downloads
    download_lives = True
    download_replays = True
    download_comments = True
    show_cookie_expiry = False
    ffmpeg_path =
    log_to_file = True
    run_at_start =
    run_at_finish =
    use_locks = True
    clear_temp_files = False

[I] End of PyInstaLive information screen.
---------------------------------------------------------------------------

Command used
Paste the command here that you are running. Don't forget to omit your username and password.
If you can reproduce the problem then, if possible, run the same command again with --verbose prior to uploading the log files.
Example: pyinstalive -u "johndoe" -p "grapefruits" -d "justinbieber"
Behavior
Accurately describe the issue you're experiencing with the script.
Steps to reproduce issue
Specify the exact steps taken to reproduce the problem. If you can't reproduce the issue try to describe the steps you've taken that eventually resulted in the issue you have experienced.

To ask a question, fill in the information below.
Question
Describe your question here as clear as possible.
I get the following error when running -df
PyInstaLive must be properly installed when using the -df argument.
I am using the Windows executable version. I updated Python in case too and the same error.

Should be an easy fix
Currently not able to do any new Windows builds (probably will put one out next week along with some other improvements) but in the meantime you can use -df by installing PyInstaLive via pip instead of using the executable.

-df should now be enabled and working in the latest build


Breakdown large jsx rendering logic into separate components for better readability

https://github.com/Monabbir-Ahmmad/Blogify-Frontend/blob/6c5bec9315d05ea27331a9b0c1cb8072743b858b/frontend/src/components/nav/Navbar.jsx#LL61C1-L140C1

The issues are fixed in the newest release


[MINOR] sync UpsertPartitioner modify of HUDI-1398 to flink/java

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
sync UpsertPartitioner modify of HUDI-1398 to flink/java
Brief change log

Modify UpsertPartitioner in flink/java client

Verify this pull request

*TestUpsertPartitioner#testUpsertPartitionerWithRecordsPerBucket *

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Codecov Report

Merging #2390 (0beb995) into master (da51aa6) will decrease coverage by 42.17%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master    #2390       +/-   ##
=============================================
- Coverage     52.21%   10.04%   -42.18%
+ Complexity     2662       48     -2614
=============================================
  Files           335       52      -283
  Lines         14983     1852    -13131
  Branches       1506      223     -1283
=============================================
- Hits           7824      186     -7638
+ Misses         6535     1653     -4882
+ Partials        624       13      -611

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
100.00% <ø> (ø)
0.00 <ø> (ø)

hudicommon
?
?

hudihadoopmr
?
?

huditimelineservice
?
?

hudiutilities
10.04% <ø> (-59.62%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Updates for deploying the solution

Incorporating guidance in the "Deploy the solution" section for supporting users deploying the updated solution.

@mik-e-kim : Thanks for your contribution! The author, @jomolesk, has been notified to review your proposed change.

Re-submitting PRs. Closing old request.

Updating documentation.


Add "coach mode" to UI for displaying context-sensitive help

To help users understand what different components in the UI do...

This is potentially a huge one that would likely go alongside #619
We can talk more about this.

need to identify aha-moments so we can guide users to them:

@tommywhitehead i've added a couple of possible aha moments in the comment above - is this something you can work with?


Daffio guide docs navigation

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/graycoreio/daffodil/blob/master/CONTRIBUTING.md#commit
[x] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?
[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?
The guide docs sidebar in daff.io isn't organized very well and is basically unreadable.
What is the new behavior?

The guide docs sidebar is organized with nested accordions.
The directory/file structure under docs is better organized. Previously, there were components under docs/shared which only pertained to the guide docs, so all of that was moved under a new docs/guide-docs directory.

Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information
More work will probably need to be done on the accordions in the future to allow for a slimmer accordion-item. Right now, they look a little big with nested accordions.

https://github.com/graycoreio/daffodil/pull/1555 should be reviewed first

There are definitely some issues with the way the sidebar is still, but I think they are all either because I need to do some updates to the README files in the packages themselves or because I need to fix a bug in the daff-nav-accordion-item.

Needs a rebase, but otherwise, LGTM

@damienwebdev Are we going to merge this and then make updates to guides-nav after daff-tree is complete?

@xelaint yes.


unable to parse key: interface conversion

Hello, getting the following error in a brand new 0.26.0 chart install.
It happens when I use the --set global.image.tag , like:
helm install trino-sql -f deployments/k8s/values.yaml  --namespace trino-sql --set global.image.tag=latest deployments/k8s/
Error: INSTALLATION FAILED: failed parsing --set data: unable to parse key: interface conversion: interface {} is nil, not map[string]interface {}

If you're using this chart as a subchart, you'd have to share your umbrella chart. Please provide enough information on how to reproduce this issue.

No umbrella chart. Using the default Chart.yaml file. Except changing v2 to v1:
apiVersion: v1

Please provide complete steps to reproduce the issue. Why are you changing apiVersion to v1?

After some more digging found that the issue is not caused by the chart. Thanks!


4.0.1 keeps throwing offsetOutOfRange error.

Questions?

Check the Troubleshooting / FAQ ?
Search the issues ?

Bug Report
Environment

Node version: 10.13.0
Kafka-node version: 4.0.1
Kafka version: 2.1.0

For specific cases also provide

Number of Brokers: 1
Number partitions for topic: 100

It happens with consumer created without supplying options (so default options).
The code is OK in kafka-node 3.0.1, but after upgrading to 4.0.1 with the same code base and subscribe to a partition whose messages is too old and was cleaned. The consumer infinitely throws offsetOutOfRange error.

Maybe it's same bug with #1179.

Which type of consumer is this?

Consumer, not ConsumerGroup
On Tue, Feb 19, 2019 at 5:06 PM Xiaoxin Lu<EMAIL_ADDRESS>wrote:

Which type of consumer is this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SOHU-Co/kafka-node/issues/1190#issuecomment-465194702,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAiuazE2KqDYurGfazufVvmjG3C7kx7fks5vPCEbgaJpZM4bBuT4
.

I tested more and found it is related to compression. When messages are not compressed (either gzip or snappy), it is ok. When it is compressed, the first message the consumer gets is ok, then it keeps throwing
rror: OffsetOutOfRange
at events.push (/Users/bobzhou/dev/speech_request_service/node_modules/kafka-node/lib/protocol/protocol.js:267:25)
at /Users/bobzhou/dev/speech_request_service/node_modules/async/dist/async.js:3880:24
at replenish (/Users/bobzhou/dev/speech_request_service/node_modules/async/dist/async.js:1011:17)
at /Users/bobzhou/dev/speech_request_service/node_modules/async/dist/async.js:1016:9
at eachOfLimit (/Users/bobzhou/dev/speech_request_service/node_modules/async/dist/async.js:1041:24)
at /Users/bobzhou/dev/speech_request_service/node_modules/async/dist/async.js:1046:16
at _parallel (/Users/bobzhou/dev/speech_request_service/node_modules/async/dist/async.js:3879:5)
at Object.series (/Users/bobzhou/dev/speech_request_service/node_modules/async/dist/async.js:4735:5)
at _decodeFetchResponse (/Users/bobzhou/dev/speech_request_service/node_modules/kafka-node/lib/protocol/protocol.js:244:9)
at /Users/bobzhou/dev/speech_request_service/node_modules/kafka-node/lib/protocol/protocol.js:193:12
at KafkaClient.Client.invokeResponseCallback (/Users/bobzhou/dev/speech_request_service/node_modules/kafka-node/lib/baseClient.js:634:18)
at KafkaClient.Client.handleReceivedData (/Users/bobzhou/dev/speech_request_service/node_modules/kafka-node/lib/baseClient.js:614:10)
at KafkaClient.handleReceivedData (/Users/bobzhou/dev/speech_request_service/node_modules/kafka-node/lib/kafkaClient.js:1340:48)
at Socket. (/Users/bobzhou/dev/speech_request_service/node_modules/kafka-node/lib/kafkaClient.js:785:10)
at Socket.emit (events.js:182:13)
at Socket.EventEmitter.emit (domain.js:442:20)
at addChunk (_stream_readable.js:283:12)
at readableAddChunk (_stream_readable.js:264:11)
at Socket.Readable.push (_stream_readable.js:219:10)

Further more, the consumer only subscribes to one topic on ONE partition. Is it related to #1072?

@bobzsj87 yes it is related to my change. its technically now 'fixed' in some cases, but see: https://github.com/SOHU-Co/kafka-node/issues/1225
While this issue is now fixed, other scenarios are now broken.

@hyperlink @aikar @bobzsj87 I am facing this issue too - Getting an infinite loop in both version 4.0.1 as well as 4.1.1 (did not have this problem with 3.x versions earlier). Please find the screenshot of the same below. Kindly help - this is the only place where I am currently stuck.

@tvvignesh has the consumer been down for a period of time and the committed is out of retention? Does this still happen with new groupid?

@hyperlink: Thanks for your response. Yup, the consumer was shut down for a period of time and the topics had expired. It doesn't happen with new group IDs - I have temporarily come over this by setting the retention period to 1 year.

Yup, the consumer was shut down for a period of time and the topics had expired.

There's nothing kafka-node can do to fetch from those offsets again. If you are using ConsumerGroup it should automatically recover from this issue through outOfRangeOffset configuration and start fetching from the earliest or latest.


fix: 2020/03 npm update dependencies

update too many dependencies

eslinter is outdated. Currently, eslint has an option to cache. so we use eslint directly.
apply eslint-plugin-mocha
apply npm audit fix(27 vulnerabilities required manual review and could not be updated)
drop support node.js < 10 in develop

Hey @yumetodo,
Do you reckon this need a major version bump given that it isn't backwards compatible?
Just to be on the safe side. There might be other packages that depend on flat-cache running deprecated versions of node that would break.

Yes.
I just now noticed that write request node>=10.
https://github.com/jonschlinkert/write/commit/e996f212fc94bc6ae4f338af91aa5972d4c0fea6
This is breaking change.
So, Major version bump is required to follow Semantic Versioning 2.0 spec.

Hey @royriojas, a gentle reminder of this PR, do you think this could be reviewed? Still needs a major bump, just putting it in your radar.

BTW, you should stop using istanbul. it is no longer maintained and 27 remained vulnerabilities comes from this.

$npm audit

                       === npm audit security report ===

                                 Manual Review
             Some vulnerabilities require your attention to resolve

          Visit https://go.npm.me/audit-guide for additional guidance

  Low             Prototype Pollution

  Package         minimist

  Patched in      >=0.2.1 <1.0.0 || >=1.2.3

  Dependency of   istanbul [dev]

  Path            istanbul > handlebars > optimist > minimist

  More info       https://npmjs.com/advisories/1179

found 1 low severity vulnerability in 818 scanned packages
  1 vulnerability requires manual review. See the full report for details.

This seems really nice! Thanks for your work 👍
Small detail but since we are talking about breaking changes and major version for next release, maybe you could change the engines in the "package.json". See: https://docs.npmjs.com/files/package.json#engines
I suggest to change this for:
"engines": {
  "node": ">=10"
}

What do you think @yumetodo ?
Let's hope @royriojas will have the occasion to check and merge this PR and do a new release 😊

hey @SuperITMan
Sorry I didn't had time these days to review this changes. I will review later today

About write update:
In this project, write.sync is an only use case and it was not changed by write's major update.

Hey @royriojas
Would you have a moment to have a look on this PR and maybe merge it + release a new version of flat-cache and flat-entry-cache ? :crossed_fingers:
Thanks for your time :blush:


Segmentation fault for gluon.ndarray.random_*

Environment info
Operating System:
Ubuntu 16.04
Compiler:
[GCC 5.4.0 20160609] on linux
Package used (Python/R/Scala/Julia):
Python
MXNet version:
v0.11
Python version and distribution:
3.5.2
Error Message:
Please paste the full error message, including stack trace.
Segmentation fault (core dumped)
Minimum reproducible example
if you are using your own code, please provide a short script that reproduces the error.
import mxnet as mx
mx.gluon.ndarray.random_uniform(2,10)

This also likely works for any random_* without mandatory args (tested with uniform and exponential). Don't think this is intended behavior.

mx.gluon.ndarray.random_uniform((2,10))

mxnet.gluon.ndarray.random_uniform((2,10)) gives :
mxnet.base.MXNetError: Invalid Parameter format for low expect float but value='(2, 10)', in operator random_uniform(name="", low="(2, 10)")
But, my issue was whether the intended output for not providing shape info was a seg fault?

I share the same reaction to this... This is to be fixed soon by @reminisce

This https://github.com/apache/incubator-mxnet/pull/7874 is supposed to fix the seg fault problem.

Random refactor has been merged thanks to @reminisce . This issue should be fixed now on the master branch. Let us know if it still persists.


Add John's 'Spring Contact' as temporary Soft Contact replacement for revised IE

As part of the revision of the Interaction Engine we've been able to rip out external dependencies on Bullet and LDATs but we need to get soft contact back in the revised system.
We can document potential problems with a pure-Unity solution here if they come up.

John's "Spring Contact" will most likely be the (possibly temporary) hands-through-things replacement for this first-pass at the Interaction Module. We'll need to do more testing but it gets us started.

We may return to the OG Soft Contact implementation or aim for a Unity equivalent as the ultimate solution for this but for now we primarily just need basic functionality here to build out and test the rest of the system.


Does onnxruntime-extensions support inference with only one node such as ‘conv’ or ‘clip’ in the onnx model?

I try to execute one node at a time, this node could be ‘conv’ or ‘clip’, etc. for example. And this node not necessarily be the input node of the model. To be more concrete, I put an example model here in Fig 1. Instead of running the whole model (set the input data for the node ‘input’, and expect to get the result of the node ‘clip’), I want to execute the ‘clip’ node (set the input data for the node ‘clip’, and expect to get the result of the node ‘clip’).

Fig.1
I tried to achieve this by change the parameters of the function ‘run’, such as:
res = sess.run([output_name], {'clip': x})
But get an error: onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:clip.
Seems like this parameter is used for the model with multiple input node, it doesn’t work for the case that you want to choose which node you really want to INPUT the data other than the input node.
Does anyone know how to achieve this? Or if the ORT has such a function?

Hi, ONNX model is a workflow and could only be executed from definite input (in your example, it's input.1） to the output, so you cannot select an arbitrary node as input. To achieve your goal, you can modify the model with onnx helper.
BTW: you can put your issue on onnx and there are more people answer your question.

Thank you. What I am puzzled is that there is a sentence--"Support Python per operator debugging, checking hook_model_op in onnxruntime_extensions Python package" on the project homepage . Its meaning is consistent with my goal. But I did not find the way to execute the individual operator. Can you help me answer it?

Maybe @wenbingl  can help answer the question.

@moonjiaoer , your requirement can be easily satisfied as the following way
new_model = build_model_from_ops(original_model, [op_names])
node_func = PyOrtFunction.from_model(new_model)
outputs = node_func(inputs)
build_model_from_ops hasn't been exposed to be a API, I may add it later.
While hook_model_op tries to solve a challenge problem, on executing on a big ONNX-graph, how can we inspect the inputs/outputs of a certain node on the fly.


Idea: Provide source code for the "boring" parts

I had this idea while I was running the Jam session this week.
What if we add in this repository:

The monolithic implementation
gcloud commands or terraform to create a project in GCP using CloudFunctions and PubSub
Snippets on sending messages and subscribing to events to the PubSub from a CloudFunction

Then we could spend much more time on the "event-driven" architecture and also easily check if our code is indee "microservice ready" by deploying the three services separately in the cloud.
Thoughts?
I can spend some time working on steps 1 and 3 using Python, but I'll need help for step 2.

+1


Ready: pdk schema and nonuniform mock pdk

@parijatm , I started align/schema/pdk
@arvuce22 , I started Nonuniform_mock_pdk. I will add via soon. Let's focus on the metal stack for now.

@arvuce22 , after a second thought, let's not add any FEOL and let's assume that PCell's are available on this PDK. I will prototype a mos generator which only add M1 for transistor pins.
@stevenmburns , this branch should enable us to integrate ADR into ALIGN once we agree on the via. ADR collaterals can be auto-generated from the new PDK schema.

This PR is ready for review. It can generate basic transistor primitives as shown below:

@Yaldiz, Soner @.> @Mukherjee, Parijat
@.> Initially, we had multiple metal widths in
the layers.json but we removed those later.
The current template for nonuniform grids is fine.

   width=[400, 500, 500, 600, 600, 500, 500],

   space=[300, 300, 400, 400, 400, 300, 300]

On Tue, Mar 16, 2021 at 12:09 PM soneryaldiz @.***>
wrote:

@.**** commented on this pull request.
In tests/schema/test_pdk.py
https://github.com/ALIGN-analoglayout/ALIGN-public/pull/571#discussion_r595374872
:

   width=[400, 500, 500, 600, 600, 500, 500],

   space=[300, 300, 400, 400, 400, 300, 300],

Arvind, I believe we agreed on this, please confirm.
One way to handle multiple metal templates (aka metal template instances)
is to define a LayerMetalSet just like LayerViaSet. I deferred this to a
future PR.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ALIGN-analoglayout/ALIGN-public/pull/571#discussion_r595374872,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKTUJYKIW4TR4ITSUS5NOZTTD6GGPANCNFSM4Y7TFLFA
.

Thanks Steve. I disabled this in the integration tests for now.
For PDK testing, I am planning to turn this PDK into a module in the next PR.
For ALIGN testing , the new schema may require revisions in DRC and PEX as well (future PR).


skip KV if there are no secrets for the 'vkv snapshot'

$ vkv snapshot save -d backup1
created backup1
created backup1/kv-project1.yaml
[ERROR] could not read secrets from kv-project2/: no secrets in kv-project2 found.

In vkv 0.6.0, if there are no secrets in the KV, the program quit and stop going through other kv.

Thanks for reporting! #204 should fix it, you might want to try it. I decided to add the --skip-errors flag, which is also available in the vkv export command. So the user is aware that some exported snapshot files might a not be complete, rather than just silently continuing on errors. I hope that does work for you.


tf.hub model with BERT preprocessing can't be loaded

I am trying to reduce the default token length with the tf.hub BERT preprocessing models. The model training and export works fine. However, I can't load the exported model, since the encoder expects a list of RaggedTensors (Colab with a minimal demo example and the error message).
If I use the default sequence length and load the preprocessing and encoder model via the KerasLayer, everything works fine.
As soon as I am running the tokenization explicitly in order to set the sequence length in the following packer step, I am running into the error.
tokenizer = hub.KerasLayer(bert_preprocess.tokenize, name='tokenizer')
segments = [tokenizer(s) for s in input_segments]

When I try to load the model with the custom sequence length with the following code
with tf.device('/job:localhost'):
    exported_model = tf.keras.models.load_model(export_path)

I am receiving this error message:
ValueError: Could not find matching function to call loaded from the SavedModel. Got:
  Positional arguments (2 total):
    * tf.RaggedTensor(values=tf.RaggedTensor(values=Tensor("inputs:0", shape=(None,), dtype=int32), row_splits=Tensor("inputs_2:0", shape=(None,), dtype=int64)), row_splits=Tensor("inputs_1:0", shape=(None,), dtype=int64))
    * False
  Keyword arguments: {}

Expected these arguments to match one of the following 2 option(s):

Option 1:
  Positional arguments (2 total):
    * [RaggedTensorSpec(TensorShape([None, None, None]), tf.int32, 2, tf.int64)]
    * False
  Keyword arguments: {}

From the docs:
The call to tokenize() returns an int32 RaggedTensor of shape [batch_size, (tokens)]. Correspondingly, the call to bert_pack_inputs() accepts a RaggedTensor of shape [batch_size, (tokens)].

What can cause that the tokenize() function only returns a single RaggedTensor instead of a list as expected?
The error is independent from the BERT model variant. I have tried a variety of types, small BERT, BERT, ALBERT. Same result.
Is this a problem with the preprocessing model or with the way I save the model?

@hanneshapke , i changed the below segments of code to save/load the model with no errors :-
# Export the model

export_path = "exported_model/"
save_options = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')
model.save(export_path, include_optimizer=False, options=save_options)

to
# Export the model

export_path = "exported_model/"
save_options = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')
model.save_weights(export_path)

and to load the model(weights) :-
# Load the exported model again ===> Exported model can't be loaded
with tf.device('/job:localhost'):
   # exported_model = tf.saved_model.load(export_path)
     exported_model = tf.keras.models.load_model(export_path)

with
# Load the exported model again ===> Exported model can't be loaded

with tf.device('/job:localhost'):
    # exported_model = tf.saved_model.load(export_path)
    # exported_model = tf.keras.models.load_model(export_path)
      exported_model = model.load_weights(export_path)

Thank you @arghyaganguly for you prompt reply. I can confirm the workflow is working. But it seems like a workaround, would you agree @arghyaganguly ? I was planning to export the saved model via TFX and reuse the model in a different pipeline, therefore I preferred the savedModel format.
Again, thank you very much for your quick reply! Very appreciated.

exported_model = tf.saved_model.load(export_path) instead of exported_model = tf.keras.models.load_model(export_path) works for me and gives a restored object that is callable. Can you confirm that it works for you?

Thank you @kempy for your reply. tf.saved_model.load seems to return a concrete function. In my downstream process where I assemble this and other models in an ensemble, I have trouble using concrete function as a layer in a new tf.keras model.
Have you seen a tf.keras implementation which is using a concrete function as a layer? If that work, this route would work for me. Otherwise I have to fall back to using tf.keras.models.load_model.

You can use hub.KerasLayer(export_path) to restore a SavedModel (from Hub or from a local filesystem) as a Keras Layer that is composable with other TF code.

Also, just to clarify, tf.saved_model.load restores an object with python functions attached that map to specific functions/concrete functions restored from the SavedModel. So the call function on the restored object is mapped to a particular function that corresponds to the call function from the serialized Keras Layer or model that was written.

Thank you again @kempy for the hint regarding hub.KerasLayer. I totally forgot this options for my own savedModels. This workflow works perfectly. I would be happy to close this issue. Thank you all for your fast replies and suggestions.

Closing this following the above commment by @hanneshapke .


LLaVA model error in VLLM through Langchain

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code
from langchain_community.llms import VLLM
llm = VLLM(
model="llava-hf/llava-1.5-7b-hf",
trust_remote_code=True,  # mandatory for hf models
max_new_tokens=128,
top_k=10,
top_p=0.95,
temperature=0.8,
)
OR
llm = VLLM(
model="llava-hf/llava-1.5-7b-hf",
trust_remote_code=True,  # mandatory for hf models
max_new_tokens=128,
top_k=10,
top_p=0.95,
temperature=0.8,
image_input_type="pixel_values",
image_token_id=123,
image_input_shape="224,224,3",
image_feature_size=512,
)
Both the ways of instantiating the VLLM class gives the same error.
Error Message and Stack Trace (if applicable)
llm = VLLM(
[rank0]:           ^^^^^
[rank0]:   File /miniforge3/envs/ipex-vllm/lib/python3.11/site-packages/pydantic/v1/main.py", line 341, in init
[rank0]:     raise validation_error
[rank0]: pydantic.v1.error_wrappers.ValidationError: 1 validation error for VLLM
[rank0]: root
[rank0]:   Provide image_input_type and other vision related configurations through LLM entrypoint or engine arguments. (type=assertion_error)
Description
I am trying to use VLLM through Langchain to run LLaVA model. I am using CPU to run my code.
I am getting this error:
"Provide image_input_type and other vision related configurations through LLM entrypoint or engine arguments. "
I went through the source code of vllm/vllm/engine/arg_utils.py:class EngineArgs:
and passed the Vision Configurations in VLLM class as above. However, I see that, even after setting,
image_input_type="pixel_values" in VLLM class (as seen above), the self.image_input_type
in the EngineArgs class shows value as None.
Name: vllm
Version: 0.4.2+cpu
Summary: A high-throughput and memory-efficient inference and serving engine for LLMs
Home-page: https://github.com/vllm-project/vllm
Author: vLLM Team
Author-email:
License: Apache 2.0
Location: /home/ceed-user/miniforge3/envs/ipex-vllm/lib/python3.11/site-packages/vllm-0.4.2+cpu-py3.11-linux-x86_64.egg
Requires: cmake, fastapi, filelock, lm-format-enforcer, ninja, numpy, openai, outlines, prometheus-fastapi-instrumentator, prometheus_client, psutil, py-cpuinfo, pydantic, requests, sentencepiece, tiktoken, tokenizers, torch, transformers, triton, typing_extensions, uvicorn
Required-by:
System Info
langchain==0.2.7
langchain-community==0.2.7
langchain-core==0.2.12
langchain-text-splitters==0.2.2

I also want to use the llm through VLLM in a RAG pipeline. Does LLaVA though VLLM support multiple images as input (which would be needed in a RAG pipeline)?

any update on this?


V3.0.0

Hi,
I ran the migration tool on quiver_hascode
Would be great if you could merge it soon.
@cbracken

Hi @escamoteur sorry for missing this earlier. Version 3.0.0 has been published.
That said, a long time ago, we had the brilliant idea to attempt to break quiver up into a bunch of packages. In the end, the main quiver package ended up remaining far more popular, so we've decided to focus our efforts there, rather than continue to maintain each library in two places.
Apologies for the disruption. If you have any questions I'm happy to answer them. As a frequent contributor to Flutter and a ton of other projects I recognise you from, sorry to have missed your issue!


[BUG] The plugin interferes with Kanban board creations (see demo)

Nevermind, this was actually templater applying my base template.


Marimo and Pixi

Marimo in the embedded browser is super cool!
For me, there are currently two Problems with how the marimo-extension slows down my user experience significantly.
First, the marimo extension tries to list all marimo apps embedded within the installed pixi-environments of the respective pixi project.
This means, that on my machine, the "Application" Column of the extension stays empty, and I'm not able to select the notebook I would like to work on until the extension has found all marimo notebooks in the pixi-environment. During this time, almost all ressources of my laptop are allocated for vs code.
I don't see a apparent reason for listing all notebooks. I would much prefer only the ones in the local workspace. I think it would be cool, if one could at least access a "exclude" setting, where such behaviour could be altered.
Additionally, I think, this search-process induces another problem: With activated marimo-extension, Pylance starts analyzing the files in the pixi-environment, which it does not do when the marimo-extension is deactivated. This leads to another surge of files apparently being screened.
Setting:

Windows 11 Pro, 10.0.22631
VS Code:

Version: 1.92.0 (user setup)
Commit: b1c0a14de1414fcdaa400695b4db1c0799bc3124
Date: 2024-07-31T23:26:45.634Z
Electron: 30.1.2
ElectronBuildId: 9870757
Chromium: 124.0.6367.243
Node.js: 20.14.0
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22631

Marimo-Extension: v0.4.2
Pylance: v2024.8.1

So, as a fun evening event, I took my first steps with vs code extensions, node.js and all of that ^^
So what I found: Critical are the following lines
The second argument controls the exclude globs. So here, adding **/marimo/** solves my problem of the extension listing the tutorials and especially, reading all the files in the subsequent lines and checking for the app = marimo.App( line.
But I'm wondering if there is any reason to argue against ommitting all folders starting with a dot? **/.*/** or **/.pixi/** would be certainly possible as well, so ommitting all files of all hidden folders or just the pixi environment.
What is your opinion on that? I can draft a PR for that.

I appreciate you debugging this.
Ignoring all **/.*/** or **/.pixi/** would be great. thank you for offering to create the PR


Client library test  problem: the case of the disappearing config file

If one of the client library tests encounters an error, then another distracting error is generated in the teardown fixture saying that it can't remove the haas.cfg file.
It's not obvious what's causing this. Running everything under strace using strace -ff py.test -x -v -s tests/unit/client.py with setup.cfg changed to disabled --boxed shows only one unlink operation against the file -- the one that returns the error that py.test is complaining about.
How did the file get removed if only the test that generated it knew about it?
create_setup() in tests/unit/client.py is where all of this happens.
Starting with the code that's part of #774 should help in debugging.
@SahilTikale

If one of the client library tests encounters an error, then another distracting error is generated in the teardown fixture saying that it can't remove the haas.cfg file.

If the test fails to execute (and not fails as intended) then that error may affect the tear down procedure.
for each test, the create_setup() creates the necessary haas.cfg file, initializes the database, starts servers and populates them. After each test it destroys all of it including the config files and database.

How did the file get removed if only the test that generated it knew about it?

The test does not generate the file, it is generated before executing the test. Correct me if I misunderstood.
I think once the test fails, and before next one starts the fixtures destroys the set up if it can (depending on what the error might be ) and  make a new setup in a new folder.
Let me know if that clarifies the issue.

@henn, what is the exact error? (do you have a travis log you can link to?)

this wouldn't be an issue once #827 is merged since we won't create or remove any config file.


Using workers: flawless in MacOS, stalls out in Ubuntu

As the title suggests, I have a threadripper on my linux distro, so with 30 threads I am actively reading from one DB, manipulating the data, and then saving to another DB.
Before I used the worker threads, I would always send the data between the main thread and workers and the main thread would read/write the DBs.
Are there any gotchas involved with worker threads that I should be aware of? If I run the script on my Macbook it runs flawless but takes much longer because I can only do 4 threads.

Put some thought into a current solution, I noticed in the docs you save the work incase the worker crashes. So I'm going to do the same, and if it completely stalls, I'll have a timer incase the work doesn't finish after "x" number of seconds, kill the worker and try again.


Migrate nrpl to the new repo. The old one is unsupported for a long time

I'll give @wheineman a chance to protest this for a few days (just in case).

Agreed!

Okay. Going to merge this. @wheineman please let us know if you want the listing to be changed back to your repo.


book need to update accroding to the template build command change

the recent template commit: https://github.com/aya-rs/aya-template/commit/5811d6ff5687ae115cc8876ddf511dd01c02539b make the getting started guide command cargo xtask build-ebpf not working.
need to update the book and example code  to reflect this change.
the build and running command should be:
cargo build

RUST_LOG=info cargo run -- --iface enp5s0

Can you send a pull request?

Note that the book doesn't mention the template so it is technically not broken. But we should update the book examples to the latest template.\


"prometheus_remote_storage_samples_pending" is missing for prometheus.write.queue

What's wrong?
We're using prometheus.write.queue for a persistent metrics gathering and sending them to a remote storage once the device is online (as in, has internet).
We want to have an overview whether Alloy is successfully able to send all of the metrics and to have a metrics that shows how many samples are there that are collected but not yet sent to a remote storage (important to note that this metrics shouldn't be a counter which means how many samples were gathered and saved to WAL, but a real number or estimate at least of samples that were saved to WAL, but not sent, in order to properly see it even when Alloy restarts).
For us it's crucial as we need to understand how much of the samples are scraped, but not yet sent, as we plan to use it as a base for alerting and observation.
The docs are saying that there's this metric:

prometheus_remote_storage_samples_pending (gauge): The number of samples pending in shards to be sent to remote storage.

and it seems to be something that we want, but apparently it's somehow not being returned by Alloy meta-monitoring (I cannot find it in Grafana using Prometheus this Alloy is writing to as a datasource).
Is it supposed to be there?
If not I suggest to update the docs to remove it and add a metric that displays the amount of samples that are stored, but not yet sent (probably a gauge as it also can decrease, unlike counter).
If yes, should there be any additional steps to enable it? (Probably need to put it into docs if so).
Steps to reproduce

Run Alloy with the config similar to what I provided below.
See no prometheus_remote_storage_samples_pending metric being written into Prometheus.

System information
No response
Software version
1.5.0
Configuration
logging {
  level  = "info"
  format = "logfmt"
}

// System metrics, from node_exporter
prometheus.exporter.unix "node_exporter" { }

// Alloy built-in metrics
prometheus.exporter.self "alloy" { }

// CAdvisor
prometheus.exporter.cadvisor "cadvisor" {
  docker_host = "unix:///var/run/docker.sock"
}

// Metrics scrape configuration
prometheus.scrape "node_exporter" {
  targets = array.concat(
    // Scraping node_exporter
    prometheus.exporter.unix.node_exporter.targets,
    // Scraping Alloy built-in metrics
    prometheus.exporter.self.alloy.targets,
    // Scraping CAdvisor metrics
    prometheus.exporter.cadvisor.cadvisor.targets,
  )

  scrape_interval = "60s"
  honor_labels    = true

  // Sending these scraped metrics to remote Prometheus via prometheus.write.queue.
  forward_to = [prometheus.write.queue.default.receiver]
}

prometheus.write.queue "default" {
  endpoint "default"{
    url = env("PROMETHEUS_HOST")
    bearer_token = env("PROMETHEUS_TOKEN")
  }

  // Keep 1 week of data, in case it wasn't sent.
  // More on WAL and its internals:
  // https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.remote_write/#wal-block
  ttl = "168h"

  persistence {
    batch_interval = "10s"
  }
}

Logs

@mattdurham I guess you're the person to ask on that? can you chime in?

At the moment this would be a documentation issue. Will submit a PR to remove that. Right now checking the in vs out timestamp is likely the best rate. Adding a more bespoke metric probably makes sense, but would need to figure out how to not make it to chatty.

Yeah we'll use the metric that you suggested for now, but ideally it should be something that reflects how much samples there are that are not yet sent to the remote storage.
So just stating that this is a metric that we want to be available and we need it (and likely we won't be the only ones who need it), so would be lovely if that would be implemented at some point.
Also I am not yet sure if the samples are always sent in order (as in, the older sample always gets sent first than the newer one)? If not, the timestamps difference metric might report the wrong results I guess.

The guarantee is that for a given series they are sent in timestamped order, in practice the timestamps are close enough. Generally I would consider anything under a minute lag time as good, though you could likely shave that a bit lower depending on your use case.
I concur that it would be good to have though. If you want to submit a PR it would be welcome github.com/alloy/walqueue, I will be unavailable for a good chunk of the rest of the year but if its still outstanding when I get back I will code it in.


Automate authenticator test case 4

Automates the manual test steps for server trust prompts in the Authenticator.
Note these tests differ in a couple of ways to the manual tests:

The manual tests verify behavior when putting the app in the background and returning to it, and also when rotating the device. In these tests we instead use a StateRestorationTester, which works slightly different (it only simulates disposing and recreating the composition) but vastly simplifies the tests. If we decide in the future that we need to do the bigger tests, we can add them then.
The manual tests have a step for tapping outside of the alert dialog to dismiss. This behavior works with the authenticator but it is very difficult to test and is functionally the same as pressing the back button, so I've excluded that test.

Thanks @eri9000 @shubham7109 @gunt0001 for the reviews


Get the PS1 Demo Working.

Vissova claims that the PS1 demo does not work with my project.
This means potentially that there needs to be enhancements to my code in order to get these files readable.

This might not be a bug. I eventually figured out that I made an accident with my build system.
Basically, I forgot that I commented out the line that compiles the FCMissionReader. So, the program that I gave to Vissova might be an earlier version without proper PS1 support.
However, v0.0.7 now holds the newer program. Which means that this map might be readable to Vissova now.

Vissova had managed to read them.
Therefore as far as I know, the PS1 Demo is supported.


chore(deps): bump dependencies

:tada: This PR is included in version 1.0.0-next.12 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


feat: FW-20 Push docker image on release

Description
These changes introduce a couple of new workflows:

A workflow to generate a release on each merge to main.
A workflow to push a Docker image into the Github repository on release.

Therefore, every time we merge into main, a new release and Docker image will be generated.
Also testing the Docker image can be built on each PR and merge to main.

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Mindmap

Did you make one for this app already?

Kinda. I created a high level flow diagram when was discussing ideas with Christo. I need to add more detail/formalize it and pass it along. Will work on this.

Nws, an alternative I learned from Erik is: "what is the ideal end user's journey" - you could do that too ...

Phase 1 complete - messages are created by tollway traffic generator and pushed to pubsub message queue
Phase 2 - process messages using Cloud Functions
Phase 3 - process messages using Cloud Run
Processing should look very similar in nature for both Phase 2 and Phase 3.

A few attributes will be taken from the message(s) and stored in redis/in memory datastore to serve high-level analytics in a visualization app. Still deciding on analytics but initial thoughts would be something like: most popular tollway, states with the highest amount of tollway traffic, most popular vehicle model across tollways, etc.
All attributes will be stored in BigQuery. Plan to design a few tables to help facilitate.

Nice, great work. Wonder if it would be beneficial to check in once more on this with Christo, say next month? (if he is available of course)

here's an updated mind map. What I'm trying to show is I'll use Cloud Function to process/route messages from Pub/Sub to redis and Cloud Storage. Then going to use Cloud Scheduler (cron job) to run once every few days/kick off Cloud Run/containers to process all messages/move all messages in Cloud Storage to BigQuery. While I'm referencing visualization still, I'm not going to build it out.

Interesting, curious to hear what Christo had to say ...

I still need to meet with him. Planning to message him today.


Need Help: Reading from Json an inserting to collection

Hi,
Im getting the following error while trying to read from json and inserting into Db
below is code:
            openLoadDef.FileName = "";
            openLoadDef.ShowDialog();
            string errordef = openLoadDef.FileName;
            List<BsonDocument> docs = new List<BsonDocument>();
            StreamReader r = new StreamReader(errordef);
            var json = r.ReadToEnd();
            var jsonObj = JsonSerializer.Deserialize(json);
            docs = BsonMapper.Global.ToDocument(jsonObj);
            using (var db = new LiteDatabase("ErrorMsg.db"))
            {
                db.GetCollection("ErrorMsg").InsertBulk(docs);
            }

I'm getting the error at BsonMapper
Cannot implicitly convert type 'LiteDB.BsonDocument' to System.Collections.Generic.List<LiteDB.BsonDocument>'
Can you please advise whats needs to be changed.
Input is a plain json file.

Is your file has a single JSON document or an JSON ARRAY of documents?.

Hi David,
The file looks like below saved as error.json
{
"MessageCode" : "SBL-GEN-00010",
"MessageDescription" : "Siebel Internal Error",
"MessageSolution" : "Resolution Undocumented"
},
{
"MessageCode" : "SBL-GEN-00011",
"MessageDescription" : "Internal: invalid arguments",
"MessageSolution" : "Resolution Undocumented"
},
{
"MessageCode" : "SBL-GEN-00012",
"MessageDescription" : "Internal: insufficient buffer space provided",
"MessageSolution" : "Resolution Undocumented"
}

It's no a valid JSON file.... try add [ ] to transform into an array and than do a for-each to insert into database

you mean like
[{
"MessageCode" : "SBL-GEN-00010",
"MessageDescription" : "Siebel Internal Error",
"MessageSolution" : "No solution is documented at this time"
},
{
"MessageCode" : "SBL-GEN-00011",
"MessageDescription" : "Internal: invalid arguments",
"MessageSolution" : "No solution is documented at this time"
},
{
"MessageCode" : "SBL-GEN-00012",
"MessageDescription" : "Internal: insufficient buffer space provided",
"MessageSolution" : "No solution is documented at this time"
}]
But wanted to try InsertBulk .. please help..

Hi David,
Im not able to convert the BsonValue (jsonObj) to BsonDoc (Docs), tried to loop the Obj but wasn't working. Please help

BsonDocument inherits BsonValue so you can cast or call .AsDocument property.
var array = JsonSerializer.DeserializeArray("[ { ... }, { .... }, ... ]"); // here an array
var docs = array.Select(x => x.AsDocument).ToList(); // here a document list (not array);

Hi! With the objective of organizing our issues, we are closing old unsolved issues. Please check the latest version of LiteDB and open a new issue if your problem/question/suggestion still applies. Thanks!


OCP 4 fails to build  (manifest unknown)

Trying use a chained build which works on on OpenShift 3.11, but fails on OpenShift 4.0. To reproduce a simple case, I used the project shown in lholmquist/react-web-app and followed the instructions.
find . | grep openshiftio | grep application | xargs -n 1 oc apply -f
oc new-app --template react-web-app -p SOURCE_REPOSITORY_URL=https://github.com/lholmquist/react-web-app

It looks like it's trying to pull its own target image before anything exists yet.  The result on OpenShift 4 is:
$ oc logs react-web-app-builder-1-build

Caching blobs under "/var/cache/blobs".
Pulling docker://nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
Getting image source signatures
Copying blob sha256:aeb7866da422acc7e93dcf7323f38d7646f6269af33bcdb6647f2094fc4b3bf7
Copying blob sha256:9291e350be08c93d4e48286db9e72d9c74c7cba5778729d167220c664656d314
Copying blob sha256:4d3620628f7cc3d26ed80469f503f521a9939c2aaac68ec1a1bb4076d92b77e8
Copying blob sha256:94d526a435e926d08aaf39d22731d1d58935203409978a2a7eb170fb76a9d4d8
Copying blob sha256:2603cb6c0d494298779d11c9f9888824a121946162ad28397377d47738b15f5e
Copying blob sha256:27d649e09bdba6830975348259572dddf16bf870b5e685f311008fe117d40baf
Copying blob sha256:d5957af9f715def3f63d1dae0583783e9fbd541a47dfddfe833d60f15283e004
Copying blob sha256:8a88affce9a736bb5e77a2dac18218488ba692a01e8a41f27d3e3b9ebf8ace22
Copying blob sha256:74fd737edd185d29986788f4045004c58c7e7ac4fff1008d3c5d326ebb001abe
Copying config sha256:5e2129220bb621b9254ee6b5fbe7e2eee59e7bc1fc05846d5287748afe854351
Writing manifest to image destination
Storing signatures
5e2129220bb621b9254ee6b5fbe7e2eee59e7bc1fc05846d5287748afe854351
Generating dockerfile with builder image nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
STEP 1: FROM image-registry.openshift-image-registry.svc:5000/cchase/react-web-app-builder:latest AS cached
error: build error: error creating build container: Error determining manifest MIME type for docker://image-registry.openshift-image-registry.svc:5000/cchase/react-web-app-builder:latest: Error reading manifest latest in image-registry.openshift-image-registry.svc:5000/cchase/react-web-app-builder: manifest unknown: manifest unknown

My version of openshift:
$ oc version
oc v4.0.0-0.139.0
kubernetes v1.11.0+c69f926354
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://***:***
kubernetes v1.12.4+f39ab668d3

On the FIRST build in OpenShift, there's a similar error ("pulling image error"), but it doesn't fail the build.
OpenShift 3.11:
Cloning "https://github.com/lholmquist/react-web-app.git " ...
	Commit:	244e2ff913b0aa151a00c53f0eb6393dcf3bffe9 (chore: update the template to use the nodeshift namespaced builder image (#2))
	Author:	Lucas Holmquist<EMAIL_ADDRESS>	Date:	Fri Oct 26 10:02:39 2018 -0400
Using nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038 as the s2i builder image
Pulling image "<IP_ADDRESS>:5000/rwa/react-web-app-builder:latest" ...
pulling image error : Error: image rwa/react-web-app-builder:latest not found

# ... more successful build log stuff ...
Push successful

If I can help with this any way I can, let me know.  I haven't written my own s2i builder image before, so I'm not clear on why it would need to pull an image before it's been builts.

I think the error that you see on the 3.11 example, is because the application is set to use incremental builds,  https://github.com/lholmquist/react-web-app/blob/master/.openshiftio/application.yaml#L92. So I think when you do a second build it might go away.   Can you try removing that line and see what happens?   maybe try that with the openshift 4 tests
Not sure this is a problem with the image, though.  But i haven't tested on Openshift 4 yet

Thanks for the quick response.  Progress!
New error now, though
Caching blobs under "/var/cache/blobs".
Pulling docker://nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
Getting image source signatures
Copying blob sha256:aeb7866da422acc7e93dcf7323f38d7646f6269af33bcdb6647f2094fc4b3bf7
Copying blob sha256:9291e350be08c93d4e48286db9e72d9c74c7cba5778729d167220c664656d314
Copying blob sha256:4d3620628f7cc3d26ed80469f503f521a9939c2aaac68ec1a1bb4076d92b77e8
Copying blob sha256:94d526a435e926d08aaf39d22731d1d58935203409978a2a7eb170fb76a9d4d8
Copying blob sha256:2603cb6c0d494298779d11c9f9888824a121946162ad28397377d47738b15f5e
Copying blob sha256:27d649e09bdba6830975348259572dddf16bf870b5e685f311008fe117d40baf
Copying blob sha256:d5957af9f715def3f63d1dae0583783e9fbd541a47dfddfe833d60f15283e004
Copying blob sha256:8a88affce9a736bb5e77a2dac18218488ba692a01e8a41f27d3e3b9ebf8ace22
Copying blob sha256:74fd737edd185d29986788f4045004c58c7e7ac4fff1008d3c5d326ebb001abe
Copying config sha256:5e2129220bb621b9254ee6b5fbe7e2eee59e7bc1fc05846d5287748afe854351
Writing manifest to image destination
Storing signatures
5e2129220bb621b9254ee6b5fbe7e2eee59e7bc1fc05846d5287748afe854351
Generating dockerfile with builder image nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
STEP 1: FROM nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
STEP 2: LABEL "io.openshift.build.commit.author"="Lucas Holmquist<EMAIL_ADDRESS>"io.openshift.build.commit.date"="Fri Oct 26 10:02:39 2018 -0400" "io.openshift.build.commit.id"="244e2ff913b0aa151a00c53f0eb6393dcf3bffe9" "io.openshift.build.commit.ref"="master" "io.openshift.build.commit.message"="chore: update the template to use the nodeshift namespaced builder image (#2)" "io.openshift.build.source-location"="https://github.com/lholmquist/react-web-app" "io.openshift.build.image"="nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038"
STEP 3: ENV OPENSHIFT_BUILD_NAME="react-web-app-builder-1" OPENSHIFT_BUILD_NAMESPACE="web-game-demo" OPENSHIFT_BUILD_SOURCE="https://github.com/lholmquist/react-web-app" OPENSHIFT_BUILD_REFERENCE="master" OPENSHIFT_BUILD_COMMIT="244e2ff913b0aa151a00c53f0eb6393dcf3bffe9" OUTPUT_DIR="build"
STEP 4: USER root
STEP 5: COPY upload/src /tmp/src
STEP 6: RUN chown -R 1001:0 /tmp/src
time="2019-02-27T20:15:36Z" level=warning msg="pkg/chroot: error unmounting \"/tmp/buildah683737400/mnt/rootfs\": error checking if \"/tmp/buildah683737400/mnt/rootfs/sys/fs/cgroup/perf_event\" is mounted: no such file or directory"
time="2019-02-27T20:15:36Z" level=warning msg="pkg/bind: error unmounting \"/tmp/buildah683737400/mnt/rootfs\": error checking if \"/tmp/buildah683737400/mnt/rootfs/sys/fs/cgroup/perf_event\" is mounted: no such file or directory"
STEP 7: USER 1001
STEP 8: RUN /usr/libexec/s2i/assemble
ls: cannot access /tmp/src/node_modules: No such file or directory
---> Installing application source
---> Building your Web Application from source
Current git config
<EMAIL_ADDRESS>url.https://.insteadof=ssh://
<EMAIL_ADDRESS>core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://github.com/lholmquist/react-web-app
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
---> Installing dependencies
---> Using 'npm install'

subprocess exited with status 1
subprocess exited with status 1
error: build error: error building at step {Env:[OUTPUT_DIR=build OPENSHIFT_BUILD_COMMIT=244e2ff913b0aa151a00c53f0eb6393dcf3bffe9 OPENSHIFT_BUILD_REFERENCE=master OPENSHIFT_BUILD_SOURCE=https://github.com/lholmquist/react-web-app OPENSHIFT_BUILD_NAMESPACE=web-game-demo OPENSHIFT_BUILD_NAME=react-web-app-builder-1 PATH=/opt/app-root/src/node_modules/.bin/:/opt/app-root/src/.npm-global/bin/:/opt/app-root/src/bin:/opt/app-root/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin SUMMARY=Platform for building Modern Web Applications that use Node.js DESCRIPTION=Web Application building with Node.js STI_SCRIPTS_URL=image:///usr/libexec/s2i STI_SCRIPTS_PATH=/usr/libexec/s2i APP_ROOT=/opt/app-root HOME=/opt/app-root/src BASH_ENV=/opt/app-root/etc/scl_enable ENV=/opt/app-root/etc/scl_enable PROMPT_COMMAND=. /opt/app-root/etc/scl_enable NODEJS_SCL=rh-nodejs8 NPM_RUN= NODE_VERSION=10.15.1 NPM_VERSION=6.4.1 NODE_LTS=false NPM_CONFIG_LOGLEVEL=info NPM_CONFIG_PREFIX=/opt/app-root/src/.npm-global NPM_CONFIG_TARBALL=/usr/share/node/node-v10.15.1-headers.tar.gz DEBUG_PORT=5858 OUTPUT_DIR=build NPM_BUILD=npm run build DEPLOY_PORT=8080       OPENSHIFT_BUILD_NAME=react-web-app-builder-1 OPENSHIFT_BUILD_NAMESPACE=web-game-demo OPENSHIFT_BUILD_SOURCE=https://github.com/lholmquist/react-web-app OPENSHIFT_BUILD_REFERENCE=master OPENSHIFT_BUILD_COMMIT=244e2ff913b0aa151a00c53f0eb6393dcf3bffe9 OUTPUT_DIR=build] Command:run Args:[/usr/libexec/s2i/assemble] Flags:[] Attrs:map[] Message:RUN /usr/libexec/s2i/assemble Original:RUN /usr/libexec/s2i/assemble}: exit status 1

I've finally setup openshift 4, and i'm getting the same thing.  Will investigate

i actually did this to create the app:  oc new-app nodeshift/centos7-s2i-web-app:latest~https://github.com/lholmquist/react-web-app

i tried using yarn instead of npm by setting YARN_ENABLED=true and everything worked fine.  So that narrows things down now to something happening during the build when using npm.  I think this line is the culprit, https://github.com/nodeshift/centos7-s2i-web-app/blob/master/s2i/assemble#L58,  but not really sure why

I updated the npm install to do npm install -d to get a bit more info
for this error:
....
npm timing action:finalize Completed in 1416ms
npm timing npm Completed in 30681ms
npm ERR! cb() never called!

npm ERR! This is an error with npm itself. Please report this error at:
npm ERR!     <https://npm.community>

So it isn't even making it to the npm run build stuff yet.
I even tried removing the package-lock.json to see if that was it, but nope

I tried running just the nodejs image with the rest-http-booster
oc new-app nodeshift/centos7-s2i-nodejs:latest~https://github.com/nodeshift/nodejs-rest-http, which worked fine.
So then i used that nodejs image with the react-app, and i got the same error as before.  so it looks like something is happening during the install

@cfchase i haven't had time to look at his since the last updated i made, but i'm planning on getting back into it

Thanks for your efforts.

little update here.  I tried running this with the node 8.x version(which we stopped making and lives at bucharestgold/centos7-s2i-web-app)
That is using npm 5.6.0,  not sure why it fails on node 10.
the hunt continues

Out of curiosity for a workaround, is there any way to run these chained s2i builds locally using the command line?
s2i build ${REPO} nodeshift/centos7-s2i-web-app:10.x ${BUILDER_IMAGE}
s2i build ${BUILDER_IMAGE_ARTIFACT} centos/nginx-112-centos7 ${RUNTIME_IMAGE}

@cfchase i'm not sure how it would work exactly if running things locally, but you could just run the "npm build" locally, then tell the nginx image to use the resulting build folder.
it sort of defeats the purpose though

Another update:  I used npm 5.6.0 with node 10 and i was able to get things to work.  so perhaps it is an issue with the 6.4.x(?) npm version?

i tried with the 11.x version of this image,  and that worked, since it is using npm 6.7.0

Some updates:
One of the tests i did, was to get a generic(using the web app builder image) pod deployed, then entered the terminal, and ran the "create-react-app" process, and then doing an npm install on that new project. This surprisingly didn't fail, which I thought was odd.
After doing a little bit of digging in the npm community forums, i stumbled across this thread, https://npm.community/t/crash-npm-err-cb-never-called/858/62, which basically said to try the latest version of npm since there had been some changes to how errors were reported.
I tried then running the image with npm 6.6.0 and instead of getting a generic error, i got an error about "EMFILE: too many open files", which relates to the ulimit value. I changed the image to print out that value(ulimit -n) when the build ran, and it was set to "1024", but in the deploy pod, it was set to "1048576". That 1048576 number was what the "ulimit -n" was set to in my 3.11 minishift for the build phase.
So it seems we need to update this value, which can't actually be done during our assemble script. I'll need to ask someone from the openshift team, if and how it is possible to increase that limit

closing since i think this is resolved with a newer version of npm that is out


Misuse of alias()?

I noticed that while creating syntax queries for hive statements.
I found that some uses of alias() (see https://tree-sitter.github.io/tree-sitter/creating-parsers#the-grammar-dsl) appear questionable to me, and I wanted to check with you.
1. Doubts: (identifier) for builtin functions
In both the following builtin functions, the function name keyword is replaced in the AST with identifier. I just wanted to check if that's really intentional?
Isn't an (identifier) rather identifying a application-domain-specific item, such as a table or field - rather than a builtin SQL feature keyword?
To my mind, not only (identifier) is wrong in this case, but the entire field('name', ...) should just be replaced by $.keyword_cast and $.keyword_count. However, field('name', ...) gives compatibility with the generic (invocation) (https://github.com/DerekStride/tree-sitter-sql/blob/393e0d35aad922a3bf4faa4ceccad2e401f7b2ab/grammar.js#L1378-L1381), where a field is necessary to extract the function name -- so it's probably only the (identifier) alias that is to be removed.
1.1. CAST
https://github.com/DerekStride/tree-sitter-sql/blob/393e0d35aad922a3bf4faa4ceccad2e401f7b2ab/grammar.js#L1341-L1350
1.2. COUNT
https://github.com/DerekStride/tree-sitter-sql/blob/393e0d35aad922a3bf4faa4ceccad2e401f7b2ab/grammar.js#L1357-L1362
2. Misuse? String literals as (identifier)
https://github.com/DerekStride/tree-sitter-sql/blob/393e0d35aad922a3bf4faa4ceccad2e401f7b2ab/grammar.js#L980-L988
Shouldn't these rather be alias($._literal_string, $.literal)?
';', '"', '\n' and '/path/data' in the Hive CREATE TABLE test end up as identifier tree item, while they clearly are literals:

See also LOCATION under https://spark.apache.org/docs/latest/sql-ref-syntax-ddl-create-table-hiveformat.html#parameters

(I'm working on a small PR.)

However, field('name', ...) gives compatibility with the generic (invocation) (

You guessed correctlty, I initially setup count to be compatible with invocation but I think you're right that we shouldn't alias it to an identifier and instead expose the keyword named node 👍
The rest of the changes look good to me too


Possible Bug: Cannot empty buckets from furnaces with any hopper

Hello! Thank you for taking the time to make this cool mod! I discovered an issue with it today, however.
TL;DR: This mod (or possibly a dependency) prevents hoppers from removing empty buckets from furnaces. It causes vanilla hoppers to break as well.
When using lava to fuel a furnace, the empty bucket is being left in the fuel slot, and it can't being removed by a hopper placed under the furnace. This prevents automatic loading of fuel with hoppers. It doesn't matter if I use Omni-hoppers or Vanilla hoppers.
I have tons of mods enabled, so the first thing I did was install fresh instances of 1.21 Vanilla and 1.21 Fabric with no additional mods. The furnace/hopper combo worked as expected with both of those installs, removing empty buckets from the furnace so the next could be loaded with a hopper from side. Then I started the process of disabling mods from my highly-modded installation one at a time to find the culprit. Once I disabled only Omni-Hopper, my hoppers started removing the empty buckets.
I'm using Modrinth and I tried both versions that were published on June 26th. They have identical names but one says it's for 1.21 while the other 1.20.5, both have Changelogs that say "Updated to 1.21," and they have different file sizes. I also noticed that the "Supports: Fabric 1.21" version does not depend on Cooldown Coordinator. I'm not sure which version is actually ideal for 1.21, but both of them suffer the same issue for me. I tried another fresh 1.21 install with just Fabric, Cicada, and Omni-Hopper, but no luck.
Thank you for taking the time to read this!

Seems like a duplicate of https://github.com/enjarai/OmniHopper/issues/23

Sigh, I spent about an hour on this making sure I tested everything so I wouldn't waste your time with a useless bug report, but then I ended up doing it anyway. I only saw the open tickets here, not the closed ones. My apologies!
The gamerule, of course, did the trick, but I was unaware of its existence. Could you possibly add it to the mod description or put it in the configuration for the mod? Those were the first things I thought to check, as well as any possible config file. It might prevent you from having to deal with dummies like me in the future.
And lastly, do you have a suggestion regarding the same file names for possibly different versions? They both load and work fine for my 1.21 installation, but I wonder which one I should be using since they both say "Updated to 1.21" and have different dependencies and version support listed.


make: Ensure that checker.DeepEquals is used everywhere

Cilium defines its own DeepEquals checker in pkg/checker which gives a
more detailed trace than DeepEquals checker from gopkg.in/check.v1.
Signed-off-by: Michal Rostecki<EMAIL_ADDRESS>

This change is

test-me-please

test-me-please

test-me-please


Genuine question: Why not use already existing syntactic conventions a-la for of ?

Not to sound dramatic, but I'm very concerned by the fact that using breaks already existing intuitions of JavaScript syntax.
Specifically, using messes with variable declarations (let/const/var), while await using gives a false impression of awaiting something on the line it has been used (i.e. it looks like it awaits a possible promise on the right-hand side of the assignment).
Here's an example of syntax that actually fits already established conventions:
const getResource = () => ({
  [Symbol.dispose /* Symbol.asyncDispose */]() { /* do disposal */ }
});

// inline
const resource = getResource();
try {
  /* await */ resource.doSomething();
} finally {
  /* await */ resource[Symbol.dispose /* Symbol.asyncDispose */]();
}

// with keyword
using /* await */ (const resource of getResource()) {
  /* await */ resource.doSomething();
} // `resource` gets disposed here, clear and simple

// with iterators
const getIterResource = () => ({
  [Symbol.dispose /* Symbol.asyncDispose */]() { /* do disposal */ }
  [Symbol.iterator /* Symbol.asyncIterator */]() { /* do iteration */ }
});

for using /* await */ (const resourceChunk of getIterResource()) {
  /* await */ processChunk(resourceChunk);
}

I do not claim to be qualified or provide a comprehensive and robust solution here, but at the very least this example attempts to work within already existing syntactic conventions.

@yohgen There are 5 years long discussions about syntax, you could check the old issues, especially #15 and #76 .

@hax Thanks for the pointers.


Scala DSL

Hi,
Any path for a Scala dsl? It can be a match made in heaven.
Thanks!

Hi @omidb
Scala native DSL would be indeed a great match, however it is not on our immediate roadmap.
We were hopping that adding support for language X would be a community driven effort, since there are many different ways of implementing an API. (i.e. in Scala - monadic API or just a Java wrapper with implicits, or continuation based etc')
I'm wondering were there any parts of the API that you looked at and was thinking to yourself "oh in Scala that would have been so much nicer?"
Thanks a lot for your feedback,
Igal


ReferenceError: __dirname is not defined in ES module scope

Using "type": "module" I'm getting this error

Thanks for your report. Please update to 0.2.1. It should be fixed.


New "Urgent Reset Options" config has a max value of 28, can that be noted in the config description?

Thanks for creating an issue! Please fill out this form so we can be sure to have all the information we need, and to minimize back and forth.

What are we trying to do?

It looks like there's actually a max value at 28 for the "Urgent Reset Options" config (https://github.com/DARIAEngineering/dcaf_case_management/issues/2338). I arrived at that conclusion after BAF attempted to set at 30, got an "invalid value" error, and then trial-and-erroring it from 10 upward.

What feature or behavior is this required for?
"Urgent Reset Options" config (https://github.com/DARIAEngineering/dcaf_case_management/issues/2338)

How could we solve this issue? (Not knowing is okay!)

Immediate/small solution: It's not a huge priority, but updating the prompt (currently reads: "Number of idle days until a patient is removed from the urgent list. Defaults to 6 days") to include the max value would be helpful from a UX perspective.
For more questions (not urgent), see below.

Anything else?

The 28 day max is not an issue for BAF (a huge help to lift the max from 6 to 28!), but our BAF CM lead did share wonderings about whether there are any concerns about funds not un-checking the Urgent flag when cases are resolved.
Our BAF CM lead shared that we always un-check the "urgent" flag once cases have been resolved, as opposed to letting the "urgent" flag lapse on its own. I.e., it's less of a concern that people aren't closing out cases and more about worrying we've let something slip through the cracks. Is there any concern from DARIA developers' POV about cases that aren't being closed out in a real/practical sense from any of the funds?

The immediate action makes sense to me - I'm tagging this as a hacknight fix, I think it's a pretty good setup-training issue. So for the developer tackling this - please update the text to include the min and max days. You can find the source in app/models/config.rb and find the view by logging in and going to Admin > Config Management on the navbar.
To your broader question, our urgents behavior was something that came up in user testing pretty early - I found an early ref to it in https://github.com/DARIAEngineering/dcaf_case_management/issues/614 and https://github.com/DARIAEngineering/dcaf_case_management/issues/119 and maybe buried in another early issue or two. I think the general behavior we observed was that cleanup doesn't tend to happen in similar situations (or happens very inconsistently), so a low-intensity bot would be pretty helpful. Put another way - if BAF makes it a point to uncheck then I think that makes you all pretty far ahead of the curve!

@elimbaum would you mind taking care of this if you get a sec in the near future? The immediate fix (including the max value in the help text) suggested by hannah should work great. Thank you!


Create an outline for the Stack Blocks tutorial

@Sahana-Srinivasan is writing the code for this tutorial

This is done.


Why hasn't this topic be ported to BC docs?

Maybe I am overlooking something, but it seems this topic does not exist as BC topic. Please include it there.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d7a572bd-8d8f-37c6-86e7-ba74ce61e93e
Version Independent ID: 405f936b-7b89-c5d8-d9e0-76d201354f55
Content: Handling Errors by Using Try Functions - Dynamics NAV
Content Source: dynamics-nav/Handling-Errors-by-Using-Try-Functions.md
Product: dynamics-nav-2018
GitHub Login: @jswymer

Thanks for the feedback. This topic will be part of the BC docs with the next doc refresh.


Closing Monterey app still does not exit the gulp watch command

@vegarringdal Yesterday I tried to implement https://github.com/Quramy/electron-connect/issues/2#issuecomment-126706085 (again, since it stopped working somehow) and I can't get it to work. client.sendMessage('closed'); is executed but not received via electron.on('closed'). Can you take a stab at this?

Sure Ill have a look at it, dunno when I get a chance, but then I have a task to do when I get time :-)
Will be online, for 1 hours now, maybe I get lucky :-)

I wish you good luck :smile:

Just tried to fix it again but still doesn't work. It would be unsensible to put more hours into this


No context menu or active linking in cell output

Applies To

[X] Notebooks (.ipynb files)
[ ] Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?
When you look at the output of a notebook cell, you can highlight text and links, but not right-click or open a context menu for the selection. URLs are automatically detected and highlighted, but cannot be clicked on to open (even with ctl and alt modifiers). If you have highlighted things, you can use ctl + c and that works, but highlighting and right clicking the selection (like you might in a terminal window) does not work.
Repro:

Open any new or existing notebook.
Create a cell with the following code and run it:
print("Hi! https://github.com/microsoft/vscode-jupyter")
Try to highlight and right click on the plain text or ctl/alt click on the link to open it in the browser. It should not work.

VS Code Version
Version: 1.70.0 (user setup) Commit: da76f93349a72022ca4670c1b84860304616aaa2 Date: 2022-08-04T04:38:16.462Z Electron: 18.3.5 Chromium: 100.0.4896.160 Node.js: 16.13.2 V8: <IP_ADDRESS>-electron.0 OS: Windows_NT x64 10.0.22000
Jupyter Extension Version
v2022.7.1002181843
Jupyter logs
I don't see any logs generated by the highlighting/copying action.

Coding Language and Runtime Version
Python 3.7.12
Language Extension Version (if applicable)
Python 3.7.12
Anaconda Version (if applicable)
conda 4.13.0
Running Jupyter locally or remotely?
Local

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


CAD 2907: Alonzo support in the workbench

Add Alonzo profiles into the workbench and
Fix a small bug that prevented profile names from being properly passed through

Test with:
make cls cluster-shell CLUSTER_PROFILE=default-alzo

bors r+


Reduce CI duration from 20 minutes to 2 minutes

At the moment CI takes ~20 minutes to complete because of the following reasons:

The Liquid contract tests run the same commands with all 131 validators that are currently registered on the Piccadilly network.
Because of an incorrect variable name all jobs in the matrix are executed with all Python versions.

This fix reduces CI execution time to 2 minutes.

Great!


Remove the current reimbursement from the same subquota panel

Example (with current reimbursement repeated in the same subquota panel): https://jarbas.datasciencebr.com/#/documentId/5893871

Changes Unknown when pulling 981a210bc655f7d6aa6510dd2fa6e224fa5f795e on cuducos-remove-duplicate-same-subquota into ** on master**.

I've seen this happening (not sure if was a single time or with every reimbursement). Was appearing all the times?

Yep. It was a bug.
Not for the same day panel though (this panel uses an specific endpoint and the current reimbursement is excluded from the results in the backend).
However the same subquota panel was implemented using the filtering options of the existing API which had no exclude option. So that's what this PR fixes: front-end validation of the results to avoid showing the current reimbursement in the panel.


SyntaxError: invalid regexp group in Firefox

polished version: ^3.0.0-beta.1

Mixin/Helper/Shorthand you were using and how you were using it:
import { fluidRange } from 'polished'

const globalStyles = css`
  html {
    ${fluidRange({
      prop: 'font-size',
      fromSize: '14px',
      toSize: '18px',
    })}
  }
`

What you are seeing:
White screen, console error:
SyntaxError: invalid regexp group
Reproduction: (Works in Chrome, does not in Firefox <= 65.0)
https://codesandbox.io/s/k2no733zyo?expanddevtools=1
I believe this is because of negative lookbehind in https://github.com/styled-components/polished/blob/v3.0.0-beta.1/src/math/math.js#L5 which is not yet supported in many environments

Let me know if there's something I can do to fix it

@selrond Thanks for catching this. I will give it a look tonight, but likely we will need to modify the code to replace the negative lookbehind to use a negative lookahead instead.

@selrond Should be good now with beta.2
https://codesandbox.io/s/8l67xqmk20

@bhough works, thanks!


guile-hall: init at 0.4.1

Description of changes
Hall is a project manager and build tool for GNU guile.
See Guile Reference Manual / 4.8 Distributing Guile Code.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/1016/18


tms endpoint - can multiple URIs be provided?

We're passing TileJSON around the OAM server API, so we theoretically have multiple entries for tiles.  It'd be nice to not lose them if possible.  In the meantime, I'm grabbing the first and sending that as uri.
(Note: this is more relevant for Mapbox / other TMSes with host aliasing)

Good point @mojodna.  I haven't spent much time in here myself, but it looks to me like the underlying schema & controller are setup to handle multiple tms uri's, but POST /tms only takes one.
Should be easy enough to allow an array for the uri parameter.  @scisco or @jflasher :+1: or :-1: ?

@jflasher @scisco I'm about to work on this, any red flags for just substituting the tms' single value with an array?

@kamicut @scisco implemented this I believe, I don't know that I've looked into it previously.


Fix LaTeX representation tests wrt PyTensor and PyMC updates

Closes #244
I suspect that this should work. Let's see if tests pass

:books: Documentation preview :books:: https://pymc-marketing--245.org.readthedocs.build/en/245/

Just update the minimum version to be the latest.
The point of testing the older is to know when it is no longer supported

So replace https://github.com/pymc-labs/pymc-marketing/blob/main/.github/workflows/ci.yml#L10 to 5.3.0?

Yes, and the requirements


making change to link downloads

Changing downloadViaLink to also include an optional headers argument.

👍  good to merge


mail: no address

Hi, I got the following error:
time="2024-04-28T09:37:31Z" level=error msg="Could not send email to<EMAIL_ADDRESS>gomail: could not send email 1: gomail: invalid address \"\": mail: no address"
gomail: could not send email 1: gomail: invalid address "": mail: no address

with the following Drone step:
  - name: back-notify
    pull: if-not-exists
    image: drillster/drone-email
    settings:
      host: smtp.163.com
      port: 465
      subject: 'Drone build: [{{ build.status }}]'
      body: 'Building result: [{{ build.status }}] {{ repo.name }} ({{ repo.branch }}) #{{ build.number }}'
      username:
         from_secret: EMAIL_USERNAME
      password:
         from_secret: EMAIL_PASSWORD
      recipients_only: true
      recipients<EMAIL_ADDRESS>    depends_on:
      - back-deploy
    when:
      status:
        - success
        - changed
        - failure

How can I fix this error?

steps:
  - name: notify
    image:
    settings:
      from.name: Drone CI
      from.address:
      host:
      port: 465
      username:
      password:
        from_secret: email_password
      recipients_only: true
      recipients<EMAIL_ADDRESS>      attachments:
        - /drone/src/someFile.txt
        # - /drone/src/someFile.html
    when:
      status:
        - failure
        # - success
depends_on:
  - default-1


StartByte and EndByte check doesn't work.

System.ArgumentOutOfRangeException: ''minValue' cannot be greater than maxValue.

Fixed by adding return;
Also fixed typos where each msgbox was an incorrect message.


MauiCompat Fix for preview 12

Description of Bug
Update script
PR Checklist

[ ] Has a linked Issue, and the Issue has been approved
[ ] Has tests (if omitted, state reason in description)
[ ] Has samples (if omitted, state reason in description)
[ ] Rebased on top of main at time of PR
[ ] Changes adhere to coding standard

[ ] Updated documentation

Thanks @VladislavAntonyuk! It looks like the CI Pipeline is still installing/using Preview 11.
I’ll take a look now at the azure-pipelines.yml file to see if I can nudge it to P12 👍


Can't repeat "vina>cross-docking>SR1" in Tabel6

Hi,
I ran GenScore on PDBbind-crossdock-core for testing (The model I tested was GT_ft_1.0_1.pth, using the default parameters for everything else.). However, for the decoys generated by vina, 505 receptor-ligand pairs (1343 in total) reported errors:
"""
Traceback (most recent call last):
File ".../GenScore/GenScore/example/genscore.py", line 256, in
main()
File ".../GenScore/GenScore/example/genscore.py", line 237, in main
ids, scores = scoring(prot=inargs.prot,
File ".../GenScore/GenScore/example/genscore.py", line 82, in scoring
data = VSDataset(ligs=lig,
File ".../GenScore/GenScore/example/../GenScore/data/data.py", line 178, in init
self.ids, self.gls = zip(*filter(lambda x: x[1] != None, zip(self.idsx, self.gls)))
ValueError: not enough values to unpack (expected 2, got 0)
"""
In all the cases where score can be generated, I calculated that the cross-docking SR1 is only 0.463 (0.59 reported in paper), and the SR1 of redocking is 0.680.
What problems do I need to correct when running genscore?
Best regards.

Can you reproduce the results when the docking poses are generated by Surflex-Dock or Glide SP?    The errors may come from those poor structures generated by Vina (some structures in .pdbqt format cannot be well recognized by RDKit though they have been processed by Open Babel).   Additionally, it should be noted that to better calculate the scoring powers, the SR1 calculated in Table 6 is based on the idea of ensemble docking, where the five complexes in the same cluster is considered as a group, thus resulting in ~285 samples rather than 1343.

Thank you for your reply.
What I understand now is that in a cluster (57 in total), there are on average 5 subsets accoring to the lignad ID, and all poses of a specific ligand across multiple receptor as a case. So there are 285 cases. If the receptor ID is the same as the ligand id, do these poses be included in the evaluation of cross-docking? Also, what tools do you use to calculate rmsd?
Looking forward to your reply

Can you reproduce the results when the docking poses are generated by Surflex-Dock or Glide SP? The errors may come from those poor structures generated by Vina (some structures in .pdbqt format cannot be well recognized by RDKit though they have been processed by Open Babel). Additionally, it should be noted that to better calculate the scoring powers, the SR1 calculated in Table 6 is based on the idea of ensemble docking, where the five complexes in the same cluster is considered as a group, thus resulting in ~285 samples rather than 1343.

Yes, just as you understand. Of course the crystalized ligand should be excluded in the evaluation of cross-docking performance. The pose with the highest score among all the poses of these four complexes is regarded as the final pose for this group.
The RMSDs for the poses in PDBbind-crossdock-core are calculated through the obrms utility in OpenBabel. Please refer to our previous paper (https://doi.org/10.1186/s13321-021-00560-w).  Of course, some tools such as spyrmsd (https://githu b.com/RMeli/pyrmsd) and DockRMSD (https://zhanglab.ccmb.med.umich.edu/DockRMSD/) can also produce substantially consistent results.

OK. Thanks again for your reply.
Best regards,
Zechen