ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None

Hi, I'm using the GRID s1 sample videos to train (unseen_speakers) and I'm getting this error.
Any idea?
Using all available GPUs.
Using TensorFlow backend.

Enumerating dataset list from disk...
Found 10 videos for training.
Found 10 videos for validation.

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
the_input (InputLayer)       (None, 75, 100, 50, 3)    0
_________________________________________________________________
zero1 (ZeroPadding3D)        (None, 77, 104, 54, 3)    0
_________________________________________________________________
conv1 (Conv3D)               (None, 75, 50, 25, 32)    7232
_________________________________________________________________
batc1 (BatchNormalization)   (None, 75, 50, 25, 32)    128
_________________________________________________________________
actv1 (Activation)           (None, 75, 50, 25, 32)    0
_________________________________________________________________
spatial_dropout3d_1 (Spatial (None, 75, 50, 25, 32)    0
_________________________________________________________________
max1 (MaxPooling3D)          (None, 75, 25, 12, 32)    0
_________________________________________________________________
zero2 (ZeroPadding3D)        (None, 77, 29, 16, 32)    0
_________________________________________________________________
conv2 (Conv3D)               (None, 75, 25, 12, 64)    153664
_________________________________________________________________
batc2 (BatchNormalization)   (None, 75, 25, 12, 64)    256
_________________________________________________________________
actv2 (Activation)           (None, 75, 25, 12, 64)    0
_________________________________________________________________
spatial_dropout3d_2 (Spatial (None, 75, 25, 12, 64)    0
_________________________________________________________________
max2 (MaxPooling3D)          (None, 75, 12, 6, 64)     0
_________________________________________________________________
zero3 (ZeroPadding3D)        (None, 77, 14, 8, 64)     0
_________________________________________________________________
conv3 (Conv3D)               (None, 75, 12, 6, 96)     165984
_________________________________________________________________
batc3 (BatchNormalization)   (None, 75, 12, 6, 96)     384
_________________________________________________________________
actv3 (Activation)           (None, 75, 12, 6, 96)     0
_________________________________________________________________
spatial_dropout3d_3 (Spatial (None, 75, 12, 6, 96)     0
_________________________________________________________________
max3 (MaxPooling3D)          (None, 75, 6, 3, 96)      0
_________________________________________________________________
time_distributed_1 (TimeDist (None, 75, 1728)          0
_________________________________________________________________
bidirectional_1 (Bidirection (None, 75,<PHONE_NUMBER>
_________________________________________________________________
bidirectional_2 (Bidirection (None, 75,<PHONE_NUMBER>
_________________________________________________________________
dense1 (Dense)               (None, 75, 28)            14364
_________________________________________________________________
softmax (Activation)         (None, 75, 28)            0
=================================================================
Total params: 4,572,156.0
Trainable params: 4,571,772.0
Non-trainable params: 384.0
_________________________________________________________________
nextVal [<lipnet.helpers.threadsafe.threadsafe_iter instance at 0x1155eecf8>]
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
Epoch 0: Curriculum(train: True, sentence_length: -1, flip_probability: 0.5, jitter_probability: 0.05)
Process Process-1:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/site-packages/keras/engine/training.py", line 606, in data_generator_task
Epoch 1/5000
Traceback (most recent call last):
  File "/Users/rad182/Documents/rizkiarm-LipNet/training/unseen_speakers/train.py", line 78, in <module>
    train(run_name, 0, 5000, 3, 100, 50, 75, 32, 10)
  File "/Users/rad182/Documents/rizkiarm-LipNet/training/unseen_speakers/train.py", line 74, in train
    pickle_safe=True)
  File "/usr/local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/keras/engine/training.py", line 1851, in fit_generator
    str(generator_output))
ValueError: output of generator should be a tuple `(x, y, sample_weight)` or `(x, y)`. Found: None

@rad182 I was getting a similar error, but after deleting datasets.cache (and possibly some other change that I'm not aware of?) the stack trace changed to
Process Process-1:
Epoch 0: Curriculum(train: True, sentence_length: -1, flip_probability: 0.5, jitter_probability: 0.05)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/m/tensorflow/lib/python2.7/site-packages/keras/engine/training.py", line 606, in data_generator_task
    generator_output = next(self._generator)
  File "/Users/m/repos/LipNet/lipnet/helpers/threadsafe.py", line 16, in next
    return self.it.next()
  File "/Users/m/repos/LipNet/lipnet/lipreading/generators.py", line 206, in next_train
    ret = self.get_batch(cur_train_index, self.minibatch_size, train=True)
  File "/Users/m/repos/LipNet/lipnet/lipreading/generators.py", line 148, in get_batch
    video = Video().from_frames(path)
  File "/Users/m/repos/LipNet/lipnet/lipreading/videos.py", line 114, in from_frames
    frames_path = sorted([os.path.join(path, x) for x in os.listdir(path)])
OSError: [Errno 20] Not a directory: '/Users/m/repos/LipNet/training/unseen_speakers/datasets/train/s1/bbas3a.mpg'
Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1596, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1023, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Users/m/repos/LipNet/training/unseen_speakers/train.py", line 76, in <module>
    train(run_name, 0, 5000, 3, 100, 50, 75, 32, 10)
  File "/Users/m/repos/LipNet/training/unseen_speakers/train.py", line 72, in train
    pickle_safe=True)
  File "/Users/m/tensorflow/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "/Users/m/tensorflow/lib/python2.7/site-packages/keras/engine/training.py", line 1851, in fit_generator
    str(generator_output))
ValueError: output of generator should be a tuple `(x, y, sample_weight)` or `(x, y)`. Found: None

The Not a directory: '/Users/m/repos/LipNet/training/unseen_speakers/datasets/train/s1/bbas3a.mpg' seemed to be caused by an assumption that we were loading the videos from frames instead of videos, (see lipnet/lipreading/generators.py, but changing that call from from_frames to from_video (and then addressing subsequent issues) didn't get me anywhere.
I did find that preprocessing the videos using scripts/extract_mouth_batch.py seems to work--at least, I was able to produce a weightsxx.h file.

@rad182 @rizkiarm Similar issue. Any help is appreciated.
`Using all available GPUs.
Using TensorFlow backend.
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
Enumerating dataset list from disk...
Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/bbizzn.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3)
Video /home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/val/s1/bbizzn.mpg has incorrect shape (75, 360, 288, 3), must be (75, 100, 50, 3)
Found 50 videos for training.
Found 50 videos for validation.

Layer (type)                 Output Shape              Param #
the_input (InputLayer)       (None, 75, 100, 50, 3)    0

zero1 (ZeroPadding3D)        (None, 77, 104, 54, 3)    0

conv1 (Conv3D)               (None, 75, 50, 25, 32)    7232

batc1 (BatchNormalization)   (None, 75, 50, 25, 32)    128

actv1 (Activation)           (None, 75, 50, 25, 32)    0

spatial_dropout3d_1 (Spatial (None, 75, 50, 25, 32)    0

max1 (MaxPooling3D)          (None, 75, 25, 12, 32)    0

zero2 (ZeroPadding3D)        (None, 77, 29, 16, 32)    0

conv2 (Conv3D)               (None, 75, 25, 12, 64)    153664

batc2 (BatchNormalization)   (None, 75, 25, 12, 64)    256

actv2 (Activation)           (None, 75, 25, 12, 64)    0

spatial_dropout3d_2 (Spatial (None, 75, 25, 12, 64)    0

max2 (MaxPooling3D)          (None, 75, 12, 6, 64)     0

zero3 (ZeroPadding3D)        (None, 77, 14, 8, 64)     0

conv3 (Conv3D)               (None, 75, 12, 6, 96)     165984

batc3 (BatchNormalization)   (None, 75, 12, 6, 96)     384

actv3 (Activation)           (None, 75, 12, 6, 96)     0

spatial_dropout3d_3 (Spatial (None, 75, 12, 6, 96)     0

max3 (MaxPooling3D)          (None, 75, 6, 3, 96)      0

time_distributed_1 (TimeDist (None, 75, 1728)          0

bidirectional_1 (Bidirection (None, 75,<PHONE_NUMBER>

bidirectional_2 (Bidirection (None, 75,<PHONE_NUMBER>

dense1 (Dense)               (None, 75, 28)            14364

softmax (Activation)         (None, 75, 28)            0
Total params: 4,572,156.0
Trainable params: 4,571,772.0
Non-trainable params: 384.0

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:910] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: GeForce GTX 1070
major: 6 minor: 1 memoryClockRate (GHz) 1.645
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 7.25GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0)
Epoch 0: Curriculum(train: True, sentence_length: -1, flip_probability: 0.5, jitter_probability: 0.05)
Process Process-1:
Traceback (most recent call last):
File "/home/deepakgupta1313/anaconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/deepakgupta1313/anaconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/deepakgupta1313/.local/lib/python2.7/site-packages/keras/engine/training.py", line 606, in data_generator_task
generator_output = next(self._generator)
File "/home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/lipnet/helpers/threadsafe.py", line 16, in next
return self.it.next()
File "/home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/lipnet/lipreading/generators.py", line 205, in next_train
ret = self.get_batch(cur_train_index, self.minibatch_size, train=True)
File "/home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/lipnet/lipreading/generators.py", line 147, in get_batch
video = Video().from_frames(path)
File "/home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/lipnet/lipreading/videos.py", line 114, in from_frames
frames_path = sorted([os.path.join(path, x) for x in os.listdir(path)])
OSError: [Errno 20] Not a directory: '/home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/datasets/train/s1/bbbmzn.mpg'
Epoch 1/5000
Traceback (most recent call last):
File "/home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/train.py", line 77, in
train(run_name, 0, 5000, 3, 100, 50, 75, 32, 50)
File "/home/deepakgupta1313/Desktop/Deepak/Programs/Github/LipNet/training/unseen_speakers/train.py", line 73, in train
pickle_safe=True)
File "/home/deepakgupta1313/.local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 88, in wrapper
return func(*args, **kwargs)
File "/home/deepakgupta1313/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1851, in fit_generator
str(generator_output))
ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None
`

Hi, I got similar problems.
Have you guys figured out how to fix them? @rad182 @michiyosony @deepakgupta1313
Thanks!

I was too able to train on frames (after running scripts/extract_mouth_batch.py). For those struggling to get this to work, please extract the lip/mouth into frames before training.

hi @DerekChia. we extracted the mouth frames using  scripts/extract_mouth_batch.py and now for each video we have directory containing 75 cropped frames around the mouth.
The problem is when we try to train we get the error -
/329/LipNet/training/random_split/datasets/video/bbal8p/mouth_063.png has incorrect shape (1, 100, 50, 3), must be (75, 100, 50, 3)
Video /cs/engproj/329/LipNet/training/random_split/datasets/video/bbal8p/mouth_018.png has incorrect shape (1, 100, 50, 3), must be (75, 100, 50, 3)
Video /cs/engproj/329/LipNet/training/random_split/datasets/video/bbal8p/mouth_064.png has incorrect shape (1, 100, 50, 3), must be (75, 100, 50, 3)
Video /cs/engproj/329/LipNet/training/random_split/datasets/video/bbal8p/mouth_016.png has incorrect shape (1, 100, 50, 3), must be (75, 100, 50, 3)
Video /cs/engproj/329/LipNet/training/random_split/datasets/video/bbal8p/mouth_011.png has incorrect shape (1, 100, 50, 3), must be (75, 100, 50, 3)
Video /cs/engproj/329/LipNet/training/random_split/datasets/video/bbal8p/mouth_058.png has incorrect shape (1, 100, 50, 3), must be (75, 100, 50, 3)
Video /cs/engproj/329/LipNet/training/random_split/datasets/video/bbal8p/mouth_024.png has incorrect shape (1, 100, 50, 3), must be (75, 100, 50, 3)
Video /cs/engproj/329/LipNet/training/random_split/datasets/video/bbal8p/mouth_023.png has incorrect shape (1, 100, 50, 3), must be (75, 100, 50, 3)
So its looks likes it wants videos....

各位大神，这个问题ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None怎么解决？

@rad182 @michiyosony @Yuren-Zhong @deepakgupta1313 @DerekChia
I also met this problem, how did you solve it, thank you very much!


Move post_install from cmd to FormulaInstaller

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] ~Have you written new tests for your changes? Here's an example~.

[x] Have you successfully run brew tests with your changes locally?

One more cmd requirement removal as part of #4059.
I went with the naive fix - just move the code from cmd/postinstall to FormulaInstaller. Do let me know if this isn't a good solution and how I should improve it.

Another great PR, thanks @mrfoto!


Svg middleware

Completed setting up the middleware functions that will convert a .perf file to .SVG flamegraph that can be viewed in the browser. tested successfully with postman. It will need other preceding middleware functions to complete the whole route handler. Also contains the new ESlint config.

LGTM with the changes you just pushed, defer to you both whether the pathing/logging issues are with addressing as part of MVP.
👍 🥳


fix(client): override the default SQLite relative file path

Change Summary
closes #400
closes #409
This PR overrides the default SQLite URL to use a path relative to the Prisma Schema file instead of the working directory. This is a breaking change but is required in order to maintain consistency between the CLI and the Client.
This PR also exports a new constant, SCHEMA_PATH which points to the location of the schema file used to generate the client.
from prisma import SCHEMA_PATH

print(SCHEMA_PATH)  # Path('/absolute/path/prisma/schema.prisma')

Checklist

[x] Unit tests for the changes exist
[x] Tests pass without significant drop in coverage
[ ] Documentation reflects changes where applicable
[x] Test snapshots have been updated if applicable

Agreement
By submitting this pull request, I confirm that you can use, modify, copy and redistribute this contribution, under the terms of your choice.

Awesome! The new constant will prove to be useful, and it's relieving that #409 has been fixed, ty Robert :)


Card-bar todo-list

[ ] Implement a preview pane to look at card text
[ ] Searchbar

Moving to main todo list


Memory / deallocated error in NavigationController

I have a navigation controller based application. At the first view there exist a button who push another ViewController or TableViewController to the navigation controller stack. This ViewController / TableViewController use the AAPullToRefresh. If I go back to the main root controller, I get the follow error message:
"An instance 0x99fca00 of class UITableView was deallocated while key value observers were still registered with it. Observation info was leaked, and may even become mistakenly attached to some other object. Set a breakpoint on NSKVODeallocateBreak to stop here in the debugger. Here's the current observation info:
<NSKeyValueObservationInfo 0x8ba4790> (
<NSKeyValueObservance 0x8bcffe0: Observer: 0x8bcc2d0, Key path: contentOffset, Options: <New: YES, Old: NO, Prior: NO> Context: 0x0, Property: 0x8cd3d80>
<NSKeyValueObservance 0x8bd0040: Observer: 0x8bcc2d0, Key path: contentSize, Options: <New: YES, Old: NO, Prior: NO> Context: 0x0, Property: 0x8cd5970>
<NSKeyValueObservance 0x8bd00e0: Observer: 0x8bcc2d0, Key path: frame, Options: <New: YES, Old: NO, Prior: NO> Context: 0x0, Property: 0x8cd5fc0>
)"
If I need a ViewController with ScrollView it's the same. Only the word "UITableView" replaced by "UIScrollView". The message replicate with every new push of the controller. What can I do?

@xzenon : can you tell me why ? :+1:

You don't have to add - (void)dealloc to AAPullToRefresh.m, just add:
- (void)dealloc {
    pullToRefresh.showPullToRefresh = NO;
}

to your ViewController. it's work for me.

@arielpollack Thanks for your answer man. I was stuck with the same issue for past 3 days.


fresh: command not found on windows

I am trying to run this on Windows 7. It returns this:
 fresh: command not found
I have a config file. I don't know any other step to run it even after reading readme file.
And i did this;
go get -u github.com/pilu/fresh
What should i do to simply run it.

This is probably due to your $GOPATH not being configured properly.  Try echo $GOPATH and see if that includes the directory where fresh was installed.

I think go get install packages to the same location. That is not that. I will check to see anyways.

So it is not about paths.

not runed fresh
GoPATH is set
C:\Users\mahdi>echo %GOPATH%
C:\Users\mahdi\go

I did it by giving full file path to the  main.go file inside the fresh folder
go run C:/go-Work/src/github.com/pilu/fresh/main.go

I fixed mine by adding the path to the go bin folder where the fresh binary is stored on my PC to the PATH environment variable. My path was C:/Users//go/bin.
Restart powershell after too.

I am trying to run this on Windows 7. It returns this:
 fresh: command not found
I have a config file. I don't know any other step to run it even after reading readme file.
And i did this;
go get -u github.com/pilu/fresh
What should i do to simply run it.

Same with linux


Update header.html

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

ok


Payload CMS v3 support

Payload v3 was released on November 19th. As far as I can tell from their release notes and migration guide, the following changes were made to the plugin system:

Plugins must use named exports
The export must have a Plugin suffix

Both are already the case for payload-plugin-oidc.
After I get this plugin to work with Payload v3, would a PR be welcome to add a note to the README.md about compatibility with different Payload versions?
https://github.com/payloadcms/payload/releases/tag/v3.0.0
https://github.com/payloadcms/payload/blob/main/docs/migration-guide/overview.mdx#plugins

@baer95 of course! let's keep this updated :)


"Before .NET"

There was no before .NET. I suspect that is supposed to say "Before .NET 2". 2 was the version that introduced generics. Also I suspect where it says "targets .NET" it should also be giving a version number.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: efb4c520-95e7-d46f-6e44-4a755bd12af3
Version Independent ID: 45213f1e-b1c8-7251-b6d9-21fb64159b57
Content: CA1003: Use generic event handler instances - Visual Studio
Content Source: docs/code-quality/ca1003-use-generic-event-handler-instances.md
Product: visual-studio-windows
Technology: vs-ide-code-analysis
GitHub Login: @gewarren
Microsoft Alias: gewarren

Looks reasonable. Updating.


Immersive portals bug

Describe the bug
I can't interact with mobs through portals (you normally can)
Steps to reproduce
1.make a portal with portal helper blocks
2.place a mob through a portal
3.try to hurt or interact with it in any way
Logs
No response
Additional context
No response

logs or modlist would help
the steps to reproduce exist for a reason

I tried this out on fabric using version 3.4.4, and while I can break blocks through the portal, I can't interact with any entites. Does this work for you on a fabric instance?

Closing as stale / cannot reproduce.
If you can provide the necessary logs and information to reproduce the problem or identify the faulty mod, please open a new issue with all necessary details.


As the ansible playbook is idempotent, can I use it to fix a broken installation of nextcloud?

I'm having a lot of cascading failures with my installation of nextcloud. Poor choices on my part I'm sure. But I'm wondering if there's a way I can start over with this process and maybe revert my installation to a more "correct" and "clean" installation of nextcloud.
Is that possible using this ansible playbook?
Or perhaps I can do it by doing a manual update to 16.0.3?

If you have still the version you used to setup nextcloud: yes.
but nextcloud would not be installed again. only mysql/nginx/php/redis would be updated and the config would be setup to a working version.
always do backups before you start touching your installation. ;-)
if you do a git fetch && git stash && git pull && git stash pop (aka get the newest version of the playbook) it might be incompatible to your steup.
in this case i would advice to setup a new server and migrate the data from the old.

@ReinerNippes  Managed to get everything fixed! Thank you!


[EMB-136] Add Facebook app id meta tag

Purpose
We have this and FB wants it.
Summary of Changes
Add Facebook app id meta tag
Side Effects / Testing Notes
n/a
Ticket
https://openscience.atlassian.net/browse/EMB-136
Reviewer Checklist

[ ] meets requirements
[ ] easy to understand
[ ] DRY
[ ] testable and includes test(s)
[ ] changes described in CHANGELOG.md

Coverage remained the same at 48.33% when pulling d4b2cb32a1e17c01c1e70e2cc1a55e5592bb6c4d on jamescdavis:add_fb_app_id_meta_tag into 056cdc28fa4e9e0135461e08f52dd06b891afbc2 on CenterForOpenScience:develop.


Configure gm dapp host

Description

Resolves # (issue)
How Has This Been Tested?

Due Dilligence

[ ] Breaking change
[ ] Requires a documentation update

tests passed
https://github.com/WalletConnect/WalletConnectSwiftV2/actions/runs/6234936178

reopening


Handle errors in send to cosmos and eth CLI

Resolves #55

looks good, if it is intentional that those errors should now halt the orchestrator.

looks good, if it is intentional that those errors should now halt the orchestrator.


I am facing a issue while hosting the project on a shared hosting. My project is working properly on localhost, but when i shifted it on the server my websocket connection is not working on client side

I am using this--
var conn = new WebSocket('ws://example.com:8080');

I doubt a shared hosing provider would allow you to do this. They're most likely blocking port 8080. You can look into opening the port for public access.


[Issue 9438][pulsar-client] Fix inconsistent equals and hashCode for MessageIds

Fixes #9438
Motivation
Since MessageId is an opaque interface that has several subclasses, in general those subclasses should try to behave the same way with respect to one another if they refer to the same message. They should also follow the standard contract for Java equals and hashCode methods.
Modifications
Provided a single equals and hashCode implementation for MessageIdImpl that takes into account the batch version as well. Changed TopicMessageIdImpl to not take into account the topic anymore.
Verifying this change

Extended unit test for BatchMessageIdImpl
Added unit test for TopicMessageIdImpl

I also noticed the compareTo implementation seemed a bit inconsistent, so I tried to simplify it, but I'm not sure what the intention was exactly so I may not have implemented it properly.

@merlimat Can you review it again?

@codelipenghui @315157973 responded to your comments.
I get that the implementation for this is a little weird, so let me know if you have any other suggestions to achieve the same goal. Ideally we would have single message ID type that can represent either batched or unbatched messages, but I'm not sure how feasible that is without breaking changes.

Thinking out loud:
What about keeping your new tests cases, they express exactly the problem.
and fix the "equals" and "hashCode" methods of all of the implementations of MessageId in a way that all of the combinations work?
(it would be great to be able to mark MessageId as a 'sealed' interface).
in order to implement hashCode you could add a static utility method that receives all of the relevant fields:
public static int computeMessageIdHashCode(long ledgerId, long entryId, long partitionIndex, long batchIndex)
the same can be done for the equals method
public static booleanMessageIdEquals(long ledgerId1, long entryId1, long partitionIndex1, long batchIndex1,    long ledgerId2, long entryId2, long partitionIndex2, long batchIndex2 )

@eolivelli That sounds workable. I'd assume we'd want to do the same for compareTo as well?

Sure

in order to implement hashCode you could add a static utility method that receives all of the relevant fields:
public static int computeMessageIdHashCode(long ledgerId, long entryId, long partitionIndex, long batchIndex)

batchIndex is an attribute that only BatchMessageId has, and it doesn't seem appropriate to put it in the top MessageId.
I don’t know much about the historical reasons, why is there such a parameter as NO_BATCH in BatchMessageIdImpl
There is a special treatment for NO_BATCH in equals. When batchIndex=-1 (NO_BATCH), only ledgerId, entryId and partitionIndex need to be equal.
Can the HashCode also be specially processed? When batchIndex=-1, batchIndex is not allowed to participate in the calculation of hashcode

The constraint is that if two objects are 'equals' to each other they have the same hash code.

The constraint is that if two objects are 'equals' to each other they have the same hash code.

If we do special processing in hashCode as in the euqals method, when batchIndex=-1, batchIndex is not allowed to participate in the calculation of hashcode, so the hashCode is equal

@315157973 I'm also unsure about why we have NO_BATCH. Can an unbatched message can't be treated like a batched message with an index of 0?

@315157973 I'm also unsure about why we have NO_BATCH. Can an unbatched message can't be treated like a batched message with an index of 0?

The current hashcode is like this
return (int) (31 * (ledgerId + 31 * entryId) + (31 * (long) partitionIndex) + batchIndex);
If it is consistent with the logic of equal, it should be:
if(batchIndex == NO_BATCH){
return (int) (31 * (ledgerId + 31 * entryId) + (31 * (long) partitionIndex));
}
return (int) (31 * (ledgerId + 31 * entryId) + (31 * (long) partitionIndex) + batchIndex);

cc  @eolivelli

@315157973 works for me.
Let's add tests that cover all edge cases

@315157973 Let me clarify what I'm asking.
As you mentioned, the current BatchMessageIdImpl#equals takes into account all fields including the batchIndex: https://github.com/apache/pulsar/blob/608929227824fe4303f46aa432e42af77bcbf625/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BatchMessageIdImpl.java#L109. If there is no batchIndex it is assumed to be NO_BATCH (-1).
And BatchMessageIdImpl#hashCode is:
return (int) (31 * (ledgerId + 31 * entryId) + (31 * (long) partitionIndex) + batchIndex);

So for BatchMessageIdImpl, it does appear consistent with equals, as it takes into account the batchIndex.
The inconsistency is that MessageIdImpl#hashCode does not take into account the batchIndex:
return (int) (31 * (ledgerId + 31 * entryId) + partitionIndex);

This can be fixed by using the the same implementation as in BatchMessageIdImpl#hashCode and assuming batchIndex = NO_BATCH, like I have done in this pull request.
I think you are suggesting changing BatchMessageIdImpl#hashCode to:
if(batchIndex == NO_BATCH){
   return (int) (31 * (ledgerId + 31 * entryId) + (31 * (long) partitionIndex));
}
return (int) (31 * (ledgerId + 31 * entryId) + (31 * (long) partitionIndex) + batchIndex);

Which would only be necessary if equals treats batchIndex = NO_BATCH as equivalent to batchIndex = 0, which it does not currently. But should it?
Or are you suggesting only treating batchIndex = NO_BATCH like batchIndex = 0 only for the purposes of hashCode?

I talked to @sijie and he said that batchIndex = 0 and batchIndex = NO_BATCH should not be treated as not equal, so it should be fine for them to be both unequal and have unequal hash codes.

@315157973 @eolivelli I updated to delegate to the static method.
I think I understand what you're saying now. Your change to BatchMessageIdImpl#hashCode works if you keep the existing code in MessageIdImpl#hashCode. The problem is now you have the logic in two different places, which I'd like to avoid, since that's probably why we ended up with this inconsistency to begin with. Having the logic in one place makes the code a lot easier to understand and reduces the chance of future breakage.

works for me

Thanks @315157973. I added some tests for some of the edge cases, but I'd appreciate feedback on any other tests I can add.

@merlimat can you take a final look ?

@merlimat Could you please help review it again

This patch is too dangerous for 2.7.2 IMHO, I have removed the release/2.7.2 label
@codelipenghui


Tunnel the same host/port to multiple destinations at the same time

Hey.
Currently, sish supports --http-load-balancer option, which allows forwarding HTTP requests to multiple destinations. The feature I'm requesting is a similar behaviour, but instead of selecting one destination and tunneling the request, sish would instead forward the HTTP request to all destinations at once and return a predefined HTTP response (from a different host/endpoint?).
This is likely outside of the scope of sish, but it's worth asking.

Unfortunately this is out of the scope of sish. There are other proxy solutions that are dedicated to that use case (traffic mirroring), such as haproxy. Sorry!
(reason it is out of scope is sish just implements a simple round robin balancer. anything more complex than that should really be a separate project)


Unable to restore and use dotnet-watch under 2.0.0-Preview1

I'm unable to install (via restore) dotnet watch tool on OS X:
  <ItemGroup>
    <DotNetCliToolReference Include="Microsoft.DotNet.Watcher.Tools" Version="1.0.0" />
    <DotNetCliToolReference Include="Microsoft.Extensions.SecretManager.Tools" Version="1.0.0" />
  </ItemGroup>

  Restoring packages for /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj...
  Restoring packages for /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj...
  Restoring packages for /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj...
/usr/local/share/dotnet/sdk/2.0.0-preview1-005977/NuGet.targets(97,5): error : Unable to resolve 'Microsoft.DotNet.Watcher.Tools (>= 1.0.0)' for '.NETCoreApp,Version=v2.0'. [/Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj]
/usr/local/share/dotnet/sdk/2.0.0-preview1-005977/NuGet.targets(97,5): error : Value cannot be null. [/Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj]
/usr/local/share/dotnet/sdk/2.0.0-preview1-005977/NuGet.targets(97,5): error : Parameter name: path [/Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj]

I'm able to install and use dotnet user-secrets tool (part of dotnet/templating default content) alone:
<ItemGroup>
    <DotNetCliToolReference Include="Microsoft.Extensions.SecretManager.Tools" Version="1.0.0" />
  </ItemGroup>

otnet restore
  Restoring packages for /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj...
  Restoring packages for /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj...
  Lock file has not changed. Skipping lock file write. Path: /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/obj/project.assets.json
  Restore completed in 150.01 ms for /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj.
  Restore completed in 737.7 ms for /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/src/PolymerApplication/PolymerApplication.csproj.

  NuGet Config files used:
      /Users/piotrblazejewicz/git/dotnet-webcomponents/Polymer2Application/NuGet.Config
      /Users/piotrblazejewicz/.nuget/NuGet/NuGet.Config

otnet user-secrets --help
User Secrets Manager 1.0.0-rtm-10308
...

Dotnet Info:
.NET Command Line Tools (2.0.0-preview1-005977)

Product Information:
 Version:            2.0.0-preview1-005977
 Commit SHA-1 hash:  414cab8a0b

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.12
 OS Platform: Darwin
 RID:         osx.10.12-x64
 Base Path:   /usr/local/share/dotnet/sdk/2.0.0-preview1-005977/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0-preview1-002111-00
  Build    : 1ff021936263d492539399688f46fd3827169983

This issue was moved to NuGet/Home#5264

@peterblazejewicz I've moved the issue to NuGet as the error you saw during restore error : Value cannot be null.  comes from NuGet, and I was unable to reproduce this locally.
You might trydotnet nuget locals all --clear to reset your NuGet caches, but the NuGet team will be able to help you more.

@natemcmaster
It appears to be fixed, what's more I'm able to add package manually now:
dotnet add package Microsoft.DotNet.Watcher.Tools
Microsoft (R) Build Engine version <IP_ADDRESS>32
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /var/folders/f7/fldvv8tx3fb4pstk_sr2nk080000gn/T/tmp3WUBt2.tmp
info : Adding PackageReference for package 'Microsoft.DotNet.Watcher.Tools' into project '/Users/piotrblazejewicz/development/WebProject/WebProject.csproj'.
log  : Restoring packages for /Users/piotrblazejewicz/development/WebProject/WebProject.csproj...
info :   GET https://api.nuget.org/v3-flatcontainer/microsoft.dotnet.watcher.tools/index.json
info :   CACHE https://api.nuget.org/v3-flatcontainer/system.net.http/index.json
info :   CACHE https://api.nuget.org/v3-flatcontainer/system.net.http/4.1.0/system.net.http.4.1.0.nupkg
info :   CACHE https://api.nuget.org/v3-flatcontainer/system.text.regularexpressions/index.json
info :   CACHE https://api.nuget.org/v3-flatcontainer/system.text.regularexpressions/4.1.0/system.text.regularexpressions.4.1.0.nupkg
info :   OK https://api.nuget.org/v3-flatcontainer/microsoft.dotnet.watcher.tools/index.json 816ms
info :   GET https://api.nuget.org/v3-flatcontainer/microsoft.dotnet.watcher.tools/1.0.1/microsoft.dotnet.watcher.tools.1.0.1.nupkg
info :   OK https://api.nuget.org/v3-flatcontainer/microsoft.dotnet.watcher.tools/1.0.1/microsoft.dotnet.watcher.tools.1.0.1.nupkg 698ms
log  : Installing Microsoft.DotNet.Watcher.Tools 1.0.1.
info : Package 'Microsoft.DotNet.Watcher.Tools' is compatible with all the specified frameworks in project '/Users/piotrblazejewicz/development/WebProject/WebProject.csproj'.
info : PackageReference for package 'Microsoft.DotNet.Watcher.Tools' version '1.0.1' added to file '/Users/piotrblazejewicz/development/WebProject/WebProject.csproj'.

Thanks!
I havent' used clear switch or anything. Could you close https://github.com/NuGet/Home/issues/5264 please?

@peterblazejewicz dotnet add package adds the package as a PackageReference, not a DotNetCliToolReference.

@natemcmaster  🤦♂  silly me :) Thanks! Now everything appears to work OK with preview build
On the side note, is there a way to install them? I've habits, hope you understand to use install command (dnu commands install SecretManager)

On the side note, is there a way to install them?

Manually editing csproj is the only way to install a DotNetCliToolReference. cc @blackdwarf

Sorry to hijack the thread. Seems it might be relevant to the issue I am having.
I see that the latest version of the watch tool is 2.0.0-preview1-final on nuget.org, thus updating the tool reference as below:
<DotNetCliToolReference Include="Microsoft.DotNet.Watcher.Tools" Version="2.0.0-preview1-final" />

My app is already configured to use netcoreapp2.0 as the target framework and everything works fine.
The output I am getting as soon as I build the project with the above configuration, I get the following two errors.
Package Microsoft.DotNet.Watcher.Tools 2.0.0-preview1-final is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package Microsoft.DotNet.Watcher.Tools 2.0.0-preview1-final supports: netcoreapp2.0 (.NETCoreApp,Version=v2.0)

and
One or more packages are incompatible with .NETCoreApp,Version=v1.0.

The problem with the first error is that, none of the projects in my application are using netcoreapp1.0 as its target.
The problem with the second error is that, I actually don't expect any of my packages to be compatible with netcoreapp1.0 but eventually something does.
If I revert back the watch tool's version to 1.0.1, everything seems to work.
I have two question in this case;

May I stay on 1.0.1 and keep running my projects targeting netcoreapp2.0?
Are the errors I am getting above expected?

@CoskunSunali  - this is a NuGet and .NET SDK issue. Let's carry on the conversation here: https://github.com/NuGet/Home/issues/5000 as there isn't any actionable item here from projects in this repo.

@natemcmaster Perfect!


elm-format: GHC 8 compatibility

Without adding "allow-newer" for aeson, base, and transformers, I get
the build failure:
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: elm-format-0.3.0 (user goal)
trying: transformers-<IP_ADDRESS>/installed-0.5... (dependency of
regex-applicative-0.3.3)
next goal: optparse-applicative (dependency of elm-format-0.3.0)
rejecting: optparse-applicative-<IP_ADDRESS>, optparse-applicative-<IP_ADDRESS>
(conflict: elm-format => optparse-applicative>=0.11 && <0.12)

etc. etc.

Can be removed as soon as elm-format and the dependencies setting unecessary
restrictions on these versions catch up, or as soon as elm-format itself has a
similar built-in workaround.

Forgot the colon after "allow-newer." Let's try that again.

@DomT4 :ship:?

👍


int64 is not JSON serializable

Hi,
I'm running into an error that I've not seen before. I'm using already-featurized trajectories stored as text numpy arrays, to compare the performance of different featurizers.
Here's the osprey config file:
estimator:
    eval: |
        Pipeline([
                ('tica', tICA()),
                ('cluster', MiniBatchKMeans()),
                ('msm', MarkovStateModel(lag_time=100, n_timescales=5, verbose=True)),
        ])
    eval_scope: msmbuilder

strategy:
    name: random

search_space:
  cluster__n_clusters:
    min: 10
    max: 10000
    type: int

  tica__lag_time:
    min: 1
    max: 1000
    type: jump
    warp: log
    var_type: int
    num: 10

  tica__commute_mapping:
    choices:
      - True
      - False
    type: enum

  tica__n_components:
    min: 1
    max: 20
    type: int

cv:
  name: shufflesplit
  params:
    n_iter: 5
    test_size: 0.25

dataset_loader:
  name: numpy
  params:
    filenames: ftrajs/*
trials:

  uri: sqlite:///osprey-trials.db

And here's the output: https://pastebin.com/gtZxxwEt
I've previously done this kind of workflow, so I don't really know what might've gone wrong.
Best regards,
Juan

Hi @jeiros!
Thanks for the report! Could you also share your conda environment? Relevant package versions would be msmbuilder, numpy, sklearn, cython, etc.

Here's a small subset of the data with which the error can be reproduced
dihedrals.zip

Did you manage to replicate the error?

So I think the problem is in specifying a list of integers in a log warped space:
  tica__lag_time:
    min: 1
    max: 1000
    type: jump
    warp: log
    var_type: int
    num: 10

I changed it to a type: int and now it works.
Perhaps the int64 is not JSON serializable error comes from within the Osprey code that deals with the creation of the integers in the log space? Is it maybe using numpy, which stores the integers as the int64 dtype and that confuses the json package? No idea, since I'm not familiar with osprey's code, it's just a guess.
Let me know what you think

Sorry! I've been busy with moving, and haven't had much time to check this out.

Perhaps the int64 is not JSON serializable error comes from within the Osprey code that deals with the creation of the integers in the log space? Is it maybe using numpy, which stores the integers as the int64 dtype and that confuses the json package? No idea, since I'm not familiar with osprey's code, it's just a guess.

This is quite plausible. Does the #236 solve the issue for you, or does this require an additional fix? That PR needs some additional work to make it py27 compatible (although, I've very open to discontinuing py27 support).


Clarify that win.title must have the same name as installed app .exe

As I understand installed app's shortcut will search after win.title.exe file in the installed directory and run that .exe file. But I didn't find any clarification on that in the documentation.

New API 2.8 produces Squirrel.windows installer in a more user-friendly way.


Multiple StreamManagers in referenced libraries - how to find total memory usage

Hello,
I  have a singleton StreamManager  using only  a small pool in blocks of 128K  (our pattern of usage)
Ver. 2.2.1 . moving  to 2.3.2  .
Our code is in  a  net standard 2.0 library  referenced in a  main project  of .Net 4.8.
I  measure the memory usage of our StreamManager  and I've set a limit in the maximumFreeSmallPoolBytes.
Meanwhile, we use an important Microsoft library in .net 4.8  which has dependency in the RecyclableMemoryStream  >= 1.3.5
I know that the MS library, possibly, can use a lot of memory buffers.
My question is  how to find the total memory usage of all Stream managers (and those of the referenced  library)  ?
Thanks

Each RecyclableMemoryStreamManager publishes statistics about its usage, include bytes in use and bytes free. That will be easy for your own reference to manage.
For a 3rd-party library, I don't think there is an easy way. You could grab a process dump and find the reference to the RMSManager object in that library, but all you will be able to see is the list of buffers currently unused. The RMSManager does not track buffers currently in use by streams.
To get all of them, you could do some clever debugger scripting to find all RMS objects on the heap, iterate through the owned buffers, then do the same thing for all RMSManager objects. Tools like ClrMD might be easier to do this with.


Rewrite internal links

At them moment when linking to another generated page, you need to know the rendered URL. It would be great to rewrite linked URLs to the rendered URL. Example: to link from one post to another without knowing that posts URL I might just use this Markdown link:
[Link to another post](./another-post.md)

and then Phenomic replaces it with the actual link.

This is an pretty good idea and should not be that hard. We should implement this using a custom remark plugin.


Text message instead of Plot

Hi!
I was really interested in this library, but when I try to run the examples I just get a text output:
VBox(children=(VBox(children=(HBox(children=(Label(value='levels:'), FloatSlider(value=0.25, max=1.0, step=0.0…
I've seen that this happened before. I'm running it on Ubuntu 16.04 with conda and my browser is Firefox 59.0.2
Thank you, and compliment for your great job

Same problem with Chrome and Chromium

I have the same issue! Please help!

Same here ... on Mac Os X both firefox and Chrome

I see I never replied to this, sorry, it fell of the radar. Do you run classical notebook or jupyter lab?

Hi, well, I kept reading threads and I made it work. On the installation page it reads that for conda you just need to install ipyvolume from conda, but that is not enough, I neded also to enable the  widgets extension for the notebook. Then, after closing everything and openning again (old technique) ipyvolume worked on classical jupyter notebook. So the final instruction (from https://ipyvolume.readthedocs.io/en/latest/install.html, which is different from what the gihut page says)   was:
conda install -c conda-forge ipyvolume
jupyter nbextension enable --py --user ipyvolume
jupyter nbextension enable --py --user widgetsnbextension
Thanks again and great software. It is the first time I am exposed to it and looks really neat.

Thanks, that is a bug in the conda-package: https://github.com/conda-forge/ipyvolume-feedstock/pull/19

Please reopen if it did not fix it.

On Mac OS X (Mojave) :
ipywebrtc 0.4.1 has requirement ipywidgets>=7.4.0, but you'll have ipywidgets 7.2.1 which is incompatible.

MacOSX Mojave: Dependencies not automatically correct, needed: pip3 install --upgrade ipywidgets
Due to:
ipywebrtc 0.4.1 has requirement ipywidgets>=7.4.0, but you'll have ipywidgets 7.2.1 which is incompatible.

Hallo,
I recently started using ipyvolume and find it a great hands-on tool!
Bu now I got a problem:
I am using ipyvolume to visualize a 3d array. I use PhyCharm(2018.4 - Community edition) on Windows 8.1 (64x) and as I run ipv.quickscatter(..) I get the following message, but no plot:
VBox(children=(Figure(camera=PerspectiveCamera(fov=46.0, position=(0.0, 0.0, 2.0), quaternion=(0.0, 0.0, 0.0, …
I read the andswer from @iluvatar1 , but I do not know how should I do it with PhyCharm.
If you could help me, how I can enable the widgetsnbextension and the ipyvolume  in my case?
In advance thank you for your answer!

Now I edited it (if it is correct)
Enabling notebook extension ipyvolume/extension...
      - Validating: ok

Enabling notebook extension jupyter-js-widgets/extension...
      - Validating: ok

but still get the same message as before without any plot.


fix: it's not needed to create a new manifest to read a feed

it's not needed to create a new manifest in order to read a feed - just fetch and decode the old one
in this case it is not needed to provide a stamp in order to read the feed
    if (!this.stamp) {
      this.stamp = await pickStamp(this.beeDebug, this.console)
    }

    const manifest = await this.bee.createFeedManifest(this.stamp, 'sequence', topic, addressString)

Duplicate of https://github.com/ethersphere/swarm-cli/issues/123


insanely_fast_whisper_colab.ipynb doesn't work

Hi,
I find your project very interesting, therefore I tried to run the demo notebook in a T4 runtime on colab.
But the execution of the cell !pipx run insanely-fast-whisper --file-name https://huggingface.co/datasets/reach-vb/random-audios/resolve/main/ted_60.wav fails for me:
/root/.cache/pipx/fbbfdc728a6642e/lib/python3.10/site-packages/pyannote/audio/core/io.py:43: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend("soundfile")
/root/.cache/pipx/fbbfdc728a6642e/lib/python3.10/site-packages/torch_audiomentations/utils/io.py:27: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend("soundfile")
config.json: 100% 1.27k/1.27k [00:00<00:00, 5.89MB/s]
model.safetensors: 100% 3.09G/3.09G [00:25<00:00, 119MB/s]
generation_config.json: 100% 3.87k/3.87k [00:00<00:00, 15.4MB/s]
tokenizer_config.json: 100% 283k/283k [00:00<00:00, 12.4MB/s]
vocab.json: 100% 1.04M/1.04M [00:00<00:00, 3.90MB/s]
tokenizer.json: 100% 2.48M/2.48M [00:00<00:00, 30.2MB/s]
merges.txt: 100% 494k/494k [00:00<00:00, 54.1MB/s]
normalizer.json: 100% 52.7k/52.7k [00:00<00:00, 48.9MB/s]
added_tokens.json: 100% 34.6k/34.6k [00:00<00:00, 93.6MB/s]
special_tokens_map.json: 100% 2.07k/2.07k [00:00<00:00, 9.21MB/s]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
preprocessor_config.json: 100% 340/340 [00:00<00:00, 1.75MB/s]
Traceback (most recent call last):
  File "/root/.cache/pipx/fbbfdc728a6642e/bin/insanely-fast-whisper", line 8, in <module>
    sys.exit(main())
  File "/root/.cache/pipx/fbbfdc728a6642e/lib/python3.10/site-packages/insanely_fast_whisper/cli.py", line 105, in main
    pipe.model = pipe.model.to_bettertransformer()
  File "/root/.cache/pipx/fbbfdc728a6642e/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4302, in to_bettertransformer
    return BetterTransformer.transform(self)
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/root/.cache/pipx/fbbfdc728a6642e/lib/python3.10/site-packages/optimum/bettertransformer/transformation.py", line 228, in transform
    raise NotImplementedError(
NotImplementedError: The model type whisper is not yet supported to be used with BetterTransformer. Feel free to open an issue at https://github.com/huggingface/optimum/issues if you would like this model type to be supported. Currently supported models are: dict_keys(['albert', 'bark', 'bart', 'bert', 'bert-generation', 'blenderbot', 'bloom', 'camembert', 'blip-2', 'clip', 'codegen', 'data2vec-text', 'deit', 'distilbert', 'electra', 'ernie', 'fsmt', 'gpt2', 'gptj', 'gpt_neo', 'gpt_neox', 'hubert', 'layoutlm', 'm2m_100', 'marian', 'markuplm', 'mbart', 'opt', 'pegasus', 'rembert', 'prophetnet', 'roberta', 'roc_bert', 'roformer', 'splinter', 'tapas', 't5', 'vilt', 'vit', 'vit_mae', 'vit_msn', 'wav2vec2', 'xlm-roberta', 'yolos']).

Best wishes
Daniel

I've reported a similar issue running on an RTX 3060 gaming PC with linux here:
https://github.com/Vaibhavs10/insanely-fast-whisper/issues/133

Hi @danielKlasss & @afiaka87 I just released a patch and this should be fixed now. The issue crept up due to an update in optimum: https://github.com/huggingface/optimum/pull/1590
Make sure to get the most recent version of insanely-fast-whisper:

pipx uninstall insanely-fast-whisper
pipx install insanely-fast-whisper==0.0.13 --force

P.S. On the Colab it should work out of the box since we create a fresh virtualenv everytime it is run :)

@Vaibhavs10 Thank you, I can confirm that the problem is fixed.


fix(bermuda): Improved condition checks and handling of MAC addresses

Summary of changes:

In the update_advertisement function, the timestamp comparison check has been enhanced. An additional check for a non-None value before comparing with the current timestamp has been added, which prevents potential errors when handling undefined values.

In the _async_update_data function, the MAC address check has been improved. A check for a non-empty string has been added before verifying whether the first character of the address is one of 4, 5, 6, 7. This change increases the reliability of address identification and prevents possible errors in parsing unformatted data.

Great, thanks!
Are you spotting these by directly looking through the code or are you encountering some of these errors? Or is there a linting tool that is picking them up? If there's a tool perhaps I should add it to my vscode config to help avoid introducing more of these.
Anyway, thanks for the improvements, much appreciated!


Update azure-stack-marketplace-azure-items.md

@kivenkat : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Hi @kivenkat - This pull request was opened in the public repo. PMs should work in the private repo, per the Microsoft Docs contributor guide. Would you make this content update in the private repo instead? Thank you! We'll close this PR.


Pin molecule == 1.25 due to 2.0rc version being released on PyPI

lgtm. Thanks for taking care of this!


Settings not available from getOption()

In a 2.3.1 project I'm not able to access ClientConfig settings via $modx->getOption('key',null,'not found');.
While $modx->getPlaceholder('+key') worked.
After some debugging it turns out to be relaeted to the system event where the ClientConfig plugin is triggered. Changing it to "OnWebPageInit" fixed the issue in my snippet and the setting is accessible with getOption.

Getting the same issue. A migxdb processor calls a snippet to send an email on publish. Not sure there's a system event applicable to this is there? Ended up including the plugin code in my snippet to process the settings

Same occurs with me on 2.3.1

We still see this also. Is there any progress on this?

:+1:

Pull requests are welcome ;)
Based on Chris' comments, the solution is to add more events to the plugin than just the OnHandleRequest, but I personally haven't researched if there's a reason this was reported for 2.3.1 but not before.
As for the issue reported by @studiorepublic that seems a different issue where connector/processors indeed don't fire OnHandleRequest or any event at all. In #44 a suggested workaround is to invoke the OnHandleRequest rather than including the plugin code.

Is #115 fixing this issue?

While OnWebPageInit worked for the frontend/snippets, the CMP fails now with "Class not found".

That has been fixed in 1.4.
Per #115 and #109 1.4 is also switched to use the OnMODXInit event instead of OnHandleRequest, which makes it work in a lot more different scenarios.


Adding ability to deploy from bundle URLs

Adding ability to deploy from bundle URLs

Added ability to deploy with operator-sdk run bundle instead of from the upstream catalog or a local build

Added a symlink to simplify calling operator-sdk

Turned off facts gathering for performance

Moved a couple when clauses for clarity

Update build/stf-run-ci/tasks/pre-clean.yml

Create working dir to download sdk into

Add required bundle_image_path args to README

Fixing smoketest

Version of oc in jenkins image doesn't have the --serviceaccount option

Co-authored-by: Leif Madsen<EMAIL_ADDRESS>Cherry picks changes from d23f5ab
$ Resolved Conflicts:
$	build/get_new_operator_sdk.sh
$	build/stf-run-ci/README.md
$	build/stf-run-ci/tasks/main.yml
$	build/stf-run-ci/tasks/setup_stf_local_build.yml
$	tests/smoketest/smoketest.sh

test

test

test

The build error is this (Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist):
Pulling image quay.io/centos/centos@sha256:dbbacecc49b088458781c16f3775f2a2ec7521079034a7ba499c8b0bb7f86875 ...
Getting image source signatures
Copying blob sha256:7a0437f04f83f084b7ed68ad9c4a4947e12fc4e1b006b38129bac89114ec3621
Copying config sha256:300e315adb2f96afe5f0b2780b87f28ae95231fe3bdd1e16b9ba606307728f55
Writing manifest to image destination
Storing signatures
Adding transient rw bind mount for /run/secrets/rhsm
STEP 1: FROM quay.io/centos/centos@sha256:dbbacecc49b088458781c16f3775f2a2ec7521079034a7ba499c8b0bb7f86875
STEP 2: RUN dnf -y install centos-release-opstools &&     dnf -y install python3-cherrypy python3-PyYAML python3-pysnmp                    python3-dateutil python3-click git                    prometheus-webhook-snmp                    python3-prometheus_client procps-ng lsof && dnf clean all
CentOS Linux 8 - AppStream                      161  B/s |  38  B     00:00
Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
error: build error: error building at STEP "RUN dnf -y install centos-release-opstools &&     dnf -y install python3-cherrypy python3-PyYAML python3-pysnmp                    python3-dateutil python3-click git                    prometheus-webhook-snmp                    python3-prometheus_client procps-ng lsof && dnf clean all": error while running runtime: exit status 1
Caching blobs under "/var/cache/blobs".

I've noticed that by default we are pulling the stable-1.2 branch of prometheus-webhook-snmp[1], log shows "Get stable-1.2 branch because same-named doesn't exist" (because there is no stf13/cp-xxxxx  branch).
Comparing stable-1.2 and stable-1.3 of that repo, I see that we have this change:
-FROM quay.io/centos/centos:centos8
+FROM quay.io/centos/centos:stream8

This could certainly be related to our error. There are a lot of other changes on the stable-1.3 branch of prometheus-webhook-snmp so I'm not sure if there is a specific reason we are pinning the CI testing back to stable-1.2 or not. This testing isn't going to affect what gets shipped, so I'm going to go ahead and adjust this default to see if that helps pass the tests.
[1] https://github.com/infrawatch/service-telemetry-operator/blob/27c8a401bf6a8b27b9332b12379935f7e702ace6/build/stf-run-ci/defaults/main.yml#L30

test

Building with the stable-1.3 branch of prometheus-webhook-snmp fixed it. Passing CI, Ready to merge.


[Discussion] Today I ran out of memory after running training for 35 hours (spectrogram processing)

I'm working on a large data set (500GB, 100k examples) tuned for splitting drums.
Over the past day and a half I've watched spleeter slowly and steadily take more and more RAM. I think I've loaded up about 128GB of WAVs (assuming the on disk caching is a 1.5 x increase in size compared to the source)... I started to get out of memory errors when I used 72 out of the 84GB of available ram.
When I checked out the dataset.py file, I do think I found that an iterator keeps adding to a variable. Now I'm wondering what is the next best step to handle this. I have some tight constraints on RAM when using a GPU in Google cloud. So I can't just load up with 256GB of ram and call it a day.
I'm confused as to why spleeter does caching to disk but doesn't clear out ram. Can I recover from this cache somehow? What should I expect for RAM sizing?
Obviously my next step is to cut down the training set but eventually I'd like to train it against the full set to see if that produces more accurate results.
Would love some insights as to how I should setup the env. I've burned through about $2k of VM/GPU time.. not something I can keep doing.

Hi @dustyny,
I've not trained a full spleeter model with a lot of data with the most recent versions of tensorflow, but I'm pretty sure you're not supposed to need that much RAM for training. So there should be something wrong in the data pipeline that causes some memory leak. The most obvious thing that could explain the issue would be caching in RAM. If you don't provide a path to the cache method of tensorflow dataset then it will cache in RAM, which in your setting is not possible (and I guess not necessary), so make sure to have filled the "training_cache" (and "validation_cache") parameters in your config file.
Otherwise, it may also be some tensorflow behavior that has changed from version 1, and that the pipeline is not handling well, and we may not be able to help on this in the short term.

It's don't think it's RAM caching, I set the parameters and I see the training and validation folders have nearly 200GB of data.
The code is a bit hard to read but I think I see an iterator in main  line 80 calling data into a variable but nothing that clears it.. I've tried tracing it in dataset.py but I keep getting lost, there's iterators calling iterators, calling iterators and reusing the variable name "dataset" for everything has made it a real mess to try to follow.

Hi, I'm not sure but I think I may be in a similar situation when I train and I'm not sure if its a bug. I have re-trained 3 times so far using MUSDB16HQ dataset  on 2 different Win10 PC's (one has 12Gb the other 24Gb) and 23 - 24hrs later, it just stops training and hangs at the same file (patrick talbot - set me free) wave file. I'm not sure if I'm running out of ram to be able to complete the last 1/4  but this is on 2 different PC's where my CPU resources are ~100%, I willl be maxxing out my ram shortly a re-train again just to see if it hangs at the same file again, Have you investigated if its tensorflow related? thx

Have you tried redownloading the MUSDB16HQ data set? Getting stuck on one file makes me think it has some sort of corruption. I haven't investigated if it's a Tensorflow related issue, I don't know the framework well enough to do that.. I think my issue was due to a VM with limited RAM mixed with some iterator that collects something in memory.. TBH I moved on to a newer OSS project.. I spent to much time and $$ trying to work this out with Spleeter..

I don't think its corrupt, it plays well in audacity and both my folders have the correct number of files (train=100, test=50).  something is preventing it from completing the task.. and its not lack of space but I think I can rule out the hardware because its on 2 computers. Just curious as to how many people have actually been able to train 100% on musdb

@deskstar90 it sounds like we have very different issues.. I haven’t used the musdb set but I do know it’s very common.

I'm not sure that ram can distinguish what type of dataset you're using, whatever it is, train will still fill the available ram.
I've tried to train at least 6x now with the same results where it just hangs. it may not be flushing the contents of Ram to be able to continue to complete the training, I don't know that's my impression. I will try using only half of the dataset to see what happens


Foundation plugins and smoothState.js

Hi there,
I am using the foundation framework and want to use smoothState.js. Does anybody know how to re-initialize foundation plugins with smoothstate.js? They only work if I hit browser refresh. All the other jQuery plugins seem to work - just foundation is giving me a headache... I think an example/demo would be really helpful, maybe the same for the other famous css framework Bootstrap?
I tried this:
Adding Plugins After Page Load
http://foundation.zurb.com/sites/docs/javascript.html#adding-plugins-after-page-load
and this:
Adding Content to Plugins
http://foundation.zurb.com/sites/docs/javascript.html#adding-content-to-plugins
...without success. I guess I just don't really when to use onAfter and onReady? And then I was given the link by @miguel-perez but I also do not understand how to include that.
https://gist.github.com/miguel-perez/476046a42d229251fec3
thank you!

Hi there,
I'm also having trouble with smoothState and Foundation 6.
I've tried using the instructions here, but to no avail. The fixes I've found via Google haven't helped, either.
I think I need some very basic help.
Below is what everything looks like, without any fixes applied.
app.js:
// @codekit-prepend "jquery.smoothState.min.js";

$(document).foundation();

$(function () {
    'use strict';
    var $page = $('#main'),
        options = {
            debug: true,
            prefetch: true,
            cacheLength: 2,
            onStart: {
                duration: 0, // Duration of our animation, was 250
                render: function ($container) {
                    // Add your CSS animation reversing class
                    $container.addClass('is-exiting');
                    // Restart your animation
                    smoothState.restartCSSAnimations();
                }
            },
            onReady: {
                duration: 0,
                render: function ($container, $newContent) {
                    // Remove your CSS animation reversing class
                    //            $container.removeClass('is-exiting');
                    // Inject the new content
                    $container.html($newContent);
                }
            },
            onAfter: function ($container) {
                $container.removeClass('is-exiting');
                console.log('after');
                $(document).foundation();
            },
        },
        smoothState = $page.smoothState(options).data('smoothState');
});

Before the closing body tag:
<script src="https://code.jquery.com/jquery-1.12.1.min.js" integrity="sha256-I1nTg78tSrZev3kjvfdM5A5Ak/blglGzlaZANLPDl3I=" crossorigin="anonymous"></script>
<script src="foundation.min.js"></script>
<script src="app-min.js"></script>

Any help is appreciated. Thanks!

I even submitted a premium request and heard nothing so far. Kind of frustrating, since my project depends on foundation and needs smooth page transitions.
If you are interested in premium support:
http://foundation.zurb.com/support/premium-support.html

@baj226 maybe we need to open a ticket here https://github.com/zurb/foundation-sites/issues. I created an entry to the forum but have not heard back:
http://foundation.zurb.com/forum/posts/39007-foundation-and-smoothstatejs

This is how I tried it, but without success.

}

I'm having no troubles with both F5 and F6.
Could you re-create in a Fiddle.

Hey @CHEWX, could you post you app.js? A Fiddle does not work as I need multiple pages, but I have put an example on a server, here: http://p352632.mittwaldserver.info/index.html
As you can see, on opening, the sticky menu on the right works normally. Now go to the second page, animated with smoothstate. Great. There the accordion works great too. But when you go to page1 again, sticky does not stick. Only Browser reload makes it work.

@phifa
What JavaScript paths are you compiling from Foundation.

This is my config.yml. It is just the standard sass based template with all plugins integrated:
# Your project's server will run on localhost:xxxx at this port
PORT: 8000

# Autoprefixer will make sure your CSS works with these browsers
COMPATIBILITY:
  - "last 2 versions"
  - "ie >= 9"

# UnCSS will use these settings
UNCSS_OPTIONS:
  html:
    - "src/**/*.html"
  ignore:
    - !!js/regexp .foundation-mq
    - !!js/regexp ^\.is-.*

# Gulp will reference these paths when it copies files
PATHS:
  # Path to dist folder
  dist: "dist"
  # Paths to static assets that aren't images, CSS, or JavaScript
  assets:
    - "src/assets/**/*"
    - "!src/assets/{img,js,scss}/**/*"
  # Paths to Sass libraries, which can then be loaded with @import
  sass:
    - "bower_components/foundation-sites/scss"
    - "bower_components/motion-ui/src"
  # Paths to JavaScript libraries, which are compined into one file
  javascript:
    # Libraries requried by Foundation
    - "bower_components/jquery/dist/jquery.js"
    - "bower_components/what-input/what-input.js"
    # Core Foundation files
    - "bower_components/foundation-sites/js/foundation.core.js"
    - "bower_components/foundation-sites/js/foundation.util.*.js"
    # Individual Foundation components
    # If you aren't using a component, just remove it from the list
    - "bower_components/foundation-sites/js/foundation.abide.js"
    - "bower_components/foundation-sites/js/foundation.accordion.js"
    - "bower_components/foundation-sites/js/foundation.accordionMenu.js"
    - "bower_components/foundation-sites/js/foundation.drilldown.js"
    - "bower_components/foundation-sites/js/foundation.dropdown.js"
    - "bower_components/foundation-sites/js/foundation.dropdownMenu.js"
    - "bower_components/foundation-sites/js/foundation.equalizer.js"
    - "bower_components/foundation-sites/js/foundation.interchange.js"
    - "bower_components/foundation-sites/js/foundation.magellan.js"
    - "bower_components/foundation-sites/js/foundation.offcanvas.js"
    - "bower_components/foundation-sites/js/foundation.orbit.js"
    - "bower_components/foundation-sites/js/foundation.responsiveMenu.js"
    - "bower_components/foundation-sites/js/foundation.responsiveToggle.js"
    - "bower_components/foundation-sites/js/foundation.reveal.js"
    - "bower_components/foundation-sites/js/foundation.slider.js"
    - "bower_components/foundation-sites/js/foundation.sticky.js"
    - "bower_components/foundation-sites/js/foundation.tabs.js"
    - "bower_components/foundation-sites/js/foundation.toggler.js"
    - "bower_components/foundation-sites/js/foundation.tooltip.js"
    - "bower_components/smoothstate/src/jquery.smoothState.js"
    # Paths to your own project code are here
    - "src/assets/js/!(app).js"
    - "src/assets/js/app.js"

@phifa
Ah, That's your problem.
Foundation 6 plugins are written in ES6 which needs to be transpiled to ES5. I'm assuming you are not doing this.
If you concat from the foundations-sites/dist/plugins/ folder, then it will work.

Hey @CHEWX, well, I am not sure I understand. The Transpiling goes via the normal foundation cli I think. When you read here: http://foundation.zurb.com/sites/docs/javascript.html#babel-required

Our JavaScript uses some features of ECMAScript 2015. If you use Foundation with a build system, you'll need to compile our code to ES5. We use Babel in our templates to do this.

I don't use Foundation like that, so not sure how you are doing it.
I would start off my including only the Foundation Plugins you are using, not all. Reduce the test case to the bare bones.

I managed to get it to work with Foundation 6.
After reading this answer on StackOverflow, I called Foundation init function on:
onAfter: function($container) {
  // Reinitialize foundation plugins
  $(document).foundation();
}


Break Handler Queue interface into producer and consumer of events interafces

This helps ensures that you only call handle where you want it to be done.
see: https://github.com/uProxy/uproxy/issues/650

done in #100


Don't display and render large execution results

Preface / Disclaimer
I'm not a frontend person. I haven't done any serious frontend / JavaScript work for a loooong time. This means that this change is likely sub-optimal, but my main goal is to get it to work without any regressions and negative impact.
Contributions and suggestions from people with more frontend experience are of course very welcome.
Description
The goal of this PR is to solve a long standing issue where st2web would freeze the whole browser window / tab when trying to view execution with large result.
It updates history details component so we don't try to render and display execution result if it's larger than some pre-defined value which can be overridden inside the config (I still need to do some more testing to come up with a reasonable default value for it).
If execution with a larger result is found, we instead display a link where user can either view or download raw result for that execution.

Implementation Details
This pull request depends on the server-side changes from https://github.com/StackStorm/st2/pull/4846.
In that PR (among many other changes), I also added a new result_size field to the each execution DB model. That field holds the size of serialized execution result in bytes.
Inside this PR I updated the history details component so we don't fetch execution result by default when fetching execution details. Then inside the action reporter component we check if result_size attribute is present and if it's not larger than the defined threshold (this attribute will only be present for new executions which utilize new DB format - that's intentional since solving it for old executions would mean more code to maintain and it's not worth the effort). If it's not, we re-fetch the whole execution including the result field and display it.
Before the result field is loaded, "Loading execution result..." message is displayed to signal the user that the result is being retrieved (honestly with those StackStorm DB changes, users shouldn't see that message all that often or for a long time since those API operations should be much, much faster now).
Ideas for a better implementation
Current implementation means we need to fetch execution twice when viewing the details when result size is smaller than the defined threshold.
This is not ideal, but it's also not the end of the world since with changes from https://github.com/StackStorm/st2/pull/4846 all those API / DB operations will now be much faster.
Technically, we could avoid that additional "get one" query and already fetch result for executions which don't exceed the threshold, but my React foo is not strong enough for that one.
It basically means that history details component would need to have access to the list of all the executions loaded in the left main panel (history panel) and then when retrieving single execution details, we would check if result_size for that execution id in executions list is smaller than the threshold and if it is, retrieve the complete execution including the result.

Alright, I pushed some more changes, cleanup, etc. The code is probably still non-ideomatic and hacky, but it seems to work.
I also tested it with CICD instance for backward compatibility reasons and it also works fine.
I did some more testing for the default value for the result size we still render and decided to go with 100 KB. Actual size very much depends on the result type (deeply nested JSON object vs more of a flat one), but 100 KB seems to be a reasonable compromise.
Anything beyond that and browser will start to get laggy and slow if result is more of a deeply nested JSON object.
In theory, we could have more complex heuristics (e.g. we could display objects up to 1 MB for results which are simple strings), but that would be kinda hard to implement since we would likely need to do it on the client side and likely the logic itself to determine if it's a "deeply nested object" / similar would be somewhat involved.
And it's worth noting that I was testing this on a relatively high end PC. On more lower powered devices (also mobile phones), it's likely that even 100 KB is already pushing it for larger JSON objects results.

Alright, I updated the code to utilize new API query parameter filter from - https://github.com/StackStorm/st2/pull/5197.
This allows me to simplify the code and make it even more efficient by always only performing a single "get one execution" API operations (aka win-win).

I see end to end tests use latest tag for docker images which points to current stable (aka 3.4.1).
This means end to end tests will fail until 3.5.0 is out since changes in st2web rely on new API added in master.
I guess one option would be to update end to end tests to respect version and branching, but this may be a bit of a ratbit hole...

OK, yeah, I tried to update it to use 3.5dev tag but that won't work since it uses outdated stackstorm/stackstorm image which I believe is deprecated.

I noticed gulp lint task produced tons of errors, but the build didn't fail.
It turns out out gulp task was not configured correctly so it didn't exit with non zero on failure.
I fixed that and also fixed existing build violations so the build passes - 65a85b0820209a8225844435d76ec30cdf5970d1, f53595b251c7df04dd494e957bc8848f42c1bde5.
Going forward we should be much more careful with issues like that. It's much easier to fix violations as they arise vs fixing them later then tons have accumulated...

I'll go ahead and merge this since it's blocking / preventing me from testing things end to end.
If there are any comments in the future, I will address them then.
And as mentioned above, this PR also contains a bunch of other fixes (lint task, using latest docker compose setup for end to end tests, etc.).

Another page / app which will need similar love in the future (right now it tries to load whole trigger instance payload which may contain large execution result).

But that's very low priority at this point. Unlike history / execution result page, this view is not used all that often.


Enchantment "Apply Effect to Actor" troubles

Talked this out with @krbz999 in the Discord and this definitely appears to be a bug, I have a feature with an enchantment activity.

As you can see, it has two effects, one that is an enchantment and one that is a "rider effect". Here is the issue. The effects both transfer successfully but I found a big issue with the way that rider effects work.
If I disable "Apply Effect to Actor" in the rider effect on the feature, then it is also disabled once enchanted to the piece of equipment. This prevents the rider effect from working properly.
If I enable "Apply Effect to Actor" in the rider effect on the feature, then it is immediately activated just by the creature possessing the effect, making the effect a permanent buff.
This is two effects that I have.

To be thorough, I also tried using the "Effect Suspended" option but the result was the same. There needs to be a way enable "Apply Effect to Actor" after it has been transferred to the enchanted item.

Rider effects are supposed to be given a flag to prevent this from happening, so we've somehow regressed on that.

Looks like we just forgot to update how we filter out these effects after we moved the flag location.


Feature request: exclude devDependencies

Your code should typically not depend on devDependencies. It would be nice if import-js didn't match on files that were in node_modules directories that belong to devDependencies.

Just for some context: this would include reading the package.json file in the project root, look at devDependencies and exclude all subfolders inside the node_modules folder with names corresponding to a key in the devDependencies object.

Yes, that's the idea.

I've been thinking about this for a while, and I don't know if it (always) makes sense to exclude devDependencies. What if you are working on a dev project? We could add a configuration option to turn it on/off, but I'm not sure that's good either. Do you want to elaborate some more on why this feature is important?

The idea here is similar to the reason we added the excludes option to prevent nested node_modules directories from being matched for import-js. Since the application doesn't explicitly depend on the module, it shouldn't be easy to import that JS into application code.
The more I think of this, however, I think instead of excluding devDependencies, we actually need to include only dependencies and peerDependencies for the node_modules directory. This is because shared dependencies can be deduped out into the top level and we don't want to import these packages unless the application explicitly depends on them. This will cover both cases.

I see, that makes sense. How about we squash together all dependencies and peerDependencies into an array of module names, then either feed that list into the find command or do some post-processing to filter out unwanted results. I like the former solution because it is more likely to become performant. Though we risk running into issues with the command becoming too long (though that's probably unlikely).
Both of these solutions assume that module names reflect folder names inside node_modules.

...it also assumes that the node_modules folder is always named node_modules.

@lencioni I tried a quick implementation. I played around with it a bit in the Brigade repo, and I'm not sure i see the value of the extra filtering. I think it has the potential of making import-js feeling non-deterministic. The behavior is "correct", but a bit magical.
Can you try it out and see if you think it's worth completing? I need to add tests and perhaps clean up some code. But I want to make sure you try it out first.

I think I got that commit backwards...

I'll wait to try it out until the commit is forwards. Let me know when you'd like me to take a look.

Alright, I turned it around:
https://github.com/trotzig/import-js/commit/166016526defb336e6a87eb48cf8dc8a03483b7b
Try it out and let me know what you think!

This is great! In the Brigade codebase the most painful thing that I regularly import is Utils which goes from:
[import-js] Pick js module to import for 'Utils': (0.16s)
1: autoprefixer/lib/utils
2: center-align/utils
3: esutils/lib/utils
4: express/lib/utils
5: flux/utils
6: handlebars/dist/amd/handlebars/utils
7: handlebars/dist/cjs/handlebars/utils
8: handlebars/lib/handlebars/utils
9: hawk/lib/utils
10: hawk/test/utils
11: inquirer/lib/utils/utils
12: intl-messageformat/lib/utils
13: intl-messageformat/src/utils
14: jstransform/src/utils
15: qs/lib/utils
16: qs/test/utils
17: react-intl/lib/utils
18: react-intl/src/utils
19: support/utils
20: tether/src/js/utils
21: uglify-js/lib/utils
Type number and <Enter> or click with mouse (empty cancels):

to:
[import-js] Pick js module to import for 'Utils': (0.08s)
1: flux/utils
2: intl-messageformat/lib/utils
3: intl-messageformat/src/utils
4: react-intl/lib/utils
5: react-intl/src/utils
6: support/utils
7: tether/src/js/utils
Type number and <Enter> or click with mouse (empty cancels):

I think we should maybe even take this a step further and treat node_modules entirely differently--instead of adding each of the dependencies and peerDependencies directories to the list of directories to find files in, we should just treat these as top-level things that can be imported. That would cut down this list from 7 to 1, which would be really nice. Plus, most of the time you are importing the package itself which uses main. For the special cases where you need something deeper, you could add some configuration to .importjs.json. What do you think?

Or if not for every dependency, perhaps just the ones with a main? You could probably combine it with some of the code from 716bfda0684.

That's a good idea, I like it. If I have time today or tomorrow, I can try it out. You are also welcome to give it a shot if you like.
I'm a little bit worried that reducing the list of matches can lead to people not trusting the plugin, or think that it's broken in some way (smarter node_modules matching will make the tool a little bit more "magic"). But I think it's fine in this case, because we can still allow people to put node_modules in lookup_paths, and then get the old find-all behavior. But if it's left out (which we can recommend in the README), we have a much more efficient importer in place.

Alright, let me know what you think of https://github.com/trotzig/import-js/commit/a15a482217b3e464b02d4c1077c41c1cd6d06b44
I tried it out quickly, and I think it might be exactly what we need! Even though I didn't focus on performance, simply avoiding to run the find command inside node_modules helps a_lot.
Make sure to modify .import-js.json and remove node_modules from lookup_paths before you try.
Oh, and Utils?
[import-js] Imported `support/utils` (0.07s)

:smiley:

I just tried it out and it seems to work perfectly. This is a solid improvement. :shipit:

Thanks! I cleaned it up a bit and squashed the WIPs into a single commit pushed to a new branch. I'll remove the WIP branch eventually.


feat: [] Next.js preview mode

What will change?
Added support for Next.js their preview mode. https://nextjs.org/docs/advanced-features/preview-mode. API routes have been added to handle previews URLs, and to reset the preview mode.
A previewGraphQlClient was added which is used in favor of the graphQlClient when the preview mode is active. The index and slug page have been modified accordingly to account for this.
Several typos have been fixed as well.

This was merged successfully! The status got messed up by a Github outage


Further improvements

[ ] Add callback for when some data has been changed

Probably something like FnMut(usize, u8) -> ()
As an alternative use a Broadcast like thing which would be more easy to implement

[ ] Add callback when the position has been changed

Something like FnMut(usize) -> ()
As an alternative use a Broadcast like thing which would be more easy to implement

[ ] Goto function

Function that takes a usize and jumps to that very position

[ ] PageUp/-Down support

I'm not sure how many lines should be jumped though. Any ideas?

[ ] Add support for non-ascii encoder

I think having a single mapping is fine, so Fn(u8) -> char should be good
Having multi-byte encoding is probably way too much to ask and not feasible for a hexcode viewer

Wow - I love the list - thanks for composing it. Most likely I am going to use all these planned features in my project.
Regarding page up/down - Cursive's scroller.scroll_down (and up) automatically move by a visible area - so a "page". Here is a bug in the documentation saying it's by one line, but definitely it's a page. Here is the recipe:
        .on_pre_event_inner(Key::PageUp, |v, _| {
            let scroller = v.get_scroller_mut();
            if scroller.can_scroll_up() {
                scroller.scroll_up(scroller.last_outer_size().y.saturating_sub(1));
            }
            Some(EventResult::Consumed(None))
        })
        .on_pre_event_inner(Key::PageDown, |v, _| {
            let scroller = v.get_scroller_mut();
            if scroller.can_scroll_down() {
                scroller.scroll_down(scroller.last_outer_size().y.saturating_sub(1));
            }
            Some(EventResult::Consumed(None))
        });


Implement higher order reducers

ref #525
this sits on the other 2 prs
closes #503

this sits on the other 2 prs

I'm assuming #708 and #709? Will looks at again when updated after both are in. :+1:

rebased


Add flag to disable apperance embedding

Sorry, I had auto-merge turned on :( I would have updated it.


Pepoloco new contribution

Description
Added shadows on the buttons when you hover on , based on the theme.
Related Issues
List any related issues or pull requests that are being addressed.
Checklist

[x] I have tested my changes locally.
[x] I have added necessary documentation or updated existing documentation.
[x] My code follows the project's coding conventions and style guidelines.
[ ] All tests pass successfully.
[ ] I have added/updated relevant unit tests.

Additional Notes
The ternary operator is not catching the dark theme only the white mode , so both themese have blue hover shadow , have to look into this.

Hey Curtis , can you check my latest push , I think i've fixed the code.
On Sun, Nov 26, 2023 at 9:57 AM Curtis Thomas @.***>
wrote:

@.**** requested changes on this pull request.
Hello :)
There is a build error in your code:
image.png (view on web)
https://github.com/Curtis-Thomas/free-api-list-with-react/assets/83663235/e8b89d9c-e450-4539-8a4d-e1277ca3bf67
—
Reply to this email directly, view it on GitHub
https://github.com/Curtis-Thomas/free-api-list-with-react/pull/92#pullrequestreview-1749298285,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANN45LCOWD4VZSMGNZBY23LYGLY57AVCNFSM6AAAAAA72JKJJSVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTONBZGI4TQMRYGU
.
You are receiving this because you authored the thread.Message ID:
@.***
com>


Desktop APP crashes with .NET 4.8

I have updated my windows to Windows 10, version 1903 and .net framework 4.8. After upgrade when I run my wpf application on my machine, it gets crashed.
My machine is 64 bit and the wpf application is 32 bit.
It is a WPF based application built using .NET 4.5 and it consumes a C++ based library also. The app is working fine with .NET 4.7 framework but gives issue with .NET 4.8 only.
Below is the event viewer logs which we got after our application got crashed "XYZ.exe"
=== Application Error (9/6/2019 10:00:01 AM)
Faulting application name: xyz.exe, version: <IP_ADDRESS>2, time stamp: 0x5bf3fdf0
Faulting module name: clr.dll, version: 4.8.3801.0, time stamp: 0x5cabfed2
Exception code: 0xc0000005
Fault offset: 0x000e58e5
Faulting process id: 0x275c
Faulting application start time: 0x01d564ad949b47c3
Faulting application path: C:\Program Files (x86) \xyz.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Report Id: 860f2394-15f2-44af-a7ed-af02c4593dca
Faulting package full name:
Faulting package-relative application ID:
=== .NET Runtime (9/6/2019 10:00:01 AM)
Application: xyz.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an internal error in the .NET Runtime at IP 708A58E5 (707C0000) with exit code 80131506.
=== Application Error (9/3/2019 8:21:45 AM)
Faulting application name: xyz.exe, version: <IP_ADDRESS>2, time stamp: 0x5bf3fdf0
Faulting module name: msvcrt.dll, version: 7.0.17763.475, time stamp: 0xba51b082
Exception code: 0xc0000005
Fault offset: 0x0008c0d4
Faulting process id: 0x3574
Faulting application start time: 0x01d56253d57367a6
Faulting application path: C:\Program Files (x86)\ xyz.exe
Faulting module path: C:\Windows\System32\msvcrt.dll
Report Id: 9b8ae8af-ed7a-4b1c-83f4-a75e017a1058
Faulting package full name:
Faulting package-relative application ID:
=== .NET Runtime (9/3/2019 8:21:44 AM)
Application: xyz.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: exception code c0000005, exception address 76A5C0D4
Appreciate if someone can share optimal solutions.

We can try to provide the crash dump files but will that help in identifying the issue ?
please load crash dump files at google drive location:
https://drive.google.com/open?id=1KSUaMm7dGbzt_4SZG0h13DDWZ92cHHMk

Same problem with a windows service.

@jeffschwMSFT @NKarnam15

@jitterbuffer thanks for reporting this.  If possible, can we move this .NET Framework issue to VS Feedback?  That way we can transfer files (perhaps privately).
To help make this actionable we need a dump at the point of the crash.  Can you follow these stesp?
o	Download procdump from: https://download.sysinternals.com/files/Procdump.zip
o	Install procdump as the just-in-time debugger:
	mkdir c:\dumps (or wherever you want dumps to go)
	procdump -ma -i c:\dumps
o	Run the app, observe the crashing behavior, and verify that a dump has been written to c:\dumps
o	Uninstall procdump as the JIT debugger:
	procdump -u
Thanks

Hi @jeffschwMSFT ,
To create a full dump of the crash, we have already asked users to create following registry entries on their machine.
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\xyz.exe]
"DumpCount"=dword:00000005
"DumpFolder"=hex(2):25,00,4c,00,4f,00,43,00,41,00,4c,00,41,00,50,00,50,00,44,
00,41,00,54,00,41,00,25,00,5c,00,43,00,72,00,61,00,73,00,68,00,44,00,75,00,
6d,00,70,00,73,00,00,00
"DumpType"=dword:00000002
I believe, this is equivalent to you what you are suggesting to achieve through crash dump. However, it looks easier to follow procdump steps suggested by you. So I am going to suggest these also to the users.

Hi @jeffschwMSFT ,
Posted issue on visual studio feedback, please see below url
https://developercommunity.visualstudio.com/content/problem/753983/desktop-app-crashes-with-net-48.html

Thanks @jitterbuffer, I can see the issue internally and is in the right place.  I am going to close this issue and track it via that item.


The following signatures were invalid: EXPKEYSIG 1BFC33752912B99C

When trying to fetch the deb from https://nuitka.net/deb, we see the following error (after adding the repository key):
The following signatures were invalid: EXPKEYSIG 1BFC33752912B99C

Related older issue: #569

It seems that the gpg is updated on the 19th last month but when I check it it show me that is an old version
wget -O - https://nuitka.net/deb/archive.key.gpg | gpg
--2024-02-01 16:00:34--  https://nuitka.net/deb/archive.key.gpg
gpg: WARNING: no command supplied.  Trying to guess what you mean ...
Resolving nuitka.net (nuitka.net)... <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, ...
Connecting to nuitka.net (nuitka.net)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3147 (3,1K) [application/pgp-keys]
Saving to: ‘STDOUT’

-                        100%[==================================>]   3,07K  --.-KB/s    in 0s

2024-02-01 16:00:34 (61,8 MB/s) - written to stdout [3147/3147]

pub   rsa4096 2013-11-10 [SC] [expired: 2023-12-18]
      D96ADCA1377F1CEB6B5103F11BFC33752912B99C
uid           Kay Hayen<EMAIL_ADDRESS>sub   rsa4096 2013-11-10 [E] [expired: 2021-12-17]

Yes, I updated it, and this matches the public key of it, or so I am thinking. I might be missing a step. They key was extended and I think the archive is signed with it, however, that's not enough.

It works! Do not forget to update it in about two years
Thanks for that!
CU
  Michael
On Mon, 2024-02-26 at 08:19 -0800, Kay Hayen wrote:

Can you retry. What happened is that my ansible config removed the
archive key I had manually deployed there. I forgot it gets on the
server automatically, so it worked for a while until I used ansible
for the web server with the full config. I am expecting it to work
now.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID:
@.***>

Yeah works like a charm. Thanks for looking into it!


Build issue

I'm on osX, and I've followed your installation guide. I know my Rust extensa toolchain works because I've been able to compile https://github.com/espressif/rust-esp32-example and make it work
when running cargo build I get:
[...]
warning: cargo:rustc-link-arg requires -Zextra-link-arg flag
warning: cargo:rustc-link-arg requires -Zextra-link-arg flag
'+memctl' is not a recognized feature for this target (ignoring feature)
'+memctl' is not a recognized feature for this target (ignoring feature)
'+memctl' is not a recognized feature for this target (ignoring feature)
'+memctl' is not a recognized feature for this target (ignoring feature)
'+memctl' is not a recognized feature for this target (ignoring feature)
'+memctl' is not a recognized feature for this target (ignoring feature)
   Compiling rust-esp32-std-hello v0.17.4 (/Users/primet/work/gouach/embedded-nobackup/rust-esp32-std-hello)
'+memctl' is not a recognized feature for this target (ignoring feature)
'+memctl' is not a recognized feature for this target (ignoring feature)
error: linking with `ldproxy` failed: exit status: 101
[...]
  = note: Running ldproxy
          thread 'main' panicked at 'Cannot locate argument '--ldproxy-linker <linker>'', /Users/primet/.cargo/registry/src/github.com-1ecc6299db9ec823/ldproxy-0.1.3/src/main.rs:43:10
          stack backtrace:
             0: _rust_begin_unwind
             1: core::panicking::panic_fmt
             2: core::option::expect_failed
             3: ldproxy::main
          note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

warning: cargo:rustc-link-arg requires -Zextra-link-arg flag
That's not a good indication. Can you run run cargo --version and rustc --version and let me know the outcome?
(the '+memctl' is not a recognized feature for this target (ignoring feature) you can safely ignore)

❯ cargo --version                                                    ±[●][main]
cargo 1.54.0-dev
❯ rustc --version                                                    ±[●][main]
rustc 1.54.0-dev