Can't load_model with error “Optimizer weight shape (256, 32) not compatible with provided weight shape (4,)”

I have 3 trained model file.

left_branch.h5
right_branch.h5
concat.h5

The model concat.h5 is fine-tuned by concatenating from the two pre-trained model as the initial model(left_branch.h5, right_branch.h5).
While left_branch.h5 and right_branch.h5 model file can be load by function keras.models.load_model(),
but I load the trained concat.h5 formatted model file, I get the error blew.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 167, in load_model
    model.optimizer.set_weights(optimizer_weight_values)
  File "/usr/local/lib/python2.7/dist-packages/keras/optimizers.py", line 97, in set_weights
    'provided weight shape ' + str(w.shape))
Exception: Optimizer weight shape (256, 32) not compatible with provided weight shape (4,)

Similar problem is mentioned in issue #3964. I tried @hadikazemi solution. It worked for me.

@xisnu Thanks for your advice! I trained many models and saved into many files, so it is laborious to delete the optimiser_weight manualy which advised by issue #3964.
Finally, I solved it by a using load_weights, as follows:
# serialize model to JSON
model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model into h5 file")

# later...

# load json and create model
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("model.h5")
print("Loaded model from disk")

# evaluate loaded model on test data
loaded_model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
score = loaded_model.evaluate(X, Y, verbose=0)

In my case, the problem was that on one machine everything was fine, on another I got "Optimizer weight shape ... not compatible with provided weight shape ..." exception.
The problem, turned out to be in Keras installed using different methods on different machines.
Although it had the same version 1.1.0 on both the machine that built the model and the machine that loaded the model, the files in /usr/local/lib/python2.7/dist-packages/keras/ were different (in particular, /usr/local/lib/python2.7/dist-packages/keras/models.py).
Uninstalling and re-installing Keras on the problematic machine, solved the error, since now files in /usr/local/lib/python2.7/dist-packages/keras/ on both the machine that built the model and the machine that loaded the model were identical (to what I had on the machine that built the model).

Confirming this issue with loading models stored after train. Issue is with weights shapes; it helps to remove optimizer from the model file. Then it loads successfully. I updated Keras to recent 1.1.0, and use Python 2.7.12 from Anaconda 4.0.0 but it does not help. I also found that if you load model in Spyder 2 times, then second load if fine. Seems to be really cool if the issue is fixed. Thanks

@fsonntag
I think the problem comes from https://github.com/fchollet/keras/commit/028aae19bf5ae6efe0b32d25d1c700224eebfcf9
If you do not use weights.sort(key=lambda x: x.name),
weights order will be different from when it is saved.
weights.sort(key=lambda x: x.auto_name) should not be used.
All the weights should have proper names and their order should be stable.

Sorry, I don't really get your point. So you're saying, name should always be set in order to solve this issue? Since Theano doesn't set name, I introduced sorting by auto_name. Alternatively one can copy the auto_name attributes to name.

Since Theano doesn't set name

@fsonntag
Is it correct???
I think Theano have both name and auto_name.
I believe you do not need to use auto_name.
https://github.com/fchollet/keras/blob/master/keras/backend/theano_backend.py#L67
https://github.com/Theano/Theano/blob/master/theano/gof/graph.py#L387

Yes, Theano variables have a name property, but it is set to None when trying the weights. That was part of the fix in 028aae1. As name was None under Theano, the sorting didn't work. Under Python 2, no error was raised. It was just ignored and not sorted. Python 3 raised an error, that's why I discovered that issue.

Yes, Theano variables have a name property, but it is set to None when trying the weights.

Do you mean creating variables without a name?
I think all the variables in keras.layers classes assign names.
Maybe you should change it to weights.sort(key=lambda x: x.name if x.name else x.auto_name).
I checked that it solves the load_model() problem.

I am still facing the same problem when I load_model()
I've upgraded the latest version of keras
Has this issue be solved?

You have to save by the latest git version and load_model() by the latest git version.
Did you do that?
It is impossible to load Keras 1.1.0 and 1.1.1 saved model.

I fix it by cloning the repo and reinstall keras
(I use pip to uninstall and install again, but it does't work until I install keras by clone the repo)
Thanks!

我模型算完了，就不重装了。
修改 /usr/local/lib/python2.7/site-packages/keras/optimizers.py
行 line 92 add
amap={}
for w in weights:
amap[str(w.shape)]=w
for pv, p, w in zip(param_values, params, weights):
w=amap[str(pv.shape)]
if pv.shape != w.shape:
print(p,w)
raise Exception('Optimizer weight shape ' +
str(pv.shape) +
' not compatible with '
'provided weight shape ' + str(w.shape))
weight_value_tuples.append((p, w))
有点坑，要print model.summary() 中的，"Param #" 列不一样才行。

I am using Keras 1.1.1 and am having the same problem. Deleting the optimizer weights as a workaround works for me.
Just in case someone needs to do the same, here's the code:
import h5py
f = h5py.File('model_file.h5', 'r+')
del f['optimizer_weights']
f.close()

I can confirm the same issue. Deleting the optimizer_weights as suggested by @dteoh works if your model is already trained.
However, I'm working on a project where I train for N number of epochs, stop training, adjust learning rate, then re-load and continue training the model.
In this case, deleting my optimizer_weights would lead to the error:
AttributeError: 'Model' object has no attribute 'optimizer'
This error makes sense since the optimizer attribute has been deleted. But as it currently stands, I can't figure out how to:

Train a (non-Sequential) model.
Serialize it.
Load it from disk.
Adjust learning rate.
Continue training.

Without the error others in this thread have mentioned.

Another solution is to name each layer in the model.
This helps if you're using checkpointing or something similar, where you cannot save the weights separately.
For ex: model.add(LSTM(10, name='lstm1'))
This worked for me in keras 1.1.1 and theano 0.8.2

@WarpTime Have you tested it for a model with CTC loss? Because that has been a problem for me.

@xisnu No. I have only tested it on crossentropy.

Sample problem here using SGD and categorical_crossentropy.

Something like this....
model = Sequential()
model.add(LSTM(100, return_sequences=True, input_shape=(timesteps, data_dim), name='input'))
model.add(Dropout(0.5, name='dr1'))
model.add(LSTM(100, name='ls1'))
model.add(Dropout(0.5, name='dr2'))
model.add(Dense(100, name='d1'))
model.add(Dense(nb_classes, activation='softmax', name='output'))
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])

After updating from Keras 1.X to Keras 2.0.1, I have the same problem:
  File "./ensemble.py", line 56, in <module>
    models = [load_model(model_path) for model_path in model_names]
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 272, in load_model
    model.optimizer.set_weights(optimizer_weight_values)
  File "/usr/local/lib/python2.7/dist-packages/keras/optimizers.py", line 79, in set_weights
    'provided weight shape ' + str(w.shape))
ValueError: Optimizer weight shape (32,) not compatible with provided weight shape (3, 3, 3, 32)

Deleting the optimizer_weights group (dataset? object? directory? what is the right term?) from the HDF5 file with hdf5view fixed it.
I updated all my models with
#!/usr/bin/env python

"""Make keras 1.x models usable in keras 2.x."""

import glob
import h5py

model_files = sorted(glob.glob('*.h5'))
for model_file in model_files:
    print("Update '{}'".format(model_file))
    with h5py.File(model_file, 'a') as f:
        if 'optimizer_weights' in f.keys():
            del f['optimizer_weights']

Also: Is it possible to save the model without optimizer parameters?

The following codes really worked! Thank you!
    with h5py.File(model_file, 'a') as f:
        if 'optimizer_weights' in f.keys():
            del f['optimizer_weights']

I had the same problem and what it seemed to work for me was to set the compile flag to False  in the load_model function. Afterwards one can compile the model with the previously used optimizer.
model = load_model('my_model.hdf5', compile=False)
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

in case you are having issue in deleting the keys from the model, check this out:
https://stackoverflow.com/questions/44236522/keyerror-couldnt-delete-link-cant-delete-self

It seems that the problem of loading optimizer weights is patched in Keras 2.0.8. The patch does not address the reason why the weights have different size but at least you no longer get an exception.

@oarriaga  It works for me, thank you !

Closing as this is resolved


Vca test

@ggmartins can you please take a look at the code and see if the changes make sense

modified the netrics integration. @ggmartins can you check now?

could you please remove: package/deb/nm-exp-active-netrics/* (including nm-exp-active-netrics) and the .deb package? thanks

shall i add it to gitignore?

yeah, that works

yeah, that works

check now


GPII-1680: Document fluid.componentForPath(path)

Code pull request: https://github.com/fluid-project/infusion/pull/747


UV Index is some outrageous number, looks like some integer overflow

Describe the bug

When viewing the data for UV Index at 4:05AM in Elk Grove, CA, USA the number is like something a billion or a trillion.
Expected behavior
I don't think that high of a UV index is possible, it should just be around 1-20 ish
Actual behavior
It shows<PHONE_NUMBER>3852000
API Endpoint
Production
Location
Elk Grove, California, United States
Other details
Viewed via Merry Sky
Troubleshooting steps

[x] I have searched this repository and Home Assistant Repository to see if the issue has already been reported.
[X] I have read through the API documentation before opening this issue.
[X] I have written an informative title.

Duplicate of #229


Implement gating mechanism form the Roslyn configuration and enable daily builds

In Beta 8, picking up a new build of Roslyn after we haven't for a while caused EF to fail, and they had to react quickly.  We should implement a "gating" mechanism similar to that @pranavkm implemented for CoreCLR:  Try out a new Roslyn.  If it's good, take it, if not, CI rotation folks would need to follow up, but we can continue to use the 'last known good' Roslyn.  The gating mechanism should probably involve trying to build EF.  We should run this daily like CoreCLR.
cc @Eilon @divega

:+1: Our codebase (all of ASP.NET and EF in particular) are good scenario tests for Roslyn and it makes sense to do this to find bugs before they become a pain.

I believe the current plan is to integrate Roslyn gate to the current gate ("CoreCLR").
@ryanbrandenburg will you be doing this work?

I haven't been instructed to do this work, but I'd be interested and willing assuming @Eilon doesn't want someone else or have other plans for me. It might be worth just confirming that we still want this for RC2 considering how late in the game it is though.

I'm fine with anyone doing this, so long as said person knows what to do here :smile:

Basically we try taking a new Roslyn in the "CoreCLR" configuration. Possibly running an EF build with it in addition to running "Build Universe, run MusicStore" test.

Looks like @pranavkm is already on it: https://github.com/aspnet/CoreCLR/pull/80

We need to unstick the Roslyn versions in our project.json to go with this change -

https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNetCore.Mvc.Razor/project.json#L19
https://github.com/aspnet/EntityFramework/blob/bfadc70c7f677641199a3c7ed80c20022379158c/test/Microsoft.EntityFrameworkCore.Relational.Design.FunctionalTests/project.json#L17
https://github.com/aspnet/Scaffolding/blob/078ee20cd6359d0d4b1ec50378ece55a6e9c072b/src/Microsoft.Extensions.CodeGeneration.Utils/project.json#L7
https://github.com/aspnet/Scaffolding/blob/078ee20cd6359d0d4b1ec50378ece55a6e9c072b/src/Microsoft.Extensions.CodeGeneration.Templating/project.json#L9

(might be more in Scaffolding).

Looks like there's also:

https://github.com/aspnet/Scaffolding/blob/078ee20cd6359d0d4b1ec50378ece55a6e9c072b/src/Microsoft.Extensions.CodeGeneration/project.json#L9
https://github.com/aspnet/Scaffolding/blob/078ee20cd6359d0d4b1ec50378ece55a6e9c072b/src/Microsoft.Extensions.CodeGeneration.EntityFrameworkCore/project.json#L10


Better harmonized dimensions for query metrics.

All query metrics now start with toolChest.makeMetricBuilder, and all of
those now start with DruidMetrics.makePartialQueryTimeMetric. Also, "id"
moved to common code, since all query metrics added it anyway.
In particular this will add query-type specific dimensions like "threshold"
and "numDimensions" to servlet-originated metrics like query/time.

👍

Failed due to #2253 - https://travis-ci.org/druid-io/druid/jobs/144765914 - bouncing.

👍 after travis


chore: Update ruff and fix lints

Fixes the lints in ruff 0.6 that are caused CI errors in https://github.com/CQCL/guppylang/pull/386

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.88%. Comparing base (d542601) to head (fac13d7).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #387      +/-   ##
==========================================
- Coverage   90.88%   90.88%   -0.01%
==========================================
  Files          48       48
  Lines        5341     5340       -1
==========================================
- Hits         4854     4853       -1
  Misses        487      487

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Please update libwebp to 1.3.2 due to CVE-2023-4863

see https://www.upguard.com/blog/libwebp-cve-2023

updated 1.3.2


open_with update warning

See discussion at https://github.com/openmicroscopy/openmicroscopy/pull/5317/
This PR describes the problem with previously-configured open_with overriding the new 'Image Viewer' config and links to the PR above for how to manually update.
cc @jburel.

Would a table of example help here? e.g. "If you did X in version Y, then now do Z..."

@joshmoore I'm all for improving clarity but I'm not sure a table would help (unless I'm not quite understanding you)? There's only a single case where you do need to do the manual update, so I'd prefer not to add a matrix of all the other cases where you don't need to.

Fair enough, but here's a heads up that I found the prose here confusing. Perhaps at least an example of before/after. If we've essentially broken a production configuration value, we should be take care to make sure it's clear.

I find this text very confusing too.
I'm also not sure we want to start a precedent of linking to PRs for instructions rather than just further reading if people are particularly interested in the nuts and bolts.

I have tried to simplify the text a little.
I could make the issue clearer by describing a lot more about open_with config and the viewer.view config and how settings are saved to etc/grid/config.xml and the issues we have with updating single items in the lists (as mentioned on the PR) as well as the fix itself?
However, I was trying not to make this too bloated since it is an edge case and won't apply to most users. I'm hoping that most users will simply read the "If.... " statement and understand that they don't need to do anything.
I'm happy to have suggestions on how to make the text clearer, but "it's confusing" is not so helpful as I've already tried to make it as clear as I can.

As discussed with @hflynn: Move the 'if' statement first, remove the background logic mentioning config.xml and mention iviewer as example because that's the only case where we've told users to config both webclient.view and open_with.

👍 I think this is clearer

@joshmoore Happy with the last commit?

Yes, thank you.


Fix dropdown menu position when explore repos

This will fix the dropdown position when explore the repos.
Before

After

LGTM

Codecov Report

Merging #2808 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2808   +/-   ##
=======================================
  Coverage   26.85%   26.85%
=======================================
  Files          89       89
  Lines       17600    17600
=======================================
  Hits         4727     4727
  Misses      12187    12187
  Partials      686      686

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d545e32...8a87bd7. Read the comment docs.

Make LG-TM work


Creates resource state builder, for transforming RDF to resource state.

closes #867.

@jcoyne 	Can you have another look at the PR?


Make the auto-import more flexible

Right now it's hardcoded to look in a /Activity subdirectory in the specified Garmin folder. I import my files from other devices (Wahoo) using other mechanisms, and I'm not using a /Activity folder. It would be nice if it could be optional to NOT look in that folder.

Hi @evensolberg, thanks for opening up that issue.
I think it would make sense to improve the flexibility of the import mechanism. Up until now I simply assumed all devices had an Activity directory in which it stores its fit files. In case the name of your "activities" folder is different this won't work of course.
One potential solution to this could be to make the folder name in which the activities are stores configurable via the settings page. This would be similar to what was done in #91 (what was changed again later on).
In order to better understand the situation with e.g. Wahoo devices, it would be great if you could share the full path in which your activity fit files are stored (once the file system of your device is mounted to your system).
For my Garmin device this would for example be
/run/user/1000/gvfs/mtp\:host\=091e_4b48_0000c4fa0516/Primary/GARMIN/Activity/<fit_file>.fit

Do you think this approach could work and solve your issue?

To be honest, I don't even know where the files are stored. I normally email them to myself and then dump them out into a directory, since I use them in multiple applications. So yes, being able to just point to an arbitrary directory would work nicely for me.
I tried copying the files to an /Activity folder, but somehow "Reimport Files" doesn't seem to work. Is there any way to see what's going on?

 In that case the settings option called "Path to Traces Directory" should do exactly what you are looking for.
For the sake of clarity, let me explain the difference:

Path to Garmin Device: Path to a directory in which your device/watch will be mounted in, in order to copy the fit files from the device onto the system workoutizer is running on. This process is called fit collector within the code base. The fit collector searches for a directory called Activity/ in order to copy the activity files from there. That is required since typically the sport devices have many folders with multiple different purposes, many of them containing fit files - in which workoutizer is not interested in. The directory specified is the source location for copying the fit files, target location is the "Path to Traces Directory".
Path to Traces Directory: Path to a directory on the same system on which workoutizer is running, e.g. your laptop (sometimes a Raspberry Pi, ...). The directory specified in this field could contain fit and gpx files in an arbitrary hierarchy. Workoutizer will check the folder for new activity files whenever it is started and will realize when a new file is added. New/unknown files are automatically imported. This processes is called File Importer in the code base. The path specified here could be any arbitrary path and does not require an Activity/ directory.

I suggest you give this option a try for importing your activity fit files into workoutizer.
Note, this holds true for the current version 0.17.1 - things might change in upcoming versions 😉

[...] but somehow "Reimport Files" doesn't seem to work. Is there any way to see what's going on?

For debugging you could run workoutizer in debug mode, by setting the following environment variable prior to running it:
export WKZ_LOG_LEVEL="DEBUG"

You should be able to see something like
04-25 19:37:33 - DEBUG - wkz.apps - started watchdog for incoming activity files in /workspaces/workoutizer/tracks

Where /workspaces/workoutizer/tracks should be the folder containing your activity files.
Let me know whether this is of any help or if I totally got your workflow wrong 😉

I will give this a try, thanks! I didn't realize that's what the other option was for. It would be nice to have some kind of feedback in the user interface during the import, though, since I have 100+ files in the directory, and no way of knowing if anything is happening.

Yea, you are right. The only feedback one could currently see, is by looking at the command line - which is of course not a good user experience... I created #123 for this purpose and will think about how this could be solved. Ideas or contributions on this are welcome ;)
Any news on the usage of Path to Traces Directory?

Nothing seems to be happening with the  Path to Traces Directory. I changed it, and clicked the Reimport button, and nothing happens. Some feedback mechanism would be nice. ;-)
As for the feedback in the UI, I'd love to see progression and maybe a tail of the last few log messages. If the logs could be output to a file, that would be handy too.

Ok, I get it... some feedback would be nice - agree on that :D (#123)
After changing the Path to Traces Directory you would need to re-run workoutizer to have the change become effective. For the initial setup/configuration and debugging I would recommend to have a closer look into the logging output in the terminal. Increasing verbosity by setting export WKZ_LOG_LEVEL="DEBUG" as mentioned above should give:
04-27 19:36:45 - DEBUG - wkz.file_importer - found 10 files in trace dir: /workspaces/workoutizer/tracks

Okay, I was finally able to get to this. Apologies for the delay. Here's what I'm seeing in the log. I went into the settings and clicked "Reimport Files" at 17:27:16.
05-02 17:26:37 - INFO - wkz.apps - using workoutizer home at /home/evensolberg/.wkz
05-02 17:26:37 - DEBUG - wkz.file_importer - triggered file importer
05-02 17:26:37 - DEBUG - wkz.file_importer - found 111 files in trace dir: /mnt/home/Drive/Drive/Data Analysis/Wahoo/Logs/FIT
05-02 17:26:37 - DEBUG - wkz.file_importer - importing file /mnt/home/Drive/Drive/Data Analysis/Wahoo/Logs/FIT/2019-04-12-140638-FITNESS 416C-2-12.fit ...
05-02 17:26:37 - DEBUG - wkz.file_importer - parsing FIT file ...
05-02 17:26:38 - INFO - django.utils.autoreload - Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
May 02, 2021 - 17:26:38
Django version 3.2, using settings 'workoutizer.settings'
Starting development server at http://<IP_ADDRESS>:8000/
Quit the server with CONTROL-C.
05-02 17:27:05 - WARNING - wkz.views - no activities found...
05-02 17:27:05 - INFO - django.server - "GET / HTTP/1.1" 200 23289
05-02 17:27:05 - DEBUG - wkz.views - fetching activity data for table page 0
05-02 17:27:05 - DEBUG - wkz.views - reached end of the table
05-02 17:27:05 - INFO - django.server - "GET /activities_page/0 HTTP/1.1" 200 4
05-02 17:27:05 - INFO - django.server - "GET /static/img/favicon.ico HTTP/1.1" 200 311
05-02 17:27:16 - INFO - django.server - "GET /settings/ HTTP/1.1" 200 17963
05-02 17:27:18 - INFO - django.server - "GET /settings/reimport HTTP/1.1" 302 0
05-02 17:27:18 - DEBUG - wkz.file_importer - triggered file importer
05-02 17:27:19 - DEBUG - wkz.file_importer - found 111 files in trace dir: /mnt/home/Drive/Drive/Data Analysis/Wahoo/Logs/FIT
05-02 17:27:19 - INFO - django.server - "GET /settings/ HTTP/1.1" 200 18201
05-02 17:27:19 - DEBUG - wkz.file_importer - importing file /mnt/home/Drive/Drive/Data Analysis/Wahoo/Logs/FIT/2019-04-12-140638-FITNESS 416C-2-12.fit ...
05-02 17:27:19 - DEBUG - wkz.file_importer - parsing FIT file ...
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/evensolberg/.local/lib/python3.8/site-packages/wkz/file_importer.py", line 57, in reimport_activity_files
    _run_file_importer(models, importing_demo_data=False, reimporting=True)
  File "/home/evensolberg/.local/lib/python3.8/site-packages/wkz/file_importer.py", line 73, in _run_file_importer
    _run_parser(models, trace_files, importing_demo_data, reimporting)
  File "/home/evensolberg/.local/lib/python3.8/site-packages/wkz/file_importer.py", line 87, in _run_parser
    activity = _parse_and_save_to_model(
  File "/home/evensolberg/.local/lib/python3.8/site-packages/wkz/file_importer.py", line 129, in _parse_and_save_to_model
    parser = _parse_data(trace_file)
  File "/home/evensolberg/.local/lib/python3.8/site-packages/wkz/file_importer.py", line 356, in _parse_data
    parser = FITParser(path_to_file=file)
  File "/home/evensolberg/.local/lib/python3.8/site-packages/wkz/file_helper/fit_parser.py", line 20, in __init__
    super(FITParser, self).__init__(path_to_file)
  File "/home/evensolberg/.local/lib/python3.8/site-packages/wkz/file_helper/parser.py", line 83, in __init__
    self._parse_records()
  File "/home/evensolberg/.local/lib/python3.8/site-packages/wkz/file_helper/fit_parser.py", line 30, in _parse_records
    for record in self.fit.get_messages():
  File "/home/evensolberg/.local/lib/python3.8/site-packages/fitparse/base.py", line 434, in get_messages
    message = self._parse_message()
  File "/home/evensolberg/.local/lib/python3.8/site-packages/fitparse/base.py", line 156, in _parse_message
    add_dev_field_description(message)
  File "/home/evensolberg/.local/lib/python3.8/site-packages/fitparse/records.py", line 450, in add_dev_field_description
    field_def_num = message.get('field_definition_number').raw_value
AttributeError: 'NoneType' object has no attribute 'raw_value'

Thanks for looking into this in more detail. The traceback provided clears things up:
Turns our workoutizer is not able to parse your 2019-04-12-140638-FITNESS 416C-2-12.fit file. Under the hood workoutizer relies on fitparse, which is a great Python library for parsing fit files and which is where the error originates from.
Could be that fitparse does not support fit files generated by wahoo devices. Could also be that wahoo fit files do not adhere to the fit format specifications. But before digging deeper in that direction I would like to rule out that the problem actually originates in workoutizer.
Therefore I would offer to analyze the problem with your fit files on my side a bit further. In case you don't mind sending me one of your files (which caused the above traceback), of course.

Sure, I can do that. Just give me a little time. Weekdays are somewhat busy. 😃

No worries, take your time 😃
Regarding transferring the file, you could either

use file.io
send me an email to<EMAIL_ADDRESS>or
zip and attach it to a comment here in this thread

2019-04-12-140638-FITNESS 416C-2-12.fit.zip
Attached is the first file it's trying to upload. Thanks for your patience.

Thanks for sharing your file.
I had a closer look and figured that the issue is actually coming from fitparse. It seems like it is related to this issue.
Which has a fix in version 1.2.0 and workoutizer currently uses 1.1.0 (due to some other migration issues). However, for testing I use both 1.2.0 and the latest master branch and both still failed. This time with a slightly different error, but I assume the root cause resides in this line returning None which leads to a subsequent KeyError in this line.
In order to have your wahoo device supported in workoutizer, it needs to be supported by fitparse first. It would therefore be great if you could open a bug ticket in their tracker.

In fact, this issue looks to be the same problem.

Closing this for now, since it is not a bug in workoutizer.
Also note, that I added feedback notifications when clicking "Reimport Files" or changing the "Path to Traces Directory" on the settings page (which closes #123) in version 0.18.0.


Feature Request: Look up simulators by UDID

It would be nice to have the option to look up the simulator by its UDID so we can automatically select the right simulator from the build scripts by using ${TARGET_DEVICE_IDENTIFIER}.
Something like:
"Usage: set-simulator-location [-c 0 0|-q San Francisco] [-s Simulator Name|-d Simulator UDID]"

This has been release as 1.4.0 https://github.com/lyft/set-simulator-location/releases/tag/1.4.0


Fixes Android RTCVideoView.onLayout issue

Android was crashing in the simulator (OS versions 5.1 and 6.0) and device (version 5.1.1). This fixes the simulator issue.

Thanks! I will test it later.

Great! Still crashing:

device
switch camera on simulator
Looking to see if I can fix.


Some questions about the environment

Hi, hongzi. I tried to reproduce your pensieve system. However, I found whatever bitrate the agent choiced, the rebuf time was always 0, then the agent would just choice one action for every chunk because it always received positive reward. I really do not understand how this happened. The data was downloaded from your dropbox. Any help would be great appreciated.

I really don't understand this either. We never (nor did those who reproduced our results) saw this before.

In common sense, when the buffer size is up to threshold, the environment should rebuffer to get next video chunk, but when the buffer is full, the rebuffer time is still 0, which is really strange.

I ran your pretrained model again and printed the result of rebuffering time, and I found it would be 0 most of the time, expect of the last few chunks. The rebuffering time here is just the fourth return value rebuf of the get_video_chunk function.

So the rebuffering time is not always 0 then. Also, I guess this means the pretrained model is doing a good job, isn't it?
Honestly I'm really confused by what you are saying but somehow I think (I might be wrong) our definitions of "rebuffering" differs. Could you check with our paper Section 2 (especially (2)), or the MPC paper [51] for reference?

It would be fine to avoid rebuffering when we test the model. However, if the rebuffering time is also 0 in training stage, the agent will always get positive reward and will choice same action as before. Then the agent will finally just choice one action for any videos. That's what confuse me most. Have you met similar problem in your training stage ?

Refer to the following code, in my training stage, the rebuffer penalty will be 0 due to the value of rebuf is always 0. Finally, the agent will learn to choice only one action to make smooth penalty as 0 because this can maximize its reward.
# reward is video quality - rebuffer penalty - smooth penalty
            reward = VIDEO_BIT_RATE[bit_rate] / M_IN_K \
                     - REBUF_PENALTY * rebuf \
                     - SMOOTH_PENALTY * np.abs(VIDEO_BIT_RATE[bit_rate] -
                                               VIDEO_BIT_RATE[last_bit_rate]) / M_IN_K

This looks strange and I think we never saw what you observed during our training. You might want to dig into this a bit and see where it went wrong. Notice that the average bandwidth is roughly in the midway of all available bitrate. Therefore, random policy (before any training) should result in fair amount of rebuffering.

I found this case will appear when I just use current state to train. If I combine past several states to train, things would be better and can get satisfactory results. Many thanks to your reply !


Using 12v ws2811, only first IC lights up.

Hello,
I'm stuck. I managed to get everything installed and got the esp8266 flashed and connected to my wifi. I can see commands coming through the serial monitor when I select different modes or colors in the web GUI. However those commands dont seem to affect the 12v ws2811 strip at all. Only the first three LEDs (1 IC) are permanently on blue. They select that at startup and then nothing else happens. Any help would be greatly appreciated, Ive been trying to figure this out nonstop for the last two days and I'm at a loss.
Here is the output from ip-address/esp_status:
"HOSTNAME" "McLighting01"
"version""2.2.0"
"heap":24376,
"sketch_size":414608,
"free_sketch_space":2727936,
"flash_chip_size":4194304,
"flash_chip_real_size":4194304,
"flash_chip_speed":40000000,
"sdk_version":"2.2.1(cfd48f3)",
"core_version":"2_4_2",
"cpu_freq":80,
"chip_id":1458376,
"animation_lib":"WS2812FX_DMA",
"pin":3,
"number_leds":100,
"button_mode":"ON",
"home_assistant":"OFF",
"legacy_animations":"ON"
,"esp8266_http_updateserver":"ON",
"arduino_ota":"ON",
"state_save":"SPIFFS"}

Since I'm using the 12v strip I changed the wiring a little bit, I don't know if that's whats causing the problem. Here's a diagram of how I wired everything up. It starts with a 12v power supply that powers the strip directly. There's a seperate line that gets stepped down to 5v for the esp8266. After that wiring is basically identical to what is in the McLighting hardware tutorial.

And here is my definitions.h file as I'm currently uploading it to the esp8266
#define USE_WS2812FX_DMA      // Uses PIN is ignored & set to RX/GPIO3  Uses WS2812FX, see: https://github.com/kitesurfer1404/WS2812FX
//#define USE_WS2812FX_UART1     // Uses PIN is ignored & set to D4/GPIO2  Uses WS2812FX, see: https://github.com/kitesurfer1404/WS2812FX
//#define USE_WS2812FX_UART2     // Uses PIN is ignored & set to TX/GPIO1  Uses WS2812FX, see: https://github.com/kitesurfer1404/WS2812FX

// Neopixel
#define PIN D2           // PIN (14 / D5) where neopixel / WS2811 strip is attached
#define NUMLEDS 100       // Number of leds in the strip
#define BUILTIN_LED 2    // ESP-12F has the built in LED on GPIO2, see https://github.com/esp8266/Arduino/issues/2192
#define BUTTON 4         // Input pin (4 / D2) for switching the LED strip on / off, connect this PIN to ground to trigger button.

const char HOSTNAME[] = "McLighting01";   // Friedly hostname

#define HTTP_OTA             // If defined, enable ESP8266HTTPUpdateServer OTA code.
#define ENABLE_OTA         // If defined, enable Arduino OTA code.
//#define ENABLE_AMQTT         // If defined, enable Async MQTT code, see: https://github.com/marvinroger/async-mqtt-client
//#define ENABLE_MQTT        // If defined, enable MQTT client code, see: https://github.com/toblum/McLighting/wiki/MQTT-API
//#define ENABLE_HOMEASSISTANT // If defined, enable Homeassistant integration, ENABLE_MQTT or ENABLE_AMQTT must be active
#define ENABLE_BUTTON        // If defined, enable button handling code, see: https://github.com/toblum/McLighting/wiki/Button-control
//#define MQTT_HOME_ASSISTANT_SUPPORT // If defined, use AMQTT and select Tools -> IwIP Variant -> Higher Bandwidth
#define ENABLE_LEGACY_ANIMATIONS // Dont disbale this for now
#define ENABLE_E131              // E1.31 implementation

#ifdef ENABLE_E131
  #define START_UNIVERSE 1                    // First DMX Universe to listen for
  #define END_UNIVERSE 2              // Total number of Universes to listen for, starting at UNIVERSE
#endif

//#define WIFIMGR_PORTAL_TIMEOUT 180
//#define WIFIMGR_SET_MANUAL_IP

#ifdef WIFIMGR_SET_MANUAL_IP
  uint8_t _ip[4] = {192,168,0,128};
  uint8_t _gw[4] = {192,168,0,1};
  uint8_t _sn[4] = {255,255,255,0};
#endif

#ifdef MQTT_HOME_ASSISTANT_SUPPORT
  #define MQTT_HOME_ASSISTANT_0_84_SUPPORT // Comment if using HA version < 0.84
#endif

#if defined(USE_WS2812FX_DMA) and defined(USE_WS2812FX_UART)
#error "Cant have both DMA and UART method."
#endif
#if defined(ENABLE_MQTT) and defined(ENABLE_AMQTT)
#error "Cant have both PubSubClient and AsyncMQTT enabled. Choose either one."
#endif
#if ( (defined(ENABLE_HOMEASSISTANT) and !defined(ENABLE_MQTT)) and (defined(ENABLE_HOMEASSISTANT) and !defined(ENABLE_AMQTT)) )
#error "To use HA, you have to either enable PubCubClient or AsyncMQTT"
#endif
#if ( !defined(ENABLE_HOMEASSISTANT) and defined(MQTT_HOME_ASSISTANT_SUPPORT) )
#error "To use HA support, you have to either enable Homeassistant component"
#endif

// parameters for automatically cycling favorite patterns
uint32_t autoParams[][4] = { // color, speed, mode, duration (seconds)
  {0xff0000, 200,  1,  5.0}, // blink red for 5 seconds
  {0x00ff00, 200,  3, 10.0}, // wipe green for 10 seconds
  {0x0000ff, 200, 11,  5.0}, // dual scan blue for 5 seconds
  {0x0000ff, 200, 42, 15.0}  // fireworks for 15 seconds
};

#if defined(ENABLE_MQTT) or defined(ENABLE_AMQTT)
  #ifdef ENABLE_MQTT
    #define MQTT_MAX_PACKET_SIZE 2048
    #define MQTT_MAX_RECONNECT_TRIES 4

    int mqtt_reconnect_retries = 0;
    char mqtt_intopic[strlen(HOSTNAME) + 4 + 5];      // Topic in will be: <HOSTNAME>/in
    char mqtt_outtopic[strlen(HOSTNAME) + 5 + 5];     // Topic out will be: <HOSTNAME>/out
    uint8_t qossub = 0; // PubSubClient can sub qos 0 or 1
  #endif

  #ifdef ENABLE_AMQTT
    String mqtt_intopic = String(HOSTNAME) + "/in";
    String mqtt_outtopic = String(HOSTNAME) + "/out";
    uint8_t qossub = 0; // AMQTT can sub qos 0 or 1 or 2
    uint8_t qospub = 0; // AMQTT can pub qos 0 or 1 or 2
  #endif

  #ifdef ENABLE_HOMEASSISTANT
    String mqtt_ha = "home/" + String(HOSTNAME) + "_ha/";
    String mqtt_ha_state_in = mqtt_ha + "state/in";
    String mqtt_ha_state_out = mqtt_ha + "state/out";

    const char* on_cmd = "ON";
    const char* off_cmd = "OFF";
    bool stateOn = false;
    bool animation_on = false;
    bool new_ha_mqtt_msg = false;
    uint16_t color_temp = 327; // min is 154 and max is 500
  #endif

  //#define ENABLE_MQTT_HOSTNAME_CHIPID          // Uncomment/comment to add ESPChipID to end of MQTT hostname
  #ifdef ENABLE_MQTT_HOSTNAME_CHIPID
    char mqtt_clientid[64];
  #else
    const char* mqtt_clientid = HOSTNAME;
  #endif

  char mqtt_host[64] = "";
  char mqtt_port[6] = "";
  char mqtt_user[32] = "";
  char mqtt_pass[32] = "";
#endif

// ***************************************************************************
// Global variables / definitions
// ***************************************************************************
#define DBG_OUTPUT_PORT Serial  // Set debug output port

// List of all color modes
#ifdef ENABLE_LEGACY_ANIMATIONS
  #ifdef ENABLE_E131
    enum MODE { SET_MODE, HOLD, OFF, SETCOLOR, SETSPEED, BRIGHTNESS, WIPE, RAINBOW, RAINBOWCYCLE, THEATERCHASE, TWINKLERANDOM, THEATERCHASERAINBOW, TV, CUSTOM, E131 };
  #else
    enum MODE { SET_MODE, HOLD, OFF, SETCOLOR, SETSPEED, BRIGHTNESS, WIPE, RAINBOW, RAINBOWCYCLE, THEATERCHASE, TWINKLERANDOM, THEATERCHASERAINBOW, TV, CUSTOM };
  #endif
  MODE mode = RAINBOW;         // Standard mode that is active when software starts
  bool exit_func = false;      // Global helper variable to get out of the color modes when mode changes
#else
  enum MODE { SET_MODE, HOLD, OFF, SETCOLOR, SETSPEED, BRIGHTNESS, CUSTOM };
  MODE mode = SET_MODE;        // Standard mode that is active when software starts
#endif

MODE prevmode = mode;

int ws2812fx_speed = 196;   // Global variable for storing the delay between color changes --> smaller == faster
int brightness = 196;       // Global variable for storing the brightness (255 == 100%)

int ws2812fx_mode = 0;      // Helper variable to set WS2812FX modes

bool shouldSaveConfig = false;  // For WiFiManger custom config

struct ledstate             // Data structure to store a state of a single led
{
  uint8_t red;
  uint8_t green;
  uint8_t blue;
};

typedef struct ledstate LEDState;     // Define the datatype LEDState
LEDState ledstates[NUMLEDS];          // Get an array of led states to store the state of the whole strip
LEDState main_color = { 255, 0, 0 };  // Store the "main color" of the strip used in single color modes

#define ENABLE_STATE_SAVE_SPIFFS        // If defined, saves state on SPIFFS
//#define ENABLE_STATE_SAVE_EEPROM        // If defined, save state on reboot
#ifdef ENABLE_STATE_SAVE_EEPROM
  char current_state[32];               // Keeps the current state representation
  char last_state[32];                  // Save the last state as string representation
  unsigned long time_statechange = 0;   // Time when the state last changed
  int timeout_statechange_save = 5000;  // Timeout in ms to wait before state is saved
  bool state_save_requested = false;    // State has to be saved after timeout
#endif
#ifdef ENABLE_STATE_SAVE_SPIFFS
  bool updateStateFS = false;
#endif

// Button handling
#ifdef ENABLE_BUTTON
  #define BTN_MODE_SHORT "STA| 1|  0|245|196|255|255|255"   // Static white
  #define BTN_MODE_MEDIUM "STA| 1| 48|245|196|255|102|  0"    // Fire flicker
  #define BTN_MODE_LONG "STA| 1| 46|253|196|255|102|  0"  // Fireworks random

  unsigned long keyPrevMillis = 0;
  const unsigned long keySampleIntervalMs = 25;
  byte longKeyPressCountMax = 80;       // 80 * 25 = 2000 ms
  byte mediumKeyPressCountMin = 20;     // 20 * 25 = 500 ms
  byte KeyPressCount = 0;
  byte prevKeyState = HIGH;             // button is active low
  boolean buttonState = false;
#endif

Since you have enabled #define USE_WS2812FX_DMA, PIN is ignored and you will have to use PIN RX/GPIO3 and not D2.
Since you are using 2811 strip, you will have to use NeoEsp8266Dma400KbpsMethod,
Replace https://github.com/toblum/McLighting/blob/c0329ef01e27cfb267fa41411868e4c441a22a02/Arduino/McLighting/McLighting.ino#L94 with
NeoEsp8266Dma400KbpsMethod* dma;

and replace https://github.com/toblum/McLighting/blob/c0329ef01e27cfb267fa41411868e4c441a22a02/Arduino/McLighting/McLighting.ino#L106
dma = new NeoEsp8266Dma400KbpsMethod(stripSize, 3);

I made it easier in Develop Branch by just uncommenting following line https://github.com/toblum/McLighting/blob/fa918443c4063d418e25698b695b18549fd4beb0/Arduino/McLighting/definitions.h#L8

Thank you so much for getting back to me so quickly, I really appreciate it! So I deleted my McLighting folder and did a new clone of the McLighting repository. Though it seems like my newly cloned definitions.h file didn't have the updates you made, so I manually copy-pasted the raw definitions.h from the git website.
Here is how I have the beginning of your new definitions.h set up (I moved my data pin to d5 and left ws2812fx commented):

//#define USE_WS2812FX_DMA       // LED_PIN is ignored & set to RX/GPIO3  Uses WS2812FX, see: https://github.com/kitesurfer1404/WS2812FX
//#define USE_WS2812FX_UART1     // LED_PIN is ignored & set to D4/GPIO2  Uses WS2812FX, see: https://github.com/kitesurfer1404/WS2812FX
//#define USE_WS2812FX_UART2     // LED_PIN is ignored & set to TX/GPIO1  Uses WS2812FX, see: https://github.com/kitesurfer1404/WS2812FX

// Neopixel
#define LED_PIN D5       // LED_PIN (14 / D5) where neopixel / WS2811 strip is attached
#define NUMLEDS 100       // Number of leds in the strip
#define LED_TYPE_WS2811  // Uncomment if LED type uses 400 KHz (classic 'v1' (not v2) FLORA pixels, WS2811 drivers)
#define BUILTIN_LED 2    // ESP-12F has the built in LED on GPIO2, see https://github.com/esp8266/Arduino/issues/2192
#define BUTTON 4         // Input pin (4 / D2) for switching the LED strip on / off, connect this PIN to ground to trigger button.

#define HOSTNAME "McLighting01"   // Friedly hostname

#define HTTP_OTA             // If defined, enable ESP8266HTTPUpdateServer OTA code.
#define ENABLE_OTA         // If defined, enable Arduino OTA code.
//#define ENABLE_AMQTT         // If defined, enable Async MQTT code, see: https://github.com/marvinroger/async-mqtt-client
//#define ENABLE_MQTT        // If defined, enable MQTT client code, see: https://github.com/toblum/McLighting/wiki/MQTT-API
//#define ENABLE_HOMEASSISTANT // If defined, enable Homeassistant integration, ENABLE_MQTT or ENABLE_AMQTT must be active
#define ENABLE_BUTTON        // If defined, enable button handling code, see: https://github.com/toblum/McLighting/wiki/Button-control
//#define MQTT_HOME_ASSISTANT_SUPPORT // If defined, use AMQTT and select Tools -> IwIP Variant -> Higher Bandwidth
#define ENABLE_LEGACY_ANIMATIONS // Dont disbale this for now
#define ENABLE_E131              // E1.31 implementation
#define USE_HTML_MIN_GZ             //comment for using index.htm & edit.htm from SPIFFs instead of PROGMEM
//#define CUSTOM_WS2812FX_ANIMATIONS  //uncomment and put animations in "custom_ws2812fx_animations.h"

WHen I try to build I get this error:
Compiling .pioenvs\nodemcuv2\src\McLighting.ino.cpp.o
In file included from C:/Users/Souren/Desktop/McLighting/Arduino/McLighting/McLighting.ino:459:0:
Arduino\McLighting\request_handlers.h: In function 'bool readStateFS()':
Arduino\McLighting\request_handlers.h:1415:25: error: 'stateOn' was not declared in this scope
if(mode != OFF) stateOn = true;
^
*** [.pioenvs\nodemcuv2\src\McLighting.ino.cpp.o] Error 1

Please use this ZIP File

Ok I delted everything and loaded the zip you provided. Building it returns this error:
hting\request_handlers.h: In function 'bool readStateFS()':
Arduino\McLighting\request_handlers.h:1415:25: error: 'stateOn' was not declared in this scope
if(mode != OFF) stateOn = true;
^
Compiling .pioenvs\nodemcuv2\libe96\AsyncMqttClient_ID346\AsyncMqttClient\Packets\UnsubAckPacket.cpp.o
Archiving .pioenvs\nodemcuv2\lib833\libArduinoJson.a
Compiling .pioenvs\nodemcuv2\lib88f\Adafruit NeoPixel_ID28\Adafruit_NeoPixel.cpp.o
Compiling .pioenvs\nodemcuv2\lib88f\Adafruit NeoPixel_ID28\esp8266.c.o
*** [.pioenvs\nodemcuv2\src\McLighting.ino.cpp.o] Error 1
Compiling .pioenvs\nodemcuv2\lib919\WS2812FX_ID1647\WS2812FX.cpp.o

Thank you, it looks like I overlooked the fact that stateOn is defined for Home Assistant. For now, could you uncomment //#define ENABLE_HOMEASSISTANT. Even if you don't use HA, McLighting will function.

Please update to master version v2.2.2, should be fixed. Please feel free to reopen if you still facing issues.


Add SendWSEvent

What are some examples of WS events you would like to be able to send?

This PR definitely requires some more explanation, including what documented gateway payloads the library does not currently support. That said, given that the PR is opened against master rather than develop, it's unmergeable in any state, so I'm going to close it. If you'd still like to see this functionality, open a new PR against develop, and please provide some more description of why this would be useful for the library.


Netty/Handler requires bc-fips in OSGi

Expected behavior
The Netty Handler bundle should resolve using the bcprov bundle and not require the bc-fips bundle from BouncyCastle.
Actual behavior
The Netty Handler bundle has a, non optional, dependency on the "org.bouncycastle.jcajce.provider" package which is provided by the bc-fips bundle from BouncyCastle.

Unable to resolve io.netty.handler/4.1.110.Final: missing requirement [io.netty.handler/4.1.110.Final] osgi.wiring.package; filter:="(osgi.wiring.package=org.bouncycastle.jcajce.provider)"

Steps to reproduce
netty-handler 4.1.109.Final has this dependency on bcprov:

org.bouncycastle.jce.provider;version="[1.69,2)";resolution:=optional

netty-handler 4.1.110.Final adds this mandatory dependency on bc-fips:

org.bouncycastle.jcajce.provider

This is most likely due to this commit: https://github.com/netty/netty/commit/d868c7b5926c0c00a2367543ba91901072cce636
It aims to make it possible to use either bcprov or bc-fips as the provider but ends up making bc-fips mandatory in OSGi.
Minimal yet complete reproducer code (or URL to code)
Not applicable
Netty version
Netty/Handler 4.1.110.Final
JVM version (e.g. java -version)
Not applicable
OS version (e.g. uname -a)
Not applicable

Is this something you could investigate why it does that? I don't see how OSGi make its decision in either case.

OSGi reads the "Import-Package" instructions in the "MANIFEST.MF" file in the jar file to resolve dependencies.
These instructions are generated by the "maven-bundle-plugin" by looking at the imports in the source files.
It then uses the dependencies in the pom file to determine versions and optionality.
The "bcprov-jdk15on" dependency has the version "1.69" and is marked as optional.
(It is not actually added to the pom file, but should probably be. The plugin resolves it anyway it seems.)
This results in an entry like this in the "MANIFEST.MF" file: "org.bouncycastle.jce.provider;version="[1.69,2)";resolution:=optional"
The "bc-fips" dependency is not added to the pom file and therefore the plugin can't determine the version and if the dependency should be optional.
The default is that a dependency is mandatory so this is the entry for it: "org.bouncycastle.jcajce.provider"
To solve this issue you can either add "bc-fips" as an optional dependency or give plugin configuration to tell it that it should be optional and which version to use.


Feature request: Add support for UTF8 bytestring wrappers (utf8-bytestring-basic, utf8-bytestring-posn...)

Alex wrappers have UTF8 encoding logic for wrappers that operate on haskell strings. I think it shouldn't be too difficuilt to add UTF8 bytestrings wrappers (for Data.ByteString.Lazy.UTF8.ByteString type) that will encode UTF8 chars in a similar fashion

The type is provided in utf8-bytestring package
https://hackage.haskell.org/package/utf8-string-1.0.2/docs/Data-ByteString-Lazy-UTF8.html


twitter stream example instantiated correctly, but not showing any results

I am using spark-notebook-0.6.3-scala-2.11.7-spark-1.6.2-hadoop-2.7.3-with-hive-with-parquet
and I updated the twitter example dependencies to Scala 2.11 and spark 1.6.2:
https://github.com/andypetrella/spark-notebook/blob/master/notebooks/streaming/Twitter stream.snb
(it seems that spark-streaming-twitter is not yet available for spark 2)
:dp org.apache.spark %% spark-streaming % 1.6.2
org.apache.spark %% spark-streaming-twitter % 1.6.2
- org.apache.spark %% spark-core % _
- org.apache.hadoop % _ % _

I set my twitter account details, and I run the notebook. The twitterStream is instantiated correctly, and the notebook does not show any error, but the topCounts60 does not show the result(r), and no data is shown in the map.
Any idea?

check the browser console logs also...


Ambassador admin on minikube - invalid urls of services

Describe the bug
When setting up ambassador on minikube cluster minikube admin returns quite confusing and not working urls - IP:PORT is taken from admin service but shouldn't.
To Reproduce
Steps to reproduce the behavior:

Run some service with ambassador annotations
Install ambassador

helm install datawire/ambassador --name ambassador --namespace api -f values-minikube.yaml

values-minikube.yaml
service:
  type: NodePort

adminService:
  type: NodePort

. open admin panel in browser
 minikube service ambassador-admin --format "http://{{.IP}}:{{.Port}}/ambassador/v0/diag/" -n api

ambassador service are accesible on different ports in minikube cluster.
Expected behavior
Valid urls of configured service should be showed by admin but got ambassador-admin IP and PORT which is invalid on minikube.
Versions (please complete the following information):

Ambassador: 0.39.0
Kubernetes environment minikube

Maybe there is a way to configure in some different way?

ambassador admin doesn't need to be exposed, only ambassador should. closing (i was thinking that it's working from different side )


Add AssistantPromptTemplate override method createMessage()

I found that when using the project, the MessageType is still USER instead of the expected ASSISTANT when using AssistantPromptTemplate.
I'm not sure if this issue is an oversight, a bug, or intentional. Therefore, I submitted this PR. If it's not an oversight or bug, please close the PR.

Yikes, it is a bug.  Great catch, it will be part of 0.8.0.  Waiting for a long lived branch/PR to be merged in first.

Yikes, it is a bug. Great catch, it will be part of 0.8.0. Waiting for a long lived branch/PR to be merged in first.

I'm really thrilled because my pull request got merged into a well-known open-source project for the first time.

Now it is merged, time to party! :partying_face:
Merged in c1d30345698cad439a4789e3255e22cedc07c062


Gaia authentication payload must include client-side data

The authentication process for authenticating gaia hub should include client-generated content in the signed payload (and will therefore need a way to relay that data to the gaia hub as well). This would prevent a malicious gaia hub from requesting arbitrary challenge texts, because the client would be signing a payload with client-generated data, in addition to the challenge text.

This was addressed in #115, and is released in the latest version of the gaia hub.


[Q] Usage of values

Hi!
I've got repository with structure like this:
├── Chart.yaml
├── charts
├── templates
│   ├── my-template.yaml
[...]
│   └── _helpers.tpl
├── values
│   ├── values-prod.yaml
│   └── values-stage.yaml
└── values.yaml

I'd like to install chart from repository without actually fetching it:
helm install name my/repo --values=values/values-prod.yaml
Is it possible to overwrite default values with my values file without having  that values-prod.yaml locally?

If it's impossible, I've came with idea to set this line in each template:
{{ $env := (index .Values .Values.env) }}
and to use
{{ $env.myValue }} instead of {{ .Values.myValue }}
Values files will be also modified to have all values from different files like this:
env: dev
dev:
  foo:
    bar:
...
stage:
  foo:
    bar:
...
etc

this workaround will allow me to install with different values without having them locally using this command:  helm install [ or upgrade ] name  my/repo --set env=stage
Is there any easier way to achieve this?

This is an interesting question and one I'd not thought of before. You can't do it using the --values flag to tell it about a file in the chart. But, there is a method you can use.
You could create a template function that takes in an environment set as a value. Then based on that you could load the content of one of those files, parse the yaml into a dict (map[string]interface{} like values), and then return this. You could use that in your templates. You can parse the yaml with the fromYaml function. I'm pointing you to the source because I can't find docs on it.
Does this make sense?

I've created an function in helpers like this:
{{- define "values" -}}
{{- if (eq .Values.env "develop") -}}
    {{ toYaml .Values.develop }}
{{- end -}}
{{- end -}}

My call to this function, located in template looks like this:
{{ template "values" . }}
```
This returns all my variables located under "develop". When I want to recieve .Values.develop.foo, seems that it doesn't work as it should be. I've got an error `executing "values" at <eq .Values.env "develop">: error calling eq: incompatible types for comparison`. Below I provide how I call my variables while trying to filter:
```
{{ template "values" .mongodb }}
```

I've also tried to return from helpers map instead of yaml and using my call to variables like this:
```
{{ template "values" .mongodb | fromYaml }}
```

I was thinking you could access the other files from within the template. There are docs on that here. You can conditionally load the content of a develop or prod configuration, parse it to yaml, attach it to a new variable, and reference that. Does that make sense?

Isn't it easier to use it like I tried to? Creating new files and then using variables located there makes sense, but developer have to come through more steps... Isn't there a way to include only one variable from helpers like:  {{ template "values" .foo}}
I've also thought about using with  statement, but I don't really know how can I select file (with this statement) depending on what variable is set in values.

I can do it like this:
{{ template values/values-(index .Values .Values.env).yaml .

But... This way is inconvinient. It's easier to inculde another file with with statement and use $.Values where we really need something from global values file.

@Cyldei If that works for you please let us know.

Yeah, I've made this quick summary to show how I resolved my problem. Thank you for help! :)


Add support for Zorin OS

Currently using Zorin OS (Core), which shows up as Unknown in Discord.

done, run fetchcord --update


Node 14.x / ES6 Support

Hey @lilly-nathanamorin 👋
Can you assign this story to me and I will try and get around to it after current workload calms down:

CHANGELOG.md creation.
Move support to imports vs require.
ES6 Support Throughout.
Update README.md with a Contribution section.

Thanks mate!
Is there anything else you think needs doing?

These look great, thanks!


docs: added link to v2 client

Added a link to v2 client that clarify clients compatibility.

@Esity can you dismiss you review and we discuss this in the ruby-maintainer group?

@hennevogel dismissed


Frustrating parentheses matching behavior in notebooks

For example, if I type
"frustrating

and then try to close the quote by typing a " I get
 "frustrating""

and simply backspacing to delete the first closing quote results in deletion of both
"frustrating

so that to get from
"frustrating

to
 "frustrating"

I need to

type a quote
arrow right
backspace

when (1) alone should do the trick.
I get similar behavior with parentheses.
I'm not sure, but I think this just appeared in 4.0. It is not an issue in Python notebooks.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/26178412-frustrating-parentheses-matching-behavior-in-notebooks?utm_campaign=plugin&utm_content=tracker%2F332399&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F332399&utm_medium=issues&utm_source=github).

It should be fixed in newest version on Hackage. Quotes are not completed, brackets and parens should work.

I still have this issue in <IP_ADDRESS>.

Have you rerun ihaskell install?

Now I have. (Duh! Sorry!)

Did that fix it?

Yes!

Sorry for the false alarm!


Fix IRE processes

This attemps to improve processing of IRE processes.
@samwebster and @siddharth-krishna could you check whether it looks okay? Please feel free to make any changes. 😄

Looks good, thanks!


[Q/B] Command throws error from fluent, work fine in cli.

Code:
FfmpegCommand()
				        .addOption('-filter_complex', "'movie=intro.mp4 [v1]; amovie=intro.mp4 [a1];movie=video.mp4, setsar=1/1 [v2]; amovie=video.mp4 [a2]; [v1] [v2] concat [outv]; [a1] [a2] concat=v=0:a=1 [outa]'")
				        .addOption('-map',"'[outv]'")
				        .addOption('-map',"'[outa]'")
				        .addOption('-ss', '00:00:00')
				        .addOption('-t','00:00:40')
				        .addOption('-movflags +faststart')
				        .output('out.mp4')
				        .on('start', start).on('progress', progress).on('end', end).on('error', error).run();

Result
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x392ce80] Invalid stream specifier: '[outv]'.
    Last message repeated 3 times
Stream map ''[outv]'' matches no streams.

Cli run:
ffmpeg -filter_complex 'movie=intro.mp4 [v1]; amovie=intro.mp4 [a1];movie=video.mp4, setsar=1/1 [v2]; amovie=video.mp4 [a2]; [v1] [v2] concat [outv]; [a1] [a2] concat=v=0:a=1 [outa]' -map '[outv]' -map '[outa]' -ss 00:00:00 -t 00:00:40 -movflags +faststart out.mp4

Result:
a working out.mp4
Do i missing something?

Hi

Stream map ''[outv]'' matches no streams.

Notice the double-single-quotes here. You have too much quoting in your command arguments (you only need quoting when using a shell).
Remove the single quotes around the parameters for -filter_complex and -map and you should be good to go.

Closing this as no feedback was given. Feel free to reopen, though.


inconsisty between semantics in abstract gringo and implementation

At the moment, the following rule has a different semantics in the gringo implementation and abstract gringo:
:- { e(X,Y) } >= 1, n(X;Y).

According to AG both X and Y have to be global variables.

Working on this also uncovered a regression in clingo-5. The program
q(1..2).
{ p(X) : q(X) } :- 2 { q(X) }.

produces wrong results with clingo-5 because variable X wrongly became global.

Working on this also uncovered a second regression in clingo-5. The program
x :- #count { 1 : p(X+Y) } >= 0, X=1, Y=a.

produces wrong results with clingo-5 because X+Y is wrongly pulled into the rule body (clingo-4 guarantees that a term always has a value).

Fixed with d0283b8365d79f052c5eece4de47510e8a5c3855


Followup Support NULL literals in where clause

Which issue does this PR close?

Follow up #11266. This is to use type_coercion for null literary to avoid hard coding in physical planner.
Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Thanks -- sorry for the back and forth @xinlifoobar - I think we could make this simpler, But if you are sick of changing things, that is fine too and I can make a follow on PR

Ya, I am ok with this :) Sometimes I may make bad changes due to a lack of knowledge of the whole repo. You guidance is always helpful here.


Rework error collection, error if tslint.json is present

This is a noisy change; meant to just send the PR to error if tslint.json is present, but found myself changing a few other things to prevent dtslint from bailing early for simple problems like "npmignore is wrong" or "OTHER_FILES.txt or tslint.json is present".
We'll now not bail out early on errors, e.g.
putting some random garbage in a dts file in a types version dir:
<EMAIL_ADDRESS>testing from 4.6 to 4.8 in /home/jabaile/work/DefinitelyTyped/types/node/ts4.8
testing from 4.9 to 5.4 in /home/jabaile/work/DefinitelyTyped/types/node
Error:
/home/jabaile/work/DefinitelyTyped/types/node/ts4.8/cluster.d.ts
  433:1  error<EMAIL_ADDRESS>4.7, 4.8 compile error:
Statements are not allowed in ambient contexts  @definitelytyped/expect
  433:1  error<EMAIL_ADDRESS>4.7, 4.8 compile error:
Cannot find name 'asdagnsdogsdf'                @definitelytyped/expect

✖ 2 problems (2 errors, 0 warnings)

    at combineErrorsAndWarnings (/home/jabaile/work/DefinitelyTyped-tools/packages/dtslint/src/index.ts:250:26)
    at runTests (/home/jabaile/work/DefinitelyTyped-tools/packages/dtslint/src/index.ts:241:18)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at main (/home/jabaile/work/DefinitelyTyped-tools/packages/dtslint/src/index.ts:103:22)
 ELIFECYCLE  Test failed. See above for more details.

Then with the new error about having a tslint.json:
<EMAIL_ADDRESS>testing from 4.6 to 4.8 in /home/jabaile/work/DefinitelyTyped/types/node/ts4.8
testing from 4.9 to 5.4 in /home/jabaile/work/DefinitelyTyped/types/node
Error: /home/jabaile/work/DefinitelyTyped/types/node/ts4.8: Should not contain 'tslint.json'. This file is no longer required; place all lint-related options into .eslintrc.json.
    at combineErrorsAndWarnings (/home/jabaile/work/DefinitelyTyped-tools/packages/dtslint/src/index.ts:250:26)
    at runTests (/home/jabaile/work/DefinitelyTyped-tools/packages/dtslint/src/index.ts:241:18)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at main (/home/jabaile/work/DefinitelyTyped-tools/packages/dtslint/src/index.ts:103:22)

Perhaps not bailing out early is unwise; before, if there were errors, we would effectively skip ATTW. Maybe I should just make it not run ATTW if there are preceding errors?

I think running everything is fine. Some eslint errors are trivial enough that you’d want to know if your entire module structure is wrong before fixing the eslint stuff.


How does hyperencalve perform memory encryption on Intel CPU?

In the paper, "To thwart physical memory attacks, such as cold boot and bus snooping attacks, HyperEnclave may leverage hardware memory encryption (such as AMD SME [44] and Intel MKTME [42]) to encrypt partial physical memory at the page granularity"
In the current implementation, it seems that only AMD SME is used to conduct memory encryption. How does memory encryption work when running on an Intel CPU? we didn’t see the use of Intel MKTME in the source code.
Thanks.

Hi @bronzeMe , thanks for your interest for HyperEnclave!
Intel's MKTME is a memory encryption feature similar to AMD's SME. HyperEnclave functions in a similar manner under Intel's MKTME. The HyperEnclave equipped with MKTME is still in the process of being developed, so we have not released it now.

Hi @bronzeMe , thanks for your interest for HyperEnclave!
Intel's MKTME is a memory encryption feature similar to AMD's SME. HyperEnclave functions in a similar manner under Intel's MKTME. The HyperEnclave equipped with MKTME is still in the process of being developed, so we have not released it now.
Thank you , @Bonjourz
So, when HyperEnclave runs on Intel CPU, how does it encrypt/decrypt the memory transparently?

under Intel's MKTME, HyperEnclave configures the encryption by setting the upper bits of the physical address field in the PTE, which used by secure world software.
Refer to
https:://software.intel.com/content/dam/develop/external/us/en/documents-tps/multi-key-total-memory-encryption-spec.pdf
for more details.

@Bonjourz
Hi , "The HyperEnclave equipped with MKTME is still in the process of being developed, so we have not released it now.". A quick question following this: is there a plan to release this feature? If so, is there a timeline for its release??

Hi @bronzeMe , thanks for your interest for HyperEnclave's incoming new features. It seems that you might aspire to try this new feature. Could you share the details with us:

What is the your purpose of trying this new feature?
What is the details in your use case(HyperEnclave equipped Intel MKTME)?

Hi @bronzeMe  ,
This repository has been moved to a new location: https://github.com/asterinas/hyperenclave, and will be will be archived later.
If you still have such issue, please copy it there: https://github.com/asterinas/hyperenclave/issues, and have a discussion.


Adds binary

Adds practice exercise for binary

Ah! My bad! I didn't realize I had already implemented all-your-base. Closing this PR because we do not need both!


APEXMALHAR-2505 retain original field order in SnapshotSchema

@amberarrow please review

This change itself looks OK and can be merged; why is the Travis build failing though ?


hold-the-node Add support for clustered Redis

Redis 6 comes with sentinel, which allows Redis to run as a A/P service with automatic failover.
Update the redis controller to support deploying Redis as a standalone 1-pod Redis service, or a 3-pods A/P clustered service. Each redis pod hosts a container that runs the redis server, and another container that runs a sentinel for the quorated cluster management.

/retest-required

/test infra-operator-build-deploy-kuttl

recheck

FYI latest update depends on https://github.com/openstack-k8s-operators/lib-common/pull/444, which is what makes the lint job fail.

recheck


Add the Ability to Call Custom Action for ResponseSelector Intents

Description of Problem:

With the ResponseSelector implementation, Rasa will look for the corresponding utter_retrieval_intent/sub_intent name. This feature request would be to enable users to define a custom action action_retrieval_intent/sub_intent that can optionally be called if an utterance is not specified.
The functionality would be similar to Forms, where it looks for utter_ask_form_name_slot by default but users can optionally define a custom action action_ask_form_name_slot if they want.
Overview of the Solution:

Ability to define ResponseSelector custom action action_retrieval_intent/sub_intent if no utterance response is provided (utter_retrieval_intent/sub_intent)
Definition of Done:

[ ] Tests are added
[ ] Feature described the docs
[ ] Feature mentioned in the changelog

@b-quachtran What is the use case for this?

@TyDunn A customer wants to have the option to, for a given ResponseSelector intent, show multiple messages back to the user and be able to dynamically change the text response.

Thanks! @dakshvar22 is there a particular reason we do not support calling custom actions for ResponseSelector intents today?

@b-quachtran If I understand correctly, the user wants to run a custom action for a specific sub-intent and not the complete retrieval intent? You could still do this really. For example, if retrieval intent is faq you can have a rule like -
  - intent: faq
  - action: utter_faq
  - action: action_custom_faq

action_custom_faq can handle the logic of doing something custom for a specific sub-intent if needed.

@dakshvar22 For the above example, they'd leave the response blank for the relevant sub-intents and would write logic inside the action based on the sub-intent right? I think from their perspective, they'd prefer not to have to write any selection logic since it can get quite large for >50 sub-intents.

@dakshvar22 Please double check but from my impression this would be a change here along the lines of this

@wochinge Sorry, took forever to come back to this. Yes, what you proposed seems reasonable. Some thoughts:

We'll need to make sure that the process method of ResponseSelector does not break when there is no corresponding utter_* response template for a retrieval intent. Specifically here it tries to retrieve all the response templates for the selected sub-intent of the retrieval intent.

What should happen when use_text_as_label is set to True for response selector but some of the sub-intents of a retrieval intent do not have a response template defined (because a custom action needs to be triggered).


Investigate novel storage system test failures

Bad error response (non-JSON)
ERROR: test_copy_existing_file (storage.TestStorageWriteFiles)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/storage.py", line 181, in test_copy_existing_file
    new_blob = self.bucket.copy_blob(blob, self.bucket, 'CloudLogoCopy')
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/storage/bucket.py", line 460, in copy_blob
    method='POST', path=api_path, _target_object=new_blob)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/connection.py", line 337, in api_request
    error_info=method + ' ' + url)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/exceptions.py", line 185, in make_exception
    payload = json.loads(content)
  File "/opt/python/2.7.9/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/opt/python/2.7.9/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/python/2.7.9/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

From https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/84536058

Missing files
======================================================================
ERROR: tearDownModule (storage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/storage.py", line 47, in tearDownModule
    bucket.delete(force=True)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/storage/bucket.py", line 358, in delete
    _target_object=None)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/connection.py", line 419, in api_request
    error_info=method + ' ' + url)
gcloud.exceptions.Conflict: 409 The bucket you tried to delete was not empty. (DELETE https://www.googleapis.com/storage/v1/b/new1440625507341)
======================================================================
FAIL: test_second_level (storage.TestStoragePseudoHierarchy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/storage.py", line 290, in test_second_level
    'parent/child/file22.txt'])
AssertionError: Lists differ: [] != ['parent/child/file21.txt', 'parent/child/file22.txt']
Second list contains 2 additional elements.
First extra element 0:
parent/child/file21.txt
- []

From https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/77405118

Backend Error (500)
======================================================================
ERROR: test_list_buckets (storage.TestStorageBuckets)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/storage.py", line 58, in tearDown
    CLIENT.bucket(bucket_name).delete()
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/storage/batch.py", line 263, in __exit__
    self.finish()
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/storage/batch.py", line 249, in finish
    self._finish_futures(responses)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/storage/batch.py", line 231, in _finish_futures
    raise make_exception(*exception_args)
gcloud.exceptions.InternalServerError: 500 Backend Error
----------------------------------------------------------------------

From:

https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/77600321
https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/82969179
https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/97488225

Delete blob -> 404 (probably eventual consistency error)
======================================================================
ERROR: setUpClass (storage.TestStoragePseudoHierarchy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/regression/storage.py", line 241, in setUpClass
    blob.delete()
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/regression3/lib/python3.4/site-packages/gcloud/storage/blob.py", line 264, in delete
    return self.bucket.delete_blob(self.name)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/regression3/lib/python3.4/site-packages/gcloud/storage/bucket.py", line 361, in delete_blob
    self.connection.api_request(method='DELETE', path=blob_path)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/regression3/lib/python3.4/site-packages/gcloud/connection.py", line 293, in api_request
    raise make_exception(response, content)
gcloud.exceptions.NotFound: 404 Not Found
----------------------------------------------------------------------

From:

https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/58811668
https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/97491709

Eventual consistency error
======================================================================
FAIL: test_paginate_files (storage.TestStorageListFiles)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/storage.py", line 231, in test_paginate_files
    self.assertEqual(len(last_blobs), truncation_size)
AssertionError: 0 != 1

From:

https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/84165044
https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/92143108
https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/93053612
https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/97374670
https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/98356544

Service unavailable (503) error
ERROR: test_list_buckets (storage.TestStorageBuckets)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/storage.py", line 76, in test_list_buckets
    bucket = CLIENT.create_bucket(bucket_name)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/storage/client.py", line 215, in create_bucket
    bucket.create(client=self)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/storage/bucket.py", line 172, in create
    data={'name': self.name}, _target_object=self)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/connection.py", line 337, in api_request
    error_info=method + ' ' + url)
gcloud.exceptions.ServiceUnavailable: 503 Backend Error (POST https://www.googleapis.com/storage/v1/b?project=precise-truck-742)

From: https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/90171219

Conflict deleting bucket in module teardown:

======================================================================
ERROR: tearDownModule (storage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tseaver/projects/agendaless/Google/src/gcloud-python/system_tests/storage.py", line 57, in tearDownModule
    retry(Config.TEST_BUCKET.delete, force=True)()
TypeError: __call__() got an unexpected keyword argument 'force'

----------------------------------------------------------------------

Found during a local run of tox -e system-tests.

That's from the eventual consistency of listing objects. Using force=True

gets a list of all objects / blobs
deletes each object / blob
deletes the bucket

but if the list isn't truly comprehensive, the bucket deletion will fail with the 409 conflict you showed. Luckily just trying bucket.delete(force=True) again will eventually succeed.

and 7) addressed via 00d42d895aba892b96ea9f4d393742aaa8cd174a.

@tseaver Since these reports are stale, let's just close out and then address as new failures occur


About caching and reusing data in stores

Now that v0.2.2 provides a nicer API for loading asynchronous data, I want to place my API requests in actions. Yet I'm not sure how to reuse data already cached in the stores.
(I'm sorry if you have already discussed this thing before, maybe I've lost the discussion somewhere. Also, I miss a gitter chat :-)
Before, i had my stores managing this:
// store listening to `load` action
onLoad() {
   if (this.getData())
     // use cached data
     this.trigger(this.getData())
   else
     // async bunga bunga
}

I was tempted to do something like this:
// actions
myActions.load.listen(() => {
    if (myStore.getData())
       // use cached data
       myActions.load.completed(myStore.getData())
    else
      // start async operation ...
});

but this won't work, since actions is required from store as well, causing a circular dependency.
I solved putting the logic in the component:
// mycomponent.jsx

componentDidMount() {
   if (store.getData())
      // reuse cache
      this.setState({data: getData()})
   else
      // async action
      actions.loadData()
}

I prefer this approach because:
– a possible loading state is managed from the component and not the store
– my getData() function could receive an argument with the fields I need from my data (in case of the data is available but incomplete)
Does this sound right or have you found a better approach?

I've tried to put my reasonings in this gist. The case is related to Combine results React getInitialState and Reflux getInitialState https://github.com/spoike/refluxjs/issues/166 and may be a case for server-rendering https://github.com/spoike/refluxjs/issues/144? (I know @appsforartists has made progress but i'm still some steps behind :-) )

@gpbl I use sessionStorage on this small app - https://github.com/simonsmith/github-user-search
I decided to allow the actions to read from the cache and let the stores save to the cache. When calling the the this.completed method from the action I pass along whether it was from the cache or not, then the stores know whether to do something with the data.
Hope that helps

I'm quite curious about the preferred approach here as well.

I personally think, that only store should be concerned with data retrieval.
Check it's state, nope, check localStorage, nope then raise ajax request.
Than approach when you move data-related stuff to actions or scatter cache setters and getters through application parts smells bad to me.

In order to clean up the issue tracker I have to close down all questions. Please use proper communications channels for questions about reflux, such as:

StackOverflow with the refluxjs tag
#reflux channel on Reactiflux Slack group. Sign up here for an account.


[FEATURE] Generate source maps for bundles

Coverage decreased (-0.3%) to 94.475% when pulling c65b2b9bcebb94bc2263adc0f27fc9ea717a4c59 on feature-bundle-source-maps into fa628b5e4abd8bae57d1d27d92f30d1e99860d94 on next.


Add docs for "transaction deadline exceeded" error, again

(Still) fixes #4552.
Original discussion in #4667 .  This change fixes a technical error I introduced while copy-editing some text of that PR.
Summary of changes (is still):

Update 'Common Errors' page with:

the retry error message

a recommendation that the user add retries to their app

a description of the state that leads to such errors (kindly
contributed by Andrei Matei)

also spruced up the txn restart section intro with some links to
reference docs on how txns work in CockroachDB for the curious.

This change is

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/ed42e2f8003a8397e2832d41436d1071faf7603b/
Edited pages:

v19.1/common-errors.md

Trying again Andrei!  Apologies for the earlier mixup.  This PR should incorporate your text describing the error with a few of the edits of the less technical parts from @lhirata from #4667

LGTM thanks!


README updates

Coverage remained the same at 91.68% when pulling 7f6322b95a7d133009c0a97d2a3a8fad2496fae8 on readme_update into e6e23041cfe847992a8c6bd310434ca818241916 on master.


DataSet: some date data helpers required.

Some basic likes being able to add datemeta data, slice and extract date components and auto-categorization of date data should be enough for a start.

@nitsrek: Just thinking about the stuff you did for int/float/single conversion here.... We could allow to auto-convert (prepared) date data to single as well.

foundations added to the branch for #632. Going to close this.


1.9.0 hae4909

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

@abhijeetSaroha , for some reason.. it lost the changes from regro-cf-autotick-bot.
but it is not a problem .. we can have just your changes here, so we can fix the CI
thanks!


Introduce --offline and --api-server for admin & ceremony add support for API

Description

As discussed with @kairoaraujo I have also introduced the --offline and --api-server
admin level options in this pr as well.
In this pr I  added "--offline" and "--api-server" as  admin-level command options which will
be reused by all admin subcommands.
This allows for a single place of exception handling for those two
arguments and consistent behavior across commands.
This allowed the removal of "--bootstrap" in ceremony and "--api-server"
options for both "ceremony" and "sign".
The "--offline" option allows you to have a local setting.SERVER
configuration but still run the commands in a disconnected manner for
testing.
Additionally, with "--api-server" and the "--save" (option for "send")
or "output" (argument for ceremony) you can upload the payload to the
RSTUF API and save the result of the command locally.
Tests for all tests are added and documentation is updated.
Additionally, in this pr I implement some of the missing features from our new admin ceremony.
The features included here are:

Add an --api-server option for communication with existing RSTUF API deployment.
Check bootstrap status before pushing to the API.
Push resulting payload to API POST /api/v1/bootstrap
Replace --save option with --output argument.
Automatically save if --bootstrap is used even if --output file argument is not provided.
The saved location is a default one - ceremony_payload.json.
Wait for the task to finish and report back to the CLI user.
Add timeout option for RSTUF API requests.
Don't allow threshold < 2.
Add tests to:

make sure the online key is not any of the root keys.
make sure at least threshold number of keys must be described with their public keys.

Signed-off-by: Martin Vrachev<EMAIL_ADDRESS>
Related to #533.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality not to work as expected)

Additional requirements

[x] Tests have been added for the bug fix or new feature
[x] Docs have been added for the bug fix or new feature

Code of Conduct
By submitting this PR, you agree to follow our Code of Conduct.

[x] I agree to follow this project's Code of Conduct

As discussed with @kairoaraujo I have also introduced the --offline and --api-server
admin level options in this pr as well in my last two commits.

Can we try to keep diffs to a minimum? I understand that it sometimes just isn't feasible, but this PR definitely mixes some unrelated things.

Also, a consistent/accurate PR description would be helpful, e.g.:

Automatically save if --bootstrap is used even if --output file argument is not provided.
The saved location is a default one - ceremony_payload.json.

vs.

This allowed the removal of "--bootstrap" in ceremony and "--api-server"
options for both "ceremony" and "sign".

This is going in the right direction. Here are a few comments on args/opts:

Moving --api-server to the admin base command is an improvement.

I'm unsure about --offline. It is only used by 3 out of 5 admin commands, and it means different things for those:

For ceremony it means don't write to the API.
For sign and update, it means don't read from or write to the API.

--offline affects the behavior of opts/args for local file input/output, in an IMO non-intuitive way. If --offline is passed, sign and update require an input parameter, and the output parameter no longer determines, if a file is written or not.

Generally, the opts/args related to local files  don't feel consistent or intuitive to me.

ceremony has an optional positional OUTPUT path argument
sign (and likely update) has an optional positional SIGNING_JSON_INPUT_FILE path argument, and a --save (OUTPUT) option

I think  we should...

consistently use options for both input and output files
maybe scope --offline to the API write operation only, and call it --dry-run, then it would also feel save to not write to a local file per default.
e.g.:

rstuf [--api-server] admin ceremony [--out [PATH]] [--dry-run]

Perform ceremony and send result to API to trigger bootstrap.

* If `--out [PATH]` is passed, result is written to local PATH (in addition to
  being sent to API).

* If `--dry-run` is passed, result is not sent to API. You can pass `--out
  [PATH]` to store the result locally. `--api-server` will be ignored and may
  be omitted.

rstuf [--api-server] admin {sign, metadata update} [--in PATH] [--out [PATH]] [--dry-run]

Perform sign/metadata update for pending event and send result to API.

* If `--in PATH` is passed, input is not read from API but from local PATH.

* If `--out [PATH]` is passed, result is written to local PATH (in addition to
  being sent to API).

* If `--dry-run` is passed, result is not sent to API. You can pass `--out
  [PATH]` to store the result locally. `--api-server` will be ignored and may
  be omitted.

(I'm open to naming suggestions)

#607 will replace the first 4 commits of this pr.
Please, review it before this one as this one will be on top of it.

This pr is superseded by: https://github.com/repository-service-tuf/repository-service-tuf-cli/pull/607 and  https://github.com/repository-service-tuf/repository-service-tuf-cli/pull/612.


can't format code and intellisense  in VS Code 1.7.1(Mac)

VSCode Version: v1.7.1-02611b40b24c9df2726ad8b33f5ef5f67ac30b44-2016-11-03T13:44:00.572Z
OS Version: macOS Sierra v10.12.1

I use Shift + Alt + F to format code, It’s not working!  I also try to run VS Code without extensions, execute:code --disable-extensions but It still can’t working. when I pressing shift+option + f key in my Mac will typing special character Ï.
like this:

Facing similar issue. I'm unable to run the command format code

i had same issue on mac.
VSCode: 1.7.1
macOSX El Capitan 10.11.6

@charliethinker This is not enough information for me to reproduce

What information would you need?
My versions are:
VSCode: 1.7.1 (1.7.1)
macOS Sierra: 10.12.1

I need to know in what language this happens and what plugins you are using. Also, I need to know if this only happens when you try this with the shortcut or if this also happens when selecting format from the context menu

In my case when i editing some c# files using older versions vscode it was working perfectly  (i don't know which one is).
After the update right click menu, intellisense, document format and shortcuts not worked. But i solved my problem.
With the lastest update all csharp files HAVE to under at least one solution (.sln) or project.json structure, otherwise it's not gonna work correctly.
So, im happy :)

@jrieken I'm in China, In Mac can only use Shift + Alt + F shortcut  to format code,  There is no way to use the context menu in Mac, I also try to run VS Code without extensions, execute: ·code --disable-extensions. but It still can’t working.
I use extensions list here:

I have to reinstall VS Code, It can work well. Today I found after I written markdown file in vs(I installed vscode-instant-markdown extension), format code and intellisense doesn't work again.

@ferhatayhan

In my case when i editing some c# files using older versions vscode it was working perfectly

I'm the same as you.

@charliethinker if you want to working with c#  read that topic. https://code.visualstudio.com/docs/languages/csharp#_roslyn-and-omnisharp

@jrieken when I use cover install VS Code, format and intellisense can't working. but I use ccleaner to uninstall VS Code and install it again ,format and intellisense was working well. Then I opened two window of VS Code(Shift+Cmd+N), one side used to write C#, while used to write the markdown file, after a period of time, format and intellisense doesn't work again.

In my case it is happening for Javascript.
I never use the keyboard shortcut, I allways open the command palette and select format code. The option does not exist anymore. Here are my extensions:

This is what happens when I try to format it:

re https://github.com/Microsoft/vscode/issues/15682#issuecomment-262238663 we renamed that for format code, it does the same but has a different label. When you want to format a selection only, you must have a selection first. @danielo515 this issue is about a C# setup problem as it seems, please don't mixin JavaScript as these are different worlds

@charliethinker re https://github.com/Microsoft/vscode/issues/15682#issuecomment-262142905 Do I understand correctly that this happens with C# files?

@jrieken re #15682 (comment) Yes.
This issue happened again today, after I open new window to writing my blog with markdown.
Then my context menu has changed:
before:

after:

I am having this same issue on Mac.
VSCode Version 1.8.0 (1.8.0)
MacOSX Version 10.11.6 (El Capitan)
When I try to format code via the shift+alt+F shortcut this character prints out instead of formatting the code: Ï. Also when I open the context menu there is no option for "Format Document". Furthermore if I open the command palette there is no option for "Format Document" as well. I am trying to do this on Python files, JavaScript files, SCSS files, and HTML files (all of which do not work).
Plugins Installed:

The "Format Document" option missing from the command palette:

The "Format Document" option missing from the context menu:

@danielo515 this issue is about a C# setup problem as it seems, please don't mixin JavaScript as these are different worlds

I can't see any evidence of that on the title neither on the description if the issue, so please forgive my confusion.

It seems like this kind of situation happened after some extensions updated, I disable all my extensions, and format (shift + alt + f) works again.

ok. closing this as being caused by an extension that.

I had same issue when I was opening a single javascript file in vs code On Mac, But when I tried "Open Folder" in explorer for the same file "Format Document" start working.

Hello i have same problem when (shift + alt + f) i have '.' and no formatting.
Nobody have solution
Thanks

Hi there,  got a problem too, maybe it's releated:
on Mac OS and since VSCode 1.16 i cannot use ALT + L (@) inside any File "no keyboard combination"

1.17 clean, no plugins installed. shift+alt+f produces Ï. Format command is not available at all (it does not appear under shift+cmd+p ">format" )

Any updates on this? I don't want to get back to Atom because of these kinds of issues...

1.17 clean, no plugins installed. shift+alt+f produces Ï. Format command is not available at all (it does not appear under shift+cmd+p ">format" )

Out of the box, without extensions, VS Code supports formatting only for a small set of languages, like JavaScript and TypeScript. That is only because we ship those extensions with VS Code. The rule of thumb is that a formatter-extension must be installed.
If a formatter exists, for lets say JavaScript, the editor context menu will include a "Format Document" command, also the Alt/Option+Shift+F sequence will be bound. If for a language, lets say Rust, no formatter is installed that doesn't happen, so no entry in the context menu, not binding the aforementioned sequence.
We do acknowledge the Alt/Option+Shift+F-confusion because on macOS it inserts by default Ï. We have recently pushed a commit that binds that sequence even when not having a formatter such that it can show a message asking you to install one.
We encourage everyone to browse the existing formatter extensions on our market-place because we cannot ship with formatters for every language.

I have renamed and locked this issue because we now show a message when no formatter is installed or available and because we want separate issues for separate things. Don't be afraid to file duplicate.


[Bug]: onSubmit from RVF breaks viewTransitions

Which packages are impacted?

[ ] @rvf/react
[X] @rvf/react-router
[ ] @rvf/zod
[ ] @rvf/yup
[ ] zod-form-data

What version of these packages are you using?
"@rvf/core": "^7.0.1",
    "@rvf/react-router": "^7.0.1",
    "@rvf/zod": "^7.0.1",

Please provide a link to a minimal reproduction of the issue.
https://stackblitz.com/edit/github-uwgzmhjv
Steps to Reproduce the Bug or Issue

The sandbox has two different versions that do the same thing. RVF w/ onSubmit added to the Form; and one without onSubmit added to the form.
Click one of the links on the main index page to see.  Clicking SUBMIT on the RVF w/ onSubmit won't do the view transition. The nav links will work on the page.  RVF w/o onSubmit works as expected performing the basic crossfrade view transition.

Expected behavior
I would expect viewTransitions to work when using RVF and it's submit handler.
Screenshots or Videos
No response
Platform
MacOS, Chrome
Additional context
No response

Good callout. I didn't even realize this was a thing now lol. Looks like this should be pretty straightforward to fix. Would you have time to submit a PR for it?
I think it's just a matter of updating the type here and pass through the viewTransition param here.

Sure thing! Let me add that.

Your PR does appear to have fixed the issue. We'll need to document this, but in order for it to work you need to pass viewTransition: true to the useForm hook rather than the Form component. Since under the hood we use useSubmit rather than the Form component from react-router. It's currently published as 7.0.2-beta.0 until we have documentation (which I could add tomorrow).

Thanks for checking that out. I was building my fork and copying the dist over my @rvf/react-router dist directory. I have no idea why I didn't see my changes work in my scrappy test, but the beta package does in fact work. Thanks again!


Change the log level to prevent false-positives

The WARN log level is being abused here which can flood the production logs (usually you are not interested in any lower log levels there). It is totally fine to not define a separate configMap for each profile that is active in the application therefore it shouldn't be a warning. The same applies to any other form of externalized configuration.

@wind57 @ryanjbaxter Good morning guys! As requested, I've opened another PR targeting 2.0.x branch.


Frontend : HTML LINKS AND IMAGES IN DOCUMENT FORM

Description 📜
I want to work on this issue and i submit my work in the document form. I explain everything in great with point to point explanation and also the code snippets that make my work more understandable.

Domain of Contribution 📊

[x] Frontend Dev (HTML, CSS, JS)

/assign

/assign

/assign