Theano error on trying to adapt the visualization example

I've minimally modified the conv_filter_visualization.py example to run on a network I've trained myself ( see https://gist.github.com/kgrm/67555890a3e07cab709a7a81cc487c31 ). The original script (with the provided weights file) works fine, However, on trying to run my modified one, I get the following theano error:
$ python alexnet_visualization.py
Using Theano backend.
Using gpu device 0: GeForce GTX TITAN X (CNMeM is enabled with initial size: 95.0% of memory, cuDNN 5004)
Weights loaded.
Model loaded.
convolution2d_1
maxpooling2d_1
batchnormalization_1
zeropadding2d_1
convolution2d_2
maxpooling2d_2
batchnormalization_2
zeropadding2d_2
convolution2d_3
zeropadding2d_3
convolution2d_4
zeropadding2d_4
convolution2d_5
maxpooling2d_3
flatten_1
dense_1
dropout_1
dense_2
dropout_2
dense_3
Processing filter 0
Traceback (most recent call last):
  File "alexnet_visualization.py", line 116, in <module>
    iterate = K.function([input_img], [loss, grads])
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 509, in function
    return Function(inputs, outputs, updates=updates, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 495, in __init__
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function.py", line 322, in function
    output_keys=output_keys)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/pfunc.py", line 480, in pfunc
    output_keys=output_keys)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 1817, in orig_function
    output_keys=output_keys).create(
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 1469, in __init__
    accept_inplace)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 177, in std_fgraph
    update_mapping=update_mapping)
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/fg.py", line 182, in __init__
self.__import_r__(output, reason="init")
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/fg.py", line 371, in __import_r__
    self.__import__(variable.owner, reason=reason)
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/fg.py", line 413, in __import__
    variable=r)
theano.gof.fg.MissingInputError: An input of the graph, used to compute DimShuffle{x,x,x,x}(keras_learning_phase), was not provided and not given a value.Use the Theano flag exception_verbosity='high',for more information on this error.

Backtrace when the variable is created:
  File "alexnet_visualization.py", line 7, in <module>
    from keras.models import Sequential
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 5, in <module>
    from . import backend as K
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/__init__.py", line 51, in <module>
    from .theano_backend import *
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 13, in <module>
    _LEARNING_PHASE = T.scalar(dtype='uint8', name='keras_learning_phase')  # 0 = test, 1 = train

Please make sure that the boxes below are checked before you submit your issue. Thank you!

[✓] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
[✓] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps
[✓] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

Same error for me!

Your model apparently has a different behavior in training and test mode, and so needs to know what mode it should be using.
Use
iterate = K.function([input_img, K.learning_phase()], [loss, grads])
and pass 1 or 0 as value for the learning phase, based on whether you want the model in training mode or test mode.

I also have similar problem, but occurs when I run use fit() or train_on_batch(), I used K.in_train_phase(pos_score, neg_score) in a user defined layer, pos_score and neg_score are both (symbolic) computed form layer input x. Even if I use K.in_train_phase(pos_score, neg_score), it shows the same error:

MissingInputError                         Traceback (most recent call last)
<ipython-input-33-09ce76894494> in <module>()
----> 1 model.predict_on_batch([src, dst])

/home/chentingpc/anaconda/lib/python2.7/site-packages/Keras-1.0.1-py2.7.egg/keras/engine/training.pyc in predict_on_batch(self, x)
   1205         else:
   1206             ins = x
-> 1207         self._make_predict_function()
   1208         outputs = self.predict_function(ins)
   1209         if len(outputs) == 1:

/home/chentingpc/anaconda/lib/python2.7/site-packages/Keras-1.0.1-py2.7.egg/keras/engine/training.pyc in _make_predict_function(self)
    687                                                self.outputs,
    688                                                updates=self.state_updates,
--> 689                                                **self._function_kwargs)
    690
    691     def _fit_loop(self, f, ins, out_labels=[], batch_size=32,

/home/chentingpc/anaconda/lib/python2.7/site-packages/Keras-1.0.1-py2.7.egg/keras/backend/theano_backend.pyc in function(inputs, outputs, updates, **kwargs)
    507                 msg = "Invalid argument '%s' passed to K.function" % key
    508                 raise ValueError(msg)
--> 509     return Function(inputs, outputs, updates=updates, **kwargs)
    510
    511

/home/chentingpc/anaconda/lib/python2.7/site-packages/Keras-1.0.1-py2.7.egg/keras/backend/theano_backend.pyc in __init__(self, inputs, outputs, updates, **kwargs)
    493                                         allow_input_downcast=True,
    494                                         on_unused_input='warn',
--> 495                                         **kwargs)
    496
    497     def __call__(self, inputs):

/home/chentingpc/anaconda/lib/python2.7/site-packages/theano/compile/function.pyc in function(inputs, outputs, mode, updates, givens, no_default_updates, accept_inplace, name, rebuild_strict, allow_input_downcast, profile, on_unused_input)
    318                    on_unused_input=on_unused_input,
    319                    profile=profile,
--> 320                    output_keys=output_keys)
    321     # We need to add the flag check_aliased inputs if we have any mutable or
    322     # borrowed used defined inputs

/home/chentingpc/anaconda/lib/python2.7/site-packages/theano/compile/pfunc.pyc in pfunc(params, outputs, mode, updates, givens, no_default_updates, accept_inplace, name, rebuild_strict, allow_input_downcast, profile, on_unused_input, output_keys)
    477                          accept_inplace=accept_inplace, name=name,
    478                          profile=profile, on_unused_input=on_unused_input,
--> 479                          output_keys=output_keys)
    480
    481

/home/chentingpc/anaconda/lib/python2.7/site-packages/theano/compile/function_module.pyc in orig_function(inputs, outputs, mode, accept_inplace, name, profile, on_unused_input, output_keys)
   1774                    profile=profile,
   1775                    on_unused_input=on_unused_input,
-> 1776                    output_keys=output_keys).create(
   1777             defaults)
   1778

/home/chentingpc/anaconda/lib/python2.7/site-packages/theano/compile/function_module.pyc in __init__(self, inputs, outputs, mode, accept_inplace, function_builder, profile, on_unused_input, fgraph, output_keys)
   1426             # OUTPUT VARIABLES)
   1427             fgraph, additional_outputs = std_fgraph(inputs, outputs,
-> 1428                                                     accept_inplace)
   1429             fgraph.profile = profile
   1430         else:

/home/chentingpc/anaconda/lib/python2.7/site-packages/theano/compile/function_module.pyc in std_fgraph(input_specs, output_specs, accept_inplace)
    175
    176     fgraph = gof.fg.FunctionGraph(orig_inputs, orig_outputs,
--> 177                                   update_mapping=update_mapping)
    178
    179     for node in fgraph.apply_nodes:

/home/chentingpc/anaconda/lib/python2.7/site-packages/theano/gof/fg.pyc in __init__(self, inputs, outputs, features, clone, update_mapping)
    169
    170         for output in outputs:
--> 171             self.__import_r__(output, reason="init")
    172         for i, output in enumerate(outputs):
    173             output.clients.append(('output', i))

/home/chentingpc/anaconda/lib/python2.7/site-packages/theano/gof/fg.pyc in __import_r__(self, variable, reason)
    358         # Imports the owners of the variables
    359         if variable.owner and variable.owner not in self.apply_nodes:
--> 360                 self.__import__(variable.owner, reason=reason)
    361         if (variable.owner is None and
    362                 not isinstance(variable, graph.Constant) and

/home/chentingpc/anaconda/lib/python2.7/site-packages/theano/gof/fg.pyc in __import__(self, apply_node, check, reason)
    472                             "for more information on this error."
    473                             % str(node)),
--> 474                             r)
    475
    476         for node in new_nodes:

MissingInputError: ("An input of the graph, used to compute DimShuffle{x}(keras_learning_phase), was not provided and not given a value.Use the Theano flag exception_verbosity='high',for more information on this error.", keras_learning_phase)

Good thing that the post right about yours explains what you need to do.

I am not sure if it is the same problem. As K.learning_phase() used in Embedding layer works just just fine with no additional input. Why does it need to be added when using user defined layer/function?

For a user defined layer that uses K.in_train_phase, it has to set self.uses_learning_phase = True, so train_on_batch/predict_on_batch and so on can set it correctly. I was following this https://github.com/fchollet/keras/wiki/Porting-your-custom-layers-from-Keras-0.3-to-Keras-1.0, and didn't notice this. It could have been better if in the guide/doc, setting self.uses_learning_phase = True was explicated mentioned.

I get this same error when I use GaussianNoise in my models.

Hi,guys! @kgrm @LouisMartin @chentingpc @fchollet @bmabey
Would you help me to solve this problem when I was getting the feature of Flatten layer.https://github.com/fchollet/keras/issues/431  Any help will be much preciate!

Thanks a lot.
when I use this blog code How convolutional neural networks see the world visual my own model's layer: conv_1, conv_2, conv_3 ...., I changed the code:

iterate = K.function([input_img], [loss, grads])
loss_value, grads_value = iterate([input_img_data])

to

iterate = K.function([input_img, K.learning_phase()], [loss, grads])
loss_value, grads_value = iterate([input_img_data, 1])

it worked fine.
Just as the FAQ: Regularization mechanisms, such as Dropout and L1/L2 weight regularization, are turned off at testing time.

Using K.function([input_img, K.learning_phase()], [loss]) gives me an error

TypeError: Unknown parameter type: <type 'int'>

How can I solve this?

I am getting all zeros for pooled_grads_value for some images.  So I followed @fchollet suggestion of adding "K.learning_phase()" and set scalar value of zero for it.  But still The entire (512,) array of pooled_grads_value is zero for some of sample images.
last_conv_layer = model.get_layer('conv2d_13')
grads = K.gradients(sample_output, last_conv_layer.output)[0]
normalization trick: we normalize the gradient
#grads = normalize_grad(grads)
pooled_grads = K.mean(grads, axis=(0, 2, 3))
iterate = K.function([model.input, K.learning_phase()], [pooled_grads, last_conv_layer.output[0]])
pooled_grads_value, conv_layer_output_value = iterate([x, 0])
I do appreciate if you can help me to resolve the issue.


New single host, multi-node tutorial

On a single host:

Start a local node
Start a 2nd node that talks to the 1st (master-slave)
Use the BIOS contracts to start 2 producers
Avoid having instructions that involve changing the config files, use the API

To replace the tutorial on the Wiki

PR submitted to the private tutorial repo. The four steps enumerated above are featured, along with the intermediate steps necessary to get from one to the next.

@rbw3821 please review this and integrate it with the tutorials in the Wiki

@rbw3821  are you reviewing my PR? I don't mention config file edits.

Yep, after reviewing I see that there are no references to config file edits.  Just doing some minor cleanup and will merge in a few.

Rick, there is a fair amount of overlap between this and the contract tutorial. Perhaps you could factor out the common part into a separate "getting started" tutorial that is referenced by both?

I've tested and it has been tested by QA.  Will merge to eos.wiki on Thursday.


Update change-credit-card.md

Deprecating Accounts portal redirecting links to Azure portal. @amberbhargava

#sign-off


Append Buttons to Answer

Describe your feature request
It would be nice to have the buttons auto attached to the answers after conversion. This would ease end user experience, especially on mobile where they do not have access to context menus at this time.

v0.1 removed this functionality.


Fix tests for pytest 7.0

See pytest-dev/pytest#9415 and pytest-dev/pytest#9061.

Thanks for the fix!


Remove public key from trunk model permissions

Companion PR: https://github.com/SubstraFoundation/substra-backend/pull/188
This allows for permissions without a "public" key (needed for trunk model permissions).

The change occured in a method that doesn't exists anymore, closing the PR.


Add test coverage to SecurityService.ts

After #58 (#60) is completed, add test coverage to the Security Service class. This is now tested because all the converters call it, but not all paths have been covered. Add a test file for this.

Doing this as part of #76


Update "account settings" link

Description of the change: Change "account settings" link on sender identity page
Reason for the change: Current link takes you to a broken page "We couldn't find the page you requested."
Link to original source: https://sendgrid.com/docs/ui/sending-email/senders/

Hello @cemurray,Thanks again for the PR! It's HACKTOBERFEST! We want to show our appreciation by sending                                              you some special Hacktoberfest swag. If you have not already, could you please fill out                                              this form so we can send it to you? Thanks!                                             Team SendGrid DX


feat: the monoidal structure on the category of presheaves of modules

This contribution was created as part of the AIM workshop "Formalizing algebraic geometry" in June 2024.
Co-authored-by: Dagur Asgeirsson<EMAIL_ADDRESS>Co-authored-by: Jack McKoen<EMAIL_ADDRESS>
Following the refactor #16667, this is a cleaner attempt as compared to #14127.

I hope this is ok even though I contributed to the initial version of this:
maintainer merge

Thanks!
bors merge


Plugin channels do not want to resolve

Using the following results in failing to resolve Scala plugin
     plugins = ["PythonCore:2<IP_ADDRESS>84.101", "org.intellij.scala:2018.2.804@nightly"]
   > Could not resolve com.jetbrains.plugins:org.intellij.scala:nightly.
     Required by:
         project :jetbrains-core
      > Could not resolve com.jetbrains.plugins:org.intellij.scala:nightly.
         > Could not get resource 'file:/Users/.../.gradle/caches/modules-2/files-2.1/com.jetbrains.intellij.idea/ideaIC/2018.2/e935ec9f9231a0789831f04f48053824c57b4bf5/ideaIC-2018.2-sources.jar/org.intellij.scala-nightly-2018.2.804.jar'.
            > Could not get file details of /Users/.../.gradle/caches/modules-2/files-2.1/com.jetbrains.intellij.idea/ideaIC/2018.2/e935ec9f9231a0789831f04f48053824c57b4bf5/ideaIC-2018.2-sources.jar/org.intellij.scala-nightly-2018.2.804.jar: could not stat file (errno 20: Not a directory)

Tried using version 0.3.10, works in 0.3.7

Ouch, sorry about this, will publish 0.3.11 asap

Custom channels still don't work in 0.3.11, for example with the plugin "org.jetbrains.kotlin:1.3.0-rc-116-IJ2018.3-1@EAP-NEXT". This plugin can be resolved in 0.3.7.


Set spellcheck=false attribute on silence filter .

This hints to iOS Safari and other browsers that they shouldn't use spellcheck, but, more importantly, shouldn't use smart quotes (like “this” instead of "this"). This matters here because Alertmanager requires quotes for a filter but won't accept smart quotes, making it tedious since the default input keyboard will normally use them automatically.

I know I'm supposed to include an updated asset/*_vfsdata.go in the PR, but I can't seem to get all the Elm dev stuff working on my machine, so would be happy for someone else to take over this PR and complete it. I've tested it manually.


refactor: authentication functionality

:tada: This PR is included in version 0.19.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


New chat welcome view

Replaces the old welcome view with one that is tailored to chat, the new layout, and is dismissible.

Collapsed view:

Test plan

CI

Loves the new welcome view @toolmantim !


Why don't accept multiple types for ListTag ?

This is not really an issue but mostly a question. I don't understand why there is this because multiple list can have multiple types.
Also, if we use setValue(), it works. So: why ? Why throw error only with default value but not when replacing it ?

A list tag in Minecraft only supports one type. You can’t have a string and byte tag for instance in the same nbt list.
In the future, please join our Discord server if you have any questions about the protocol / NBT. The issues section is more for reporting issues


How to run the Code on Windows + Anaconda ?

When trying to use the code example it won't work.
Probably it works on Linux, but Anaconda can't understand the current extra lines or something like that, I'm not sure how to make it work on Anaconda as I'm a newbie.
If anyone can explain step-by-step how to run the code in Anaconda on Windows it will be very helpful, until the official project will also show / explain how it's done on the main page.
Thanks ahead!

Same issue here tried my best to make it work with anaconda but the inference doesn't work for me

Windows support has been added. See here: https://github.com/google-research/frame-interpolation#change-log


gobgp: 2.29.0 -> 2.30.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/osrg/gobgp/releases.
meta.description for gobgp is: "A CLI tool for GoBGP"
meta.homepage for gobgp is: "https://osrg.github.io/gobgp/"
meta.changelog for gobgp is: ""
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

/nix/store/x5mni8k390b0wmvw50nqhy4rm5iph9s0-gobgp-2.30.0/bin/gobgp passed the binary check.

1 of 1 passed binary check by having a zero exit code.

1 of 1 passed binary check by having the new version present in output.

found 2.30.0 with grep in /nix/store/x5mni8k390b0wmvw50nqhy4rm5iph9s0-gobgp-2.30.0

found 2.30.0 in filename of file in /nix/store/x5mni8k390b0wmvw50nqhy4rm5iph9s0-gobgp-2.30.0

directory tree listing: https://gist.github.com/fa8afdf61a3c7946b7e911723413c344

du listing: https://gist.github.com/797c28345928b3fb543a87807cb97dd5

Rebuild report (if merged into master) (click to expand)

875 total rebuild path(s)

332 package rebuild(s)

332 x86_64-linux rebuild(s)
277 i686-linux rebuild(s)
12 x86_64-darwin rebuild(s)
254 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
EBTKS
_1password
apacheHttpdPackages.mod_tile
apacheHttpdPackages_2_4.mod_tile
bareos
bicgl
bicpl
broot
cdo
ceph
ceph-client
ceph-csi
ceph-dev
chromium
cloudcompare
cntk
conglomerate
cp2k
dl-poly-classic-mpi
eccodes
elmerfem
epson-escpr2
ezminc
fast-cli
freecad
gdal
gdal_2
gdu
getdp
globalarrays
gmt
gobgp
googleearth-pro
gplates
gpt2tc
grass
grib-api
gromacsDoubleMpi
gromacsMpi
hdf5-mpi
highfive-mpi
hpcg
hpl
icingaweb2
icingaweb2-ipl
inormalize
ior
knot-resolver
lammps-mpi
libLAS

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/x5mni8k390b0wmvw50nqhy4rm5iph9s0-gobgp-2.30.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A gobgp https://github.com/r-ryantm/nixpkgs/archive/c16d933595f7eb9fc27d0c3e8e65efcc1239e889.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/x5mni8k390b0wmvw50nqhy4rm5iph9s0-gobgp-2.30.0
ls -la /nix/store/x5mni8k390b0wmvw50nqhy4rm5iph9s0-gobgp-2.30.0/bin

Pre-merge build results
NixPkgs review skipped

Maintainer pings
cc @higebu @kalbasit for testing.

Result of nixpkgs-review pr 132381 run on x86_64-linux 1

1 package built:

gobgp


Is Felony still being maintained?

I am wondering whether this project is still maintained.
Last release was more than a year ago and there are problems installing on windows for some time now.

Yes.  New commits are still coming in all the time.  That list only shows commits in the master branch.
Issues #14, #30, #32, and #96 are all about failures when installing on Windows.  The developer has asked for some help with this.
That's all I know.

Yes! I'll get a new release built here soon.


Timestamp of when the event occurred

Please document how to get the timestamp of when the key-down or key-up event occurred.  Alternatively, please explain why the timestamp is unavailable/unsupported for keyboard events.
In other words, what is the equivalent of Windows.UI.Input.PointerPoint.Timestamp for keyboard events?
In the old WPF-Classic, you could get the timestamp via System.Windows.Input.KeyEventArgs.Timestamp (inherited from System.Windows.Input.InputEventArgs).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 32569952-2610-a154-ae03-20dfdf09476a
Version Independent ID: 857856a1-3984-ce9d-3aaa-d41fda28a845
Content: Keyboard events - Windows UWP applications
Content Source: windows-apps-src/design/input/keyboard-events.md
Product: uwp
GitHub Login: @Karl-Bridge-Microsoft
Microsoft Alias: kbridge

@verelpode - Thanks for the suggestion! This looks like product feedback rather than documentation feedback. If you are looking for functionality that isn’t currently provided by the product, please use the Windows developer feedback site. We’d be happy to have this on record, where others can view it and upvote it.


Computer build recommendation?

Hi All,
My apologies if this isn't the right place to ask.
TL;DR version:
Are there any recommended build guides or system specs for building a machine specifically to maximize the performance of RawCooked?  Or what resources to focus on?  (cpu threads, cpu frequency, memory, gpu, storage I/O, etc.)
A bit more detail:
I've been testing RawCooked and while watching my system resources I found (sorry for the subjective numbers/wording)
Command used: RAWcooked.exe --all ./dpx_2k_files/ -o G:\temp\rc_dpx_2k.mkv
(this is reading from one SSD and writing to another - both drives are local)
During analysis:
50% read speed from source
CPU is only "semi-busy" (plenty of headroom for more work)
No visible bottlenecks
Then a long pause with no indication of what it's doing during which:
very low cpu usage
source drive still at 50% read (plenty of headroom to read faster)
No visible bottlenecks
Once ffmpeg starts
good cpu usage (50% over all cores)
no source drive activity
low destination drive activity (24%)
CPU wasn't pinned, but it was the only thing doing any real work
Long pause at end of ffmpeg
good cpu usage
minimal activity elsewhere
No visible bottlenecks
System Specs:

i9-7920X 12c/24t (2.90GHz)
64GB RAM
GPU: GTX 1080Ti  (though I don't think RawCooked uses the GPU.  Mentioning for completeness)
Source and Destination are SSD (Samsung SSD 850 EVO or 860  EVO)

Just wondering if I were to upgrade this system or get a new system, specifically for RawCooked, if there is anything I should be focusing on?
Thank you!  And thank you for a nice (and very useful) piece of software!
Dan

My apologies if this isn't the right place to ask.

This is the right place to ask.

Are there any recommended build guides or system specs for building a machine specifically to maximize the performance of RawCooked? Or what resources to focus on? (cpu threads, cpu frequency, memory, gpu, storage I/O, etc.)

Currently there is no a complete reference for that, we mainly recommend to have good I/O on separate fast (SSD) disks (or network) able to read and write at the speed the CPU can handle.
We base our estimation on 50 Mbps of content per core per Ghz per pass (analysis pass is expected to be fast compared to others). content is width x height x components x bit depth x frame rate, so e.g. HDTV content is 1920x1080x3x10x30 = ~1900 Mbps, for realtime processing you need (for 1 pass):

A CPU able to handle that: 1900 Mbps / 50 Mbps per core / classic 3 GHz = 12 cores.
A input able to handle 1900 Mbps = 250 MB/s.
A output able to handle 1900 Mbps = 250 MB/s. Not all SSD are able to write at this speed during hours.

That said, main issue is to know how much content you have to compress, in how much time. Because real time is not needed, e.g. if you have 100 hours of content to handle in 1 month, a computer able to compress at 0.2x real time is enough.

Just wondering if I were to upgrade this system or get a new system, specifically for RawCooked, if there is anything I should be focusing on?

Coherency between CPU and storage speed.
e.g. real world performance of your SSD is around 300 MB/s so having a bigger CPU would not help (blocked by I/O) and having a faster SSD would not help too (blocked by CPU).

During analysis:
50% read speed from source

This part is opening all DPX files, and we plan to work on having more processing in parallel.

Then a long pause with no indication of what it's doing during which:

Weird, but I guess that this is still analysis.

Once ffmpeg starts
good cpu usage (50% over all cores)

This may be due to issues with RAM bandwidth there, but difficult to say. Definitely more work to do on that part.

Long pause at end of ffmpeg

this is the 2nd pass, decode and check reversibility, linked to CPU

(though I don't think RawCooked uses the GPU.

GPU not yet used but we try to.
We started by having a proof of concept, we have now the proof that the tool is useful for lot of people, and we plan to work on improving speed, but no ETA (long todo-list with also other projects).
Note that in the meanwhile some entities use RAWcooked with GNU Parallel in order to run several instances of RAWcooked at the same time, it waives a lot the slowdown of one instance and would put your CPU and you SSD at its maximum.

Thank you! And thank you for a nice (and very useful) piece of software!

Thank you :).


[Bug] - Seems to have a lot of bugs

Issue: Unable to unmarshal json if one of the field data type is different from actual data.
Code:

type Provider struct {
	Name           string        `gorm:"name" json:"name" form:"name" query:"name"`
	Slug           string        `gorm:"slug" json:"slug" form:"slug" query:"slug"`
	Alias          string        `gorm:"alias" json:"alias" form:"alias" query:"alias"`
	Description    string        `gorm:"description" json:"description" form:"description"`
	ProviderType   string        `gorm:"provider_type" json:"provider_type" form:"provider_type" query:"provider_type"`
	IsActive       bool          `gorm:"is_active" json:"is_active" form:"is_active"`
	Currency       string        `gorm:"currency" json:"currency" form:"currency"`
	Balance        money.Decimal `gorm:"column:balance;type:decimal(11,4);" json:"balance" form:"balance"`
	AuthType       string        `gorm:"auth_type" json:"auth_type" form:"auth_type" query:"auth_type"`
	Addr           string        `gorm:"addr" json:"addr" form:"addr" query:"addr"`
	AuthAddr       string        `gorm:"auth_addr" json:"auth_addr" form:"auth_addr" query:"auth_addr"`
	AccessToken    string        `gorm:"-" json:"access_token" form:"access_token" query:"access_token"`
	ExpiresIn      time.Time     `gorm:"-" json:"expires_in" form:"expires_in" query:"expires_in"`
	Username       string        `gorm:"username" json:"username" form:"username" query:"username"`
	Password       string        `gorm:"password" json:"password" form:"password" query:"password"`
	SystemType     string        `gorm:"system_type" json:"system_type" form:"system_type" query:"system_type"`
	Throttle       int           `gorm:"throttle" json:"throttle" form:"throttle" query:"throttle"`
	MaxConnection  int           `gorm:"max_connection" json:"max_connection" form:"max_connection" query:"max_connection"`
	ReadTimeout    time.Duration `gorm:"read_timeout" json:"read_timeout" form:"read_timeout" query:"read_timeout"`
	WriteTimeout   time.Duration `gorm:"write_timeout" json:"write_timeout" form:"write_timeout" query:"write_timeout"`
	RebindDuration time.Duration `gorm:"rebind_duration" json:"rebind_duration" form:"rebind_duration" query:"rebind_duration"`
}
func main() {
	var provider Provider
	data := []byte(`{"type":{"title":"Smpp"},"name":"test 123","slug":"","alias":"test","description":"","addr":"test:2775","system_type":"","throttle":10,"read_timeout":"20","write_timeout":10,"rebind_duration":10,"auth_type":"basic","username":"test","password":"test","max_connection":1,"provider_type":"smpp"}`)
	json.Unmarshal(data, &provider)
	fmt.Println(provider)
	fmt.Println(provider.ReadTimeout)
}

For e.g. ReadTimeout is time.Duration and passed string "20". Data is not unmarshalled to struct as intended

@sujit-baniya
You are wrong. This input data cannot be decoded properly . ( See https://play.golang.org/p/KlxUiTwOKKD )

Also, please correct this issue's title as it gives unnecessary misunderstandings .

@goccy yes with panic it can't be decoded... but if the error is skipped then I suppose go-json and encoding/json works in identical manner, isn't it?

encoding/json does not guarantee the result when an error occurs during Unmarshal.
In other words, it is not a specification, so it should be considered separately from the compatibility story.

Got it! Thanks @goccy


Upgrade MITRE ATT&CK Navigator to v16

MITRE ATT&CK just released v16
We should switch to it
https://medium.com/mitre-attack/attack-v16-561c76af94cf

Changes done in commit cec11f7b7ea7cef397b92b01ede4d903ebf272a7


Potential error in documentation of keras.random.uniform.

In the documentation of keras.ranom.uniform I find
seed: A Python integer or instance of
    `keras.random.SeedGenerator`.
    Used to make the behavior of the initializer
    deterministic. Note that an initializer seeded with an integer
    or None (unseeded) will produce the same random values
    across multiple calls. To get different random values
    across multiple calls, use as seed an instance
    of `keras.random.SeedGenerator`.

For me this implies that
keras.random.unform(shape=(), seed=None) == keras.random.unform(shape=(), seed=None)
Which however is not the case.
See the colab here to see that calling keras.random.unform(shape=(), seed=None) produces a sequence of random numbers.
On the other hand seed=1 really creates the same number for all calls in keras but not in Tensorflow, which is what I was expecting from the documentation.
Am I misunderstanding this or is the documentation about seed=None wrong?

It's a bit subtle because the behavior isn't quite the same when executing eagerly and when compiling.
seed=None gets converted to a random int seed. When executing eagerly, you get a different random int seed each time, so you see a different output each time. But when you compile, the int seed generated at function tracing time gets baked into the compiled graph, and so your compiled function will output the same value each time it's executed.
So if you want to generate different values at each execution even when compiling, you really need seed=SeedGenerator(). The documentation is accurate when running compiled functions. It is not quite accurate for eager execution though. If you can propose some edits to improve it, please do.

Ok, I can suggest a reformulation, but I'd first like to confirm that we agree on what is happening. Follwing your explanation, I looked this up in the code and I found here the implementation of the uniform function for the tensorflow backend
def uniform(shape, minval=0.0, maxval=1.0, dtype=None, seed=None):
    dtype = dtype or floatx()
    seed = _cast_seed(draw_seed(seed))
...

with draw_seed (here) using a global seed generator when seed is None:
...
    elif seed is None:
        return global_seed_generator().next(ordered=False)
...

If I read the global_seed_generator implementation correctly, it works the same for all backends, in compiled or eager mode, besides for the jax backend, where the use of a global SeedGenerator is not supported. So I would conclude that for seed=None a global seed generator is used, and random numbers will not be the same across calls.
If you are fine with this conclusion, then I would suggest changing
      Note that an initializer seeded with an integer
      or None (unseeded) will produce the same random values
      across multiple calls. To get different random values
      across multiple calls, use as seed an instance
      of `keras.random.SeedGenerator`.

      Note that passing an integer as a seed value will always produce the same random values
      across multiple calls. To get different random values
      across multiple calls, you need to use an instance
      of `keras.random.SeedGenerator` as seed value. In case you pass `None` as seed
      a global (internal) SeedGenerator will be used. Note however that the use of the global
     `keras.random.SeedGenerator` is not supported when tracing functions for jit compilation
      with the jax backend.

I could create a pull request if this would be fine for you. I would then also propose to change the documentation of the SeedGenerator class saying that when seed is set to None the python random generator is used to create a seed value, and that the creation of reproducible results can then be achieved using keras.utils.set_random_seed.

You're right, the very first implementation of seed=None converted it to a random int, but at some point I changed it to use a global SeedGenerator (in order to avoid a common bug where people don't see their generators and expect different values at each call). However with the JAX backend doing this will fail (with a very explicit and clear error message) since any element of state must be tracked locally.
Your description is accurate.


Hearing artifacts in android example in Lyra

Tried Lyra android example on Pixel 4 but heard quite audible artifacts during decoding. There were frequent truncation instances too where I would only part of a sentence on playback with sudden cutoff.

Thank you for your report.

Artifacts are worth looking into - one thing you should try is being close to the microphone vs far.  Re:truncation, the demo only records up to 5 seconds of audio.  You can change that in the code if you want to experiment with longer durations.

The new Lyra 1.2.0 release has significant quality improvements, if you want to try it out. Please let us know if you still experience these artifacts.


about txn_test_gen_plugin question

i run curl --data-binary '["eosio", "5KQwrPbwdL6PhXujxW37FSSQZ1JiwsST4cqQzDeyXtP79zkvFD3"]' http://<IP_ADDRESS>:8888/v1/txn_test_gen/create_test_accounts
and have error:
2018-08-02T03:37:54.325 thread-0   net_plugin.cpp:2517           operator()           ] accept txn threw  3040000 transaction_exception: Transaction exception action's authorizing actor 'txn.test.t' does not exist     {"account":"txn.test.t"}     thread-0  controller.cpp:1590 validate_referenced_accounts
I guess it's because of the parallel processing of create account transactions  and set code transactions
and i try let create_test_accounts function's "create some test accounts" and "set txn.test.t contract to hotio.token & initialize it"  on one's own,and success.
like:
static uint64_t flag = 1;
if(flag == 1) { //create some test accounts flag=2; } else if(flag == 2) { //set txn.test.t contract to hotio.token & initialize it }
so,is i use txn_test_gen_plugin wrong way, or this is issues?

This plugin is only for performance testing only. If you still have the problem and wish to discuss further, please post it in https://eosio.stackexchange.com.


support for amd64 cpu?

Hello,
can i create an image or iso for installing Raspiblitz in VMware server?
thanks

@openoms any thoughts on this?

The most tricky part is to get the simulated disks right but certainly possible.
No permission just go ahead and try @xpsx.
You can browse this issue about some clues when we last worked on this:
https://github.com/openoms/raspiblitz/issues/46
Personally I have enough SBC nodes and no servers with superfluous RAM so don't really feel the need.

Looks similar to #1386 to me. Did a quick internet search. Disk names are /dev/sdXn. Same as in my linux kvm/qemu vm.
I dont have VMWare so @xpsx can test #1297

Running in an Ubuntu VM on TrueNAS. See the notes here: https://github.com/rootzoll/raspiblitz/issues/1500#issuecomment-917444439

closing this for inactivity, see the newer VM related issues


Slottransactions are not being cleared

SpongeForge version:  (Tested with some build provided by @bloodmc with some extra debug logging)

SpongeVanilla version: BETA-67

Java version: 1.8

Operating System: Windows 10

Plugins/Mods:
All code to reproduce the issue can be found here: (branch inventories)
https://github.com/NeumimTo/spongebugs/tree/inventories/src/main/java/bugs/bugs

The issue is present on both implementations, forge and vanialla.
Steps to reproduce:
Install the plugin and break any block
An inventory full of items opens
Click on any item
Now you should see some debug log in console, it should tell you that the transaction array size was 1 which is correct
Click on any other item
Now the debug log tells you that there are 7 slot transactions in the array.
If you will continue randomly clicking on items the slot transaction array will grow in size without ever being cleared.
If you attempt to shift click the gray glass panes multiple times withing a very short time interval it usually timeouts the client after a while
This is was not happening with previous stable builds.

Due to my current upload speed currently being around 7kb/s i cannot upload the forge build with extra debug output :(


Preparing PyLint for subpackage breakup.

This change started out in #2367 but I figured we'd get it out of the way here since it's not as much in dispute.

@tseaver Sorry I just realized I left out the important part: Taking the docstrings out of namespace package __init__.py.

@tseaver LMK if the 2nd commit was problematic.

@dhermes Nope.


Add .packed folder in the gitignore file

https://github.com/MyPrivateRepositories/guide-open-source/issues/32

https://github.com/MyPrivateRepositories/guide-open-source/pull/34


A new "Flask extensions registry" page would be really useful to have

As the title suggests, I believe that a "Flask extensions registry" page would be really useful to have in this new website as well, as we had it in the previous version of the website (https://github.com/pallets/flask-website/tree/master/flask_website/templates/extensions)
It was a very useful resource especially for newcomers, as we know that some of the extensions that are available for the framework are better than others, and having to look for the best available solution to a problem on PyPi can sometimes be challenging.
Please consider adding this feature again!

It was a very useful resource especially for newcomers, as we know that some of the extensions that are available for the framework are better than others

That the list gave these impressions was one of the core reasons the list was removed.
The list was unfortunately just way too difficult to maintain, and made many things that were not official seem more important. It hadn't been updated in years, the "approved" marker was impossible to keep validated, some projects were no longer maintained or obsolete, and it only listed things from authors who knew to request the listing.


JetBrains: "Copy" button border gets cut for generated test in Cody window.

Installation Information
Plugin version: 5.3.27
IDE version: IC-233.14475.28
Describe the bug
Steps to reproduce:

Have Cody plugin installed in your JetBrains IntelliJ IDEA editor
User is already logged in to Cody with Pro subscription
Select code and click on "Generate Test" command
Check the generated test from the response

Observed behavior:
Border for "Copy" button gets cut.
Expected behavior
"Copy" button should be clearly visible.
Additional context (logs, images, etc)
Screenshot:

Happens on linux for pycharm using the old style jetbrain UI as well


naive bayes

Have you read the Contributing Guidelines on Pull Requests?
yes
Description
added naive bayes algo
Checklist

[x] I've read the contribution guidelines.
[x] I've checked the issue list before deciding what to submit.
[x] I've edited the README.md and link to my code.

Related Issues or Pull Requests
#7310

@ankitaggarwal23   @bhav09 do checkout please

I haven't added the output , as in that case I have to add one or more file ( such as data  for training and all) So I have only added main algorithm.


Modernized app/views/account/*

Also fixed bug in api key manager: jquery.show() doesn't seem to be able to show something hidden with class="hidden".  Changed it to "display:none" and now jquery.show() can manage it.  Weird.  I could've sworn this worked before.
Also added data-autofocus as an alternative to plain vanilla autofocus.  I don't know about other browsers, but autofocus causes firefox to scroll stupidly down the page, hiding the title and stuff.  If instead wewait until the page is loaded and then focus on the first element with data-autofocus=true, then it just focuses without scrolling.

Yeah, it's annoying.  Check out bootstraps css for hidden.  jQuery doesn't
change the visibility or override the !important.
.hidden { display: none !important; visibility: hidden !important;}
Ray Suelzer
Mobile:<PHONE_NUMBER>
On Tue, Mar 3, 2015 at 5:47 PM, Jason Hollinger<EMAIL_ADDRESS>wrote:

Merged #133
https://github.com/MushroomObserver/mushroom-observer/pull/133.
--
Reply to this email directly or view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/133#event-245605414
.

Oh, is that why?  I figured jquery changed both visibility and display, but
didn't consider !important.  Out of curiosity, how would you override an
!important styling in javacsript / jquery?
On Tue, Mar 3, 2015 at 5:10 PM, Raymond Suelzer<EMAIL_ADDRESS>wrote:

Yeah, it's annoying. Check out bootstraps css for hidden. jQuery doesn't
change the visibility or override the !important.
.hidden { display: none !important; visibility: hidden !important;}
Ray Suelzer
Mobile:<PHONE_NUMBER>
On Tue, Mar 3, 2015 at 5:47 PM, Jason Hollinger<EMAIL_ADDRESS>wrote:

Merged #133
https://github.com/MushroomObserver/mushroom-observer/pull/133.
--
Reply to this email directly or view it on GitHub
<
https://github.com/MushroomObserver/mushroom-observer/pull/133#event-245605414
.

—
Reply to this email directly or view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/133#issuecomment-77077692
.

you'd have to do
$('#foo').style('display', 'block');
$('#foo').style('visiblity', 'visible');
but if you really just want to hide or unhide things in bootstrap it's
easiest to use  $('#foo').toggleClass('hidden');
Ray Suelzer
Mobile:<PHONE_NUMBER>
On Tue, Mar 3, 2015 at 8:12 PM, Jason Hollinger<EMAIL_ADDRESS>wrote:

Oh, is that why? I figured jquery changed both visibility and display, but
didn't consider !important. Out of curiosity, how would you override an
!important styling in javacsript / jquery?
On Tue, Mar 3, 2015 at 5:10 PM, Raymond Suelzer<EMAIL_ADDRESS>wrote:

Yeah, it's annoying. Check out bootstraps css for hidden. jQuery doesn't
change the visibility or override the !important.
.hidden { display: none !important; visibility: hidden !important;}
Ray Suelzer
Mobile:<PHONE_NUMBER>
On Tue, Mar 3, 2015 at 5:47 PM, Jason Hollinger <
<EMAIL_ADDRESS>wrote:

Merged #133
https://github.com/MushroomObserver/mushroom-observer/pull/133.
--
Reply to this email directly or view it on GitHub
<

https://github.com/MushroomObserver/mushroom-observer/pull/133#event-245605414

.

--
Reply to this email directly or view it on GitHub
<
https://github.com/MushroomObserver/mushroom-observer/pull/133#issuecomment-77077692

.

--
Reply to this email directly or view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/133#issuecomment-77077968
.

vanilla javascript would be something like:
document.getElementById('foo').style.visibility = 'visible';      i
think...
Ray Suelzer
Mobile:<PHONE_NUMBER>
On Tue, Mar 3, 2015 at 8:20 PM, Raymond Suelzer<EMAIL_ADDRESS>wrote:

you'd have to do
$('#foo').style('display', 'block');
$('#foo').style('visiblity', 'visible');
but if you really just want to hide or unhide things in bootstrap it's
easiest to use  $('#foo').toggleClass('hidden');
Ray Suelzer
Mobile:<PHONE_NUMBER>
On Tue, Mar 3, 2015 at 8:12 PM, Jason Hollinger<EMAIL_ADDRESS>wrote:

Oh, is that why? I figured jquery changed both visibility and display, but
didn't consider !important. Out of curiosity, how would you override an
!important styling in javacsript / jquery?
On Tue, Mar 3, 2015 at 5:10 PM, Raymond Suelzer<EMAIL_ADDRESS>
wrote:

Yeah, it's annoying. Check out bootstraps css for hidden. jQuery doesn't
change the visibility or override the !important.
.hidden { display: none !important; visibility: hidden !important;}
Ray Suelzer
Mobile:<PHONE_NUMBER>
On Tue, Mar 3, 2015 at 5:47 PM, Jason Hollinger <
<EMAIL_ADDRESS>wrote:

Merged #133
https://github.com/MushroomObserver/mushroom-observer/pull/133.
--
Reply to this email directly or view it on GitHub
<

https://github.com/MushroomObserver/mushroom-observer/pull/133#event-245605414

.

--
Reply to this email directly or view it on GitHub
<
https://github.com/MushroomObserver/mushroom-observer/pull/133#issuecomment-77077692

.

--
Reply to this email directly or view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/133#issuecomment-77077968
.


No Sitemap

Should generate sitemap

https://github.com/jekyll/jekyll-sitemap/

Added c9d52d0febafbe9fe3285ff38cbe82821c4ef2fa


Wrong kernel version string for 3.18.7-v7+

Hi @notro,
I tried rpi-source on Raspbian for kernel 3.18.7-v7+, but it builds modules for version 3.18.7+-v7 (note the position of the plus sign). When I try to insmod I get "Invalid module format", presumably because of the version mismatch.
Not sure if this is related to issue #9.

Please post output when run with debuging enabled.

Here is the output:
....................................................................................................................................................................................................................
....................................................................................................................................................................................................................
.......................................................................................................................................................  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  SHIPPED scripts/kconfig/zconf.tab.c
  SHIPPED scripts/kconfig/zconf.lex.c
  SHIPPED scripts/kconfig/zconf.hash.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
scripts/kconfig/conf --silentoldconfig Kconfig
  CHK     include/config/kernel.release
  UPD     include/config/kernel.release
  WRAP    arch/arm/include/generated/asm/auxvec.h
  WRAP    arch/arm/include/generated/asm/bitsperlong.h
  WRAP    arch/arm/include/generated/asm/cputime.h
  WRAP    arch/arm/include/generated/asm/current.h
  WRAP    arch/arm/include/generated/asm/emergency-restart.h
  WRAP    arch/arm/include/generated/asm/errno.h
  WRAP    arch/arm/include/generated/asm/exec.h
  WRAP    arch/arm/include/generated/asm/hash.h
  WRAP    arch/arm/include/generated/asm/ioctl.h
  WRAP    arch/arm/include/generated/asm/ipcbuf.h
  WRAP    arch/arm/include/generated/asm/irq_regs.h
  WRAP    arch/arm/include/generated/asm/kdebug.h
  WRAP    arch/arm/include/generated/asm/local.h
  WRAP    arch/arm/include/generated/asm/local64.h
  WRAP    arch/arm/include/generated/asm/mcs_spinlock.h
  WRAP    arch/arm/include/generated/asm/msgbuf.h
  WRAP    arch/arm/include/generated/asm/param.h
  WRAP    arch/arm/include/generated/asm/parport.h
  WRAP    arch/arm/include/generated/asm/poll.h
  WRAP    arch/arm/include/generated/asm/preempt.h
  WRAP    arch/arm/include/generated/asm/resource.h
  WRAP    arch/arm/include/generated/asm/rwsem.h
  WRAP    arch/arm/include/generated/asm/scatterlist.h
  WRAP    arch/arm/include/generated/asm/sections.h
  WRAP    arch/arm/include/generated/asm/segment.h
  WRAP    arch/arm/include/generated/asm/sembuf.h
  WRAP    arch/arm/include/generated/asm/serial.h
  WRAP    arch/arm/include/generated/asm/shmbuf.h
  WRAP    arch/arm/include/generated/asm/siginfo.h
  WRAP    arch/arm/include/generated/asm/simd.h
  WRAP    arch/arm/include/generated/asm/sizes.h
  WRAP    arch/arm/include/generated/asm/socket.h
  WRAP    arch/arm/include/generated/asm/sockios.h
  WRAP    arch/arm/include/generated/asm/termbits.h
  WRAP    arch/arm/include/generated/asm/termios.h
  WRAP    arch/arm/include/generated/asm/timex.h
  WRAP    arch/arm/include/generated/asm/trace_clock.h
  WRAP    arch/arm/include/generated/asm/unaligned.h
  CHK     include/generated/uapi/linux/version.h
  UPD     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  UPD     include/generated/utsrelease.h
  HOSTCC  scripts/basic/bin2c
  Generating include/generated/mach-types.h
  CC      kernel/bounds.s
  GEN     include/generated/bounds.h
  CC      arch/arm/kernel/asm-offsets.s
  GEN     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  SHIPPED scripts/dtc/dtc-lexer.lex.c
  SHIPPED scripts/dtc/dtc-parser.tab.h
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  SHIPPED scripts/dtc/dtc-parser.tab.c
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTLD  scripts/dtc/dtc
  HOSTCC  scripts/genksyms/genksyms.o
  SHIPPED scripts/genksyms/parse.tab.c
  HOSTCC  scripts/genksyms/parse.tab.o
  SHIPPED scripts/genksyms/lex.lex.c
  SHIPPED scripts/genksyms/keywords.hash.c
  SHIPPED scripts/genksyms/parse.tab.h
  HOSTCC  scripts/genksyms/lex.lex.o
  HOSTLD  scripts/genksyms/genksyms
  CC      scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  CC      scripts/mod/devicetable-offsets.s
  GEN     scripts/mod/devicetable-offsets.h
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTLD  scripts/mod/modpost
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/pnmtologo
  HOSTCC  scripts/conmakehash
  HOSTCC  scripts/recordmcount
  HOSTCC  scripts/sortextable
Check for update to rpi-source
download: https://api.github.com/repos/notro/rpi-source/git/refs/heads/master

 *** gcc version check: OK

 *** Using: /usr/share/doc/raspberrypi-bootloader/changelog.Debian.gz
Latest changes: raspberrypi-firmware (1.20150214-1) unstable; urgency=low

 *** Latest firmware revision: 8aca5762
Latest change made by<EMAIL_ADDRESS>Sun, 15 Feb 2015 11:19:20 +0000
download: https://raw.githubusercontent.com/raspberrypi/firmware/8aca5762/extra/git_hash

 *** Linux source commit: 0be82f722c097340632a59b879fbfee9c6148f53

 *** Download kernel source
wget  -O /root/linux-0be82f722c097340632a59b879fbfee9c6148f53.tar.gz https://github.com/raspberrypi/linux/archive/0be82f722c097340632a59b879fbfee9c6148f53.tar.gz

 *** Unpack kernel source
cd /root && tar --checkpoint=100 --checkpoint-action=dot -xzf /root/linux-0be82f722c097340632a59b879fbfee9c6148f53.tar.gz

 *** set 'EXTRAVERSION = +' in Makefile
sed -i 's/EXTRAVERSION =.*/EXTRAVERSION = +/' /root/linux-0be82f722c097340632a59b879fbfee9c6148f53/Makefile

 *** Create symlink: /root/linux
rm -f /root/linux
ln -s /root/linux-0be82f722c097340632a59b879fbfee9c6148f53 /root/linux

 *** Create /lib/modules/<ver>/{build,source} symlinks
sudo rm -f /lib/modules/$(uname -r)/build /lib/modules/$(uname -r)/source
sudo ln -sf /root/linux /lib/modules/$(uname -r)/build
sudo ln -sf /root/linux /lib/modules/$(uname -r)/source

 *** .config
writef(/root/linux-0be82f722c097340632a59b879fbfee9c6148f53/.config)

 *** Module.symvers
download_to: https://raw.githubusercontent.com/raspberrypi/firmware/8aca5762/extra/Module7.symvers -> /root/linux-0be82f722c097340632a59b879fbfee9c6148f53/Module.symvers
cd /root/linux-0be82f722c097340632a59b879fbfee9c6148f53 && cp -a Module.symvers Module.symvers.backup

 *** make modules_prepare
cd /root/linux && make modules_prepare

 *** Help: https://github.com/notro/rpi-source/wiki

I think the problem is some mixup with EXTRAVERSION and LOCALVERSION.

Yes, that's the output of rpi-source --verbose. Stdout anyway. Stderr only contained download progress.

Ah, sorry, read this on email and didn't scroll down far enough to see the bits I was looking for !PeterO
 On Tuesday, 5 May 2015, 8:17, Alexander Klauer<EMAIL_ADDRESS>wrote:

Yes, that's the output of rpi-source --verbose. Stdout anyway. Stderr only contained download progress.—
Reply to this email directly or view it on GitHub.

Fixed https://github.com/notro/rpi-source/commit/f5809e7d744d010b0d6e420dd2659efd238a5851


Can reference images be processed invidually ?

Hi!
In the code, it seems that the ref latents are parallelly concatenated with the own latent and being denoised simultaneously ? May I know if it is equivalent to first invidually get the self-attention key and value of each ref invidually, and then concat them together during denoising ?

yes, you can get the self-attention key and value of each ref invidually first, and then concat them together during denoising, it will save your GPU memory

Thanks


Documentation Request

After looking at this unified repository I quite like the new approach being taken on creating a monorepo with all available tools sharing a unified framework.
There is a distinct lack of any documentation that shows real life usage. For example, ESRGAN and EDVR both lack usage examples. Whilst looking at the config code I think that I understand how to train new models. There is no usage guide. There is no documented way of running validation against a large image set and no obvious way to use different data sets other than the project defaults.
In previous repositories for ESRGAN (and other implementations) it was possible to execute test.py with an appropriately configured YAML file for the test. The same is not true here and there is no documentation explaining how to use the utilities!
I'd recommend creating a usage page on the wiki demonstrating how to make use of the pretrained models, etc... for each sub tool. Is this on the roadmap?
Anyway, keep up the good work! This set of tools is a fantastic effort and deserves lots of praise! Cutting edge stuff, keep it up!

Hi, @DanielHaroldLane. Thank you for letting us be aware of the lack of a clearer documentation!
You can take a look at the GettingStarted page for the basic usage. For example, you can use below command to test a model. If you want to replace your own dataset, you may need to read the code under mmedit/datasets/ and create your own dataset. Also remember to replace the dataset type in model configs (train_dataset_type , val_dataset_type  etc.).
# single-gpu testing
python tools/test.py configs/example_config.py work_dirs/example_exp/example_model_20200202.pth --out work_dirs/example_exp/results.pkl

# multi-gpu testing
./tools/dist_test.sh configs/example_config.py work_dirs/example_exp/example_model_20200202.pth --save-path work_dirs/example_exp/results/

We are also working on more concrete tutorials that may serve as a wiki. Thank you for you patient!


Fixes revoke response throwing exception

Closes thatJavaNerd/JRAW#126.
Adds code to RestResponse to create a proper object for a HTTP 204 response. Prevents OAuthHelper.revokeToken() from throwing an exception on a 204 response. Let me know if you have a better solution here.

Good work, thank you!


Fix Microsoft Endpoint Manager broken link

In this directory https://microsoft.github.io/PartnerResources/modern-workplace/security/

Appears to be resolved now.


fix acquire deadlock

Coverage increased (+13.8%) to 65.426% when pulling 89b36f13bde3ea1dcc1aa9f08f38816d006f9696 on fix-acquire-deadlock into ea6efecfdb60af74815b565e3d07010140f623f6 on master.


⚠ Bofrost.es has degraded performance

In 1c64a18, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 2691 ms

Resolved: Bofrost.es performance has improved in 9dc46dd after 17 minutes.


Upgrade to Laravel 7

Allow illuminate ~7.0 dependencies
NOTE: depends on ixudra/core this package should have a new release supporting Laravel 7 first. The package version should be updated in this package as well in order to work.

Fixes #1

Thank you for contributing!


[Feature]: Converation statistics data visualization

Contact Details
<EMAIL_ADDRESS>What feature are you requesting?
Display certain data visualization for certain conversation, such as world cloud. Use Bokeh lib for  visualizations.

https://github.com/SkywardAI/chat-frontend/issues/107


users ssh keys don't allow sos access to read boot logs

Just using Equinix Metal's SOS will require adding new SSH keys to the Equinix Metal project.
One way this could be achieved is through some sort of shim

Any luck here? I think you found a way to pass keys.


Require<EMAIL_ADDRESS>

I was getting these errors after doing git pull && npm install in clinic doctor:
/home/goto-bus-stop/Code/nearform/node-clinic-doctor/node_modules/turbo-json-parse/index.js:27
  gen.scope.console = console
                    ^
TypeError: Cannot set property 'console' of undefined
    at compile (/home/goto-bus-stop/Code/nearform/node-clinic-doctor/node_modules/turbo-json-parse/index.js:27:21)

Because I still had an old generate-function version in the node_modules
tree, and npm considered it up to date enough.
Since turbo-json-parse uses functionality from v2.2.0 now it should also depend on 2.2.0+

CI killed the node 6 test for too much memory maybe? Seems unrelated to this PR anyhow.

thanks!


Remove all remaining unnecessary casts

There is still such a cast in the goog library itself.
See https://github.com/google/closure-library/pull/637.
If/when the PR is merged in closure library, I will create a new PR to enable
unnecessary casts checks.

Travis timed out.

Rebased onto master. Any review?

@fredj found out there is a way to restrict the checks to our code.
I enabled this restriction and removed the jscomp_off for unneccessaryCasts.
Please review.

This looks great.  Thanks @gberaudo.

Thanks for the review @tschaub.


DS-177: update academy color and gradient usage with new brand colors

Jira
https://pegadigitalit.atlassian.net/browse/DS-177
Summary
Updates Academy demo pages' color and gradient usage with new brand colors.
Details

Purple gradient is deprecated, new one is called Pink.
Indigo gradient is deprecated, new one is called Navy.
The two deprecated gradients will continue to work, but they will render Pink and Navy visually respectively.
Outdated home page and related CSS are removed.
Added berry, wine, and violet as options for icon color.

How to test
Run the branch locally and check all Academy pages, make sure no gradients look broken.

As discussed with @remydenton, we've deprecated a few colors.


Unable to find xrf file

Hi!
I was trying to run hgdb-vitis on a simple design to see if I have installed it right. But I got this issue that I cannot solve:
AssertionError: Unable to find file solution1/.debug/top_Pipeline_VITIS_LOOP_12_1.xrf

Would you please help me with this?

Did you turn the debug flag on in Vitis?

Thank you for quick respond!
Do you mean config_debug? I tried to, but in the GUI of Vitis it would let me change the value of config_debug, so I had to run vitis in interactive mode in terminal to use command to change the value of config_debug and than open the solution, but I still get the same error.

You should use config_debug -enable in your setup script and then run the entire HLS flow, i.e. C-simulation and synthesis, then you should find the xrf file.

I have tried it. But I still get that error. Here is my tcl script. Am I doing it right?

Hmm I'm not sure manually editing tcl script would work, since Vitis overrides that file quite often. Here is what I did:

Then if you run C SYNTHESIS, you will see the files in the debug folder:
$ ls .debug/*.xrf
.debug/convolve_Pipeline_VITIS_LOOP_6_1.xrf  .debug/convolve.xrf

Oh yeah it worked! But I'm running into some other error

Make sure you also run the C-simulation. I think these bitcode files are created from the C sim.

Yeah I did run the C-simulation. Am I using the proper config?

That looks correct. Do you have a function called top by any chance? It could be that the heuristics I used may not pick up the design hierarchy properly. If it is possible, can you share your design file here?

Sure! I'm just trying to run it on a very simple prime check design. The top function is called top. But I have tried to change it to something else and it still doesn't work. It seems the .c file is not allowed to be posted here, I'll just put an image.

Sorry for the delay. I'm swamped by the workload I have right now. I will definitely get back to you once I have more free time.

Yeah, sure!

Hello, my name is Nathan and Shang has passed this project onto me.
I think I have figured out the most recent issue of not finding the debug bitcode. When I searched the .autopilot/db/ directory for the bc file I found a few different ones but one in particular named "PC.bc". I think that this means hgdb-vitis is looking for a bitcode based off the function name (top.bc) but vitis is writing it based off the filename (PC.bc). I changed the file and function names to match and I believe that I have progressed further into the program but let me know if my conclusion is wrong and that's an unrelated file.
Since then I have run into a different issue as shown in my screenshot. It says that it is unable to determine the location for my file pc.c but it has given the exact path for the file so I'm not sure what it is unable to find.
Thanks for your help thus far by the way.

@NathanRizza Thanks for the info. I think you're right that Vitis generates the bitcode object by filename, not the function name. During my tests I always keep them the same so I never noticed it. I will figure out a way to detect filename automatically.
From the error message it seems like it has obtained all the debugging information but failed to produce a valid symbol table. Here is code section that's throwing the error: https://github.com/Kuree/hgdb-vitis/blob/5b15700a6e008c243dd00910c3f69693c4b0fe2c/python/ir.cc#L987
Is it possible to zip up your solution folder and upload it somewhere? You can see some example file structure here. Basically it needs syn, .autopilot, and .debug folder.
I should have more free time this week and I will take a look. Hopefully it's an easy fix.

@Kuree I have uploaded the soluition folder to this link: "https://mega.nz/file/tfI3kIzI#lrHHJH4AuX1jJguGmC8KyufPfozjIOCxikeK6DSozUI". Thanks for taking a look at it.

@NathanRizza I am able to reproduce this bug. When I ran hgdb-vitis, LLVM i actually complaining the invalid debug info:
DILocation not allowed within this metadata node
!13 = !{!"fpga.top", !"user", !14}
!14 = !DILocation(line: 1, column: 16, scope: !7)

This means that the bitcode is produced using a much older compiler. What version of Vitis are you using? I'm using 2021.1 and it seems to produce fairly recent bitcode files.
I've successfully opened the bitcode file with much older llvm, e.g. llvm-7. I will keep experimenting.

@Kuree I am using vitis version 2022.2. Also I am running ubuntu 22.04 LTS. When Checking the version of llvm I am running I get this "llvm is already the newest version (1:14.0-55~exp2)" so I think you're observations are correct. Is this an issue that can be fixed for the newer version of llvm or should I downgrade to llvm-7?

I don't think you need to downgrade. I just need to find a way to set up the code such that it should dynamically detect the Vitis version and choose the correct LLVM to build against (and download if not exist). I will keep you posted.

Should be fixed in the master branch. Feel free to reopen or create a new issue if it doesn't work for you. I was able to create a symbol table given your solution files.


How do I disable automatic data synchronization in the HDFS?

alluxio config
alluxio.user.file.metadata.sync.interval=-1
unpin /
$ ./bin/alluxio fs getSyncPathList
The following paths are under active sync
No synchronization function is enabled

How did this garbage file get loaded into Alluxio?Nor is there any Alluxio service to call to read the directory's data?How do I disable it from automatically synchronizing data from HDFS?

read cache blacklist How to configure it?  I only see this one, so it doesn't have any effect, same cache.Trash data
alluxio.master.persistence.blacklist=.staging,.tmp,.Trash,.hive-staging_hive

We don't support read-cache blacklist, let's mark this issue as a feature request,  thanks @liangrui1988


Updating cols doesn't update the GridLayout v.0.17.1

Colleagues, I want to note that the new version has ceased to support previously announced functionality (v0.16.6). In particular, it does not recount ResponsiveGridLayout when updating columns:
https://codesandbox.io/s/polished-https-v9sb5

@STRML any updates on it?

@STRML Please reopen this issue, because it's serious issue and still not fixed. And would be great if you could kill this annoying github-action bot, because it's not for the first time when it closed active issue. Otherwise I will have to open new issue.

@misha-panyushkin
Looked into the code and found the bug. The bug is already fixed but not uploaded into npm repository. So here is workaround, install dependency from github:
npm i STRML/react-grid-layout#master --save-dev
or go into node_modules/react-grid-layout/build/ResponsiveReactGridLayout.js and change line  187:
if (lastBreakpoint !== newBreakpoint || this.state.breakpoints !== breakpoints || this.state.cols !== cols) {


classes/install: issue in freetype2 lib pc file

if i build the lib freetype2 with bob, i get in dist directory a broken pc file:
Requires:  ,

Fixed with #38

Fixed with #38


Update testing-with-copilot.md

The code was updated to use modern ES6 import syntax instead of require. Additionally, it now uses the OpenAI class directly instead of OpenAIApi and Configuration, simplifying the initialization process. This modernization improves readability and aligns with current JavaScript standards. No changes were made to the functionality.
Description

This pull request addresses the issue described here: #<4671>

In this pull request, I have changed the chat completions method to be able to use it with last version of openAI.

For features/enhancements:

[x] I have added/updated the relevant references in the documentation file.

Thanks for the contribution @mustafahsz!


add useCapture prop

Hi,
Thank you for your great work!
Could you be kind enough to add support of passing capture prop to addEventListener and removeEventListener respectively?
You can name prop as useCapture (as it is used in documentation).
You have a line in readme:

the coded uses stopPropagation={true} but sibling events still make it to "parents".

This change will fix it 😉

Thanks for the ePR, but I'm going to have to decline it for programming reasons: this would solve the problem only f this was the only usecapture issued, but the moment you have two or more (says some other event also generates with usecapture=true) then there is --as per the spec-- zero guarantee on which one gets to go first, and so you end up back in the same situation as before.
That's why I haven't added it: without usecapture, you know sibling events will always still happen and you will have to make sure your code knows how to deal with that fact: it is 100% predictable and reliable behaviour. However, with usecapture, you think they won't run into this situation, but you actually might, and now you have a near-untracable source of bugs due to non-deterministic browser behaviour.
Having deterministic code behaviour is far more important here =)

Yes, I know. Thats why I ask you to make this property optional (having false` by default).
I use another third-party library for some clickable elements on the page, that do not provide enough way to handle their events. So I look for a way of preventing them from outside.
It is not a breaking change but an opportunity to make your component more flexible.
Thank you.

It sounds like you're trying to use useCapture to solve a problem that it was not designed to solve.  Order of execution is 100% not guaranteed so using useCapture is literally a bug if you think you're using it to add some kind of order of execution. If you really that kind of event examination, then write an event handler that you can pass to two or more components, and have the logic that determines "which gets to go through" live inside that function.

what spec is this exactly? based on this
https://dom.spec.whatwg.org/#dom-eventtarget-addeventlistener

First all object’s ancestor event listeners whose capture variable is set to true are invoked, in tree order. Second, object’s own event listeners are invoked. And finally, and only if event’s bubbles attribute value is true, object’s ancestor event listeners are invoked again, but now in reverse tree order.

it shows they are invoked in tree order. For my case I would like to block an onClick event completely before it reaches the bubble phase in certain scenarios. Even if the capture events weren't in a determined order, it seems that they will all finish before the bubbling phase, no?


Json file does not appear

Here is my conf:

var JSONReporter = require('jasmine-json-test-reporter');

var json_reporter = new JSONReporter({
    file: 'jasmine-test-results.json',
    beautify: true,
    indentationLevel: 4 // used if beautify === true
});

onPrepare: function() {
    jasmine.getEnv().addReporter(reporter);
    jasmine.getEnv().addReporter(json_reporter);
  },

When I tried to console log the result...
console.log(options.file);
console.log(resultsOutput);
I got the expected result but the file is no generated.
any idea?
Thanks, great plugin.

The solution is to pass the relative path.
const fnDirectory = path.join(__dirname,'../test-results/protractor/');
//under the options.
file: fnDirectory+'jasmine-test-results.json',


PPX-less API for creating DOM elements and props.

As discussed in https://github.com/ml-in-barcelona/jsoo-react/discussions/113. This adds a PPX-less API using plain functions to create DOM elements and props.
I didn't manage to actually get started using jsoo-react this week, so none of this is actually tested in the real world. And there's a few essential parts missing still, see the TODO list below. This is a still good point to critique the general approach though.
TODO:

[x] Flesh out Event.Pointer
[x] Implement ref prop
[x] Implement style prop
[x] Implement dangerouslySetInnerHTML prop
[x] Add SVG elements and props
[x] Add proper interface
[x] tests
[x] Context API?
[x] forwardRef?

Closes #105

A couple of issues have been surfaced from rewriting the tests:
1. Namespace pollution causing naming collisions
Importing all these names into the namespace is bound to cause collisions. There are a couple ways we can address this:
a. Import the names in a larger scope. This pollutes even more, of course, but at least allows local bindings to shadow imports. And if done at the top of the file will always allow shadowed names to be accessed by fully qualifying them.
b. Selective imports. Not very idiomatic OCaml, and quite verbose, but this, along with point a above, is how it's done in Elm etc.
c. Qualify each invocation, perhaps using an aliased short-name, e.g. module H = Html module, then H.div [|H.className "foo"|] [].
d. Avoid binding all these names by using and API like h "div" [|p "className" "foo"|] [] for example.
e. Bow down to the deity that is PPX.
2. Conditional props
There's currently no direct support for conditionally passing props. With the current API this is done using optional labeled arguments, and in JSX is done by borrowing the same semantics, but this isn't possible when creating an array directly.
A few options to address this:
a. Offer something like Prop.none as an alias to any "" "", effectively offering a no-op that can be used with if-expressions.
b. Offer some primitives for conveniently concatenating arrays, e.g. Prop.(concat [ [|className "foo"|]; if bar then [|href "..."|] else [||] ]) .
c. Offer some kind of builder-style API to build a props array.
d. Bow down to the deity that is PPX.

Thanks for the work on this and detailed explanation on issues :)

Namespace pollution causing naming collisions

I feel inclined to follow either your suggestion in a. (import the names at the top of the module where the component implementation is being defined). or, using the approach you already applied in existing code (just rename prop to append some underscore, or something renaming solution). Alternatively, selective imports sound great as well, but as you say it can be quite cumbersome in this case, as there are tens of elements and props that are used in a single component tree.

Conditional props

Hm, this is an inconvenient issue indeed. Maybe an alternative option besides the ones you mention (kind of building on option b.) is to add a helper to produce either empty array or array with some prop, then combine this with Array.concat, e.g. for the example in PR:
(* In Dom_html.Prop *)
  let option f value = match value with
  | Some v -> [|f v|]
  | None -> [||]

(* In tests *)
let%component make ~href:href_ =
  React.Dom.Html.(a (Prop.option href href_) [])

Then can be used in combination with Array.concat when other props are involved:
let%component make ~href:href_ =
React.Dom.Html.(
  a
    (Array.concat
       [ Prop.option href href_
       ; [|className "value"; id "some-large-id"; hidden true; lang "es"|]
       ] )
    [])

Bow down to the deity that is PPX.

I hope with the work you're showcasing in this PR we can get to a point where PPX-less is decent enough 💪 😄 Then we can build PPX on top of it, for those that like adventure and want more thrill in their lives.

Re. namespace pollution, I agree with your inclination and definitely don't think this is a deal-breaker, just a bit awkward. The error messages currently emitted when this happens are also not very nice, but should improve when the prop type is abstracted.

(* In Dom_html.Prop *)
let option f value = match value with
  | Some v -> [|f v|]
  | None -> [||]

Good idea! And inspires me to make a further refinement! What if instead we define option to be a prop modifier that hides the no-op:
(* In Dom_html.Prop *)
let option prop = function
  | Some value -> prop value
  | None -> any "" ""
(* val option : ('a -> prop) -> 'a option-> prop *)

(* In tests *)
let%component make ~href:href_ =
  React.Dom.Html.(a [|option href href_|] [])

And inspires me to make a further refinement! What if instead we define option to be a prop modifier that hides the no-op

Looks great! And it avoids the need to create empty arrays just for this purpose ✨

What do you think about option vs maybe for the name?
let%component make ~href:href_ =
  React.Dom.Html.(a [|maybe href href_|] [])

I think maybe reads better, and often use it to name optional variables too since maybe_value reads better than value_opt or something. Then again, option might be more suggestive of this modifying the prop to take an option value.

Or optional...

I like maybe! 🙂 option is really overloaded already...

Made some further refinements, including a breaking change to the Context API to align it with rescript-react and allow both Reason and OCaml-style components to be created.
The last thing puzzling me is the forwardRef API, which I can't even understand the purpose of.
Apart from that, I think the only thing that remains is a proper interface.

Interfaces done

Make ppx changes so that instead of translating lowercase components to calls to createElement we just leave calls to div, p and such (user should put them in scope)

What's the reason for this? To aid migration?

Migrate types somehow from Html module in ppx to new Dom_html

What are these really used for? Outside of the PPX itself I mean.

What's the reason for this? To aid migration?

The main reason is that, if we do:

the above (make Reason ppx use the new DSL)
fix locations so they make sense to merlin (this is complex task, but as soon as I can find some time I'd like to tackle it because it's crucial)

then we get all docs and editor tooling upsides for free. You can hover over <div ..> and merlin will show the docs of the div function in the dsl, etc etc. I am not sure yet we can make everything work (props etc), but I am pretty sure it's a much clearer path than that we were going down before (using ppx to do this stuff).

What are these really used for? Outside of the PPX itself I mean.

They are used for the ppx transformation only, yes. But that plays a fundamental role at the moment on the lowercase component type checking. The ppx will generate type annotations for each prop, so you get compiler tell you about props with wrong types. But again, we get all this "for free" when we generate calls to DSL functions 🎉

...then we get all docs and editor tooling upsides for free

I see. Yeah I think that makes sense. But we'd also get that even if we fully qualify the functions, and that would hopefully make it a seamless transition.

But again, we get all this "for free" when we generate calls to DSL functions :tada:

:tada:

Thanks for the work on this PR @glennsl. As there were conflicts with main I solved them and merged manually.

But we'd also get that even if we fully qualify the functions, won't we, and that would hopefully make it a seamless transition?

I am not sure I get this part, could you elaborate?

re: the tests (keeping both Reason and OCaml) I am not sure it makes a lot of sense once we move to using the dsl for everything. Reason syntax should already be covered by ppx tests, so there's not much advantage on keeping 2 files for the "end to end" tests.

I am not sure I get this part, could you elaborate?

To quote what you said originally:

Make ppx changes so that instead of translating lowercase components to calls to createElement we just leave calls to div, p and such (user should put them in scope)

You've explained well the reason for using the new DSL instead of createElementdirectly, but not why the user should put the DSL in scope.
It seemed to me that fully qualifying all the DSL calls (e.g. React.Dom.Dsl.Html.div) would make the transition seamless. But now that I've thought it through a bit more, I realize this will only work for known HTML elements. It wouldn't work for SVG or anything else, unless we have some convention for "escaping" the HTML scope.
I also realize another problem with using the DSL instead of createElement directly, that it would no longer support custom elements and custom props. Although JSX doesn't support actual custom elements anyway, since they should contain a dash (-) which isn't a valid character in an identifier. But not supporting custom props is a bit of a bummer.

Another question on your suggestions for next steps:

Add docs and make sure they work with ppx locations 😬

What kind of docs, and level of detail, do you imagine this should be?


Update exportify to not necessarily assume a list of IDs

The specific use-case is to be able to read features from STDIN the way Python3 code does:

https://github.com/whosonfirst/py-mapzen-whosonfirst-export/blob/master/scripts/wof-exportify#L53-L66

Details here:

https://github.com/whosonfirst/go-whosonfirst-exportify/issues/3

Blocked on:

https://github.com/whosonfirst/go-reader/issues/2

Done, as of v0.8.0


Add PyQt5 tests to Linux and OSX CI

Description
sorry about missing that API difference in #866 !  Here's an update to our tests that will also test PyQt5 in Linux and OSX.  It fails on the current master until we merge @kne42's fix.
until it finishes testing here, here's an example failed test on cirrus:
https://cirrus-ci.com/task/4594543160983552
thoughts @kne42 ?
Type of change

[x] update to our CI

References
closes #768

Looks good to me too. Would be good to get #866 merged into master and then into this PR and re-run the tests to make sure they all pass before we merge this one.

yeah I agree.  I'll merge #866 and pull into this

this seems to be working now... but the osx tests are hung waiting to run due to concurrency limits.

@tlambert03 just so I understand this for each of our 6 configurations, (python 3.6, 3.7 across 3xOS) we will now in serial run all our tests first with pyside2 and then with pyqt5 following an uninstall of pyside2? That means all of our model tests which are Qt independent will run twice, which I think is fine, they are fast, and then all the Qt stuff will be tested once with PySide2 and once with PyQt5?

I guess this now needs master to check that it's passing?

that's mostly correct except it's currently only testing PyQt5 on linux and osx (which are usually done before windows anyway), and not testing pyqt5 on windows.  I figured the serial approach is probably better than parallel, since the environment setup is a significant part of the task time, and concurrency limits at cirrus prevent us from having that many parallel tasks.

I guess this now needs master to check that it's passing?

yeah, we need to merge the zarr changes here to test them

also, it would be trivial to only run pytest napari/_qt for PyQt5 testing if you prefer to limit that second test to Qt specific stuff

only run pytest napari/_qt for PyQt5 testing if you prefer to limit that second test to Qt specific stuff

Clever! I think that's a very good idea!
I'm ok with only testing PyQt5 on some platforms. Ultimately, chasing a complete test matrix is impossible, and we should just get "as good as reasonably possible".

ok, merged the master zarr changes, and change pyqt5 to only test napari/_qt.  will see how it goes here

looking good!  merge?

wait, napari/test also contains lots of Qt tests too

yeah, good point.  since it really doesn't add much time at all (maybe a couple minutes, and we're generally waiting on windows anyway during those minutes), let's just keep it testing the whole thing until we decide we need it to be faster.

PyQt5 on linux and osx (which are usually done before windows anyway), and not testing pyqt5 on windows.

Just so I understand, PySide2 is tested on all 6, and PyQt5 is now tested on four, everything apart from windows. Is there a reason we're not testing PyQt5 on windows? I agree with @jni might not be needed for complete the full matrix, but just curious why this choice.
Also rerunning PyQt5 tests only on tests with Qt stuff makes sense, but we actually have three places right now where that happens napari/_qt, napari/_vispy, and napari/tests so all three of those should be included before merge

Ok, testing whole repo is fine, the model tests are very fast

correct.  the rationale was that we're almost always waiting on the windows tests, so it wouldn't really add any time to put it on linux and osx only... we would still catch all api differences (and I don't know what sorts of things are likely to be platform and qt backend specific, not that can't exist).  i don't have a strong opinion though...

waiting on the windows tests

agreed but I think most of that is waiting for the machines to be available / things to kick off, but then once it gets running I don't think our tests are that long such that it adds much more time. I'd probably prefer to just have the duplication and know everything is tested everywhere rather then be unpleasantly surprised about Qt binding and platform specific differences.
I also don't find myself waiting on tests to actually make a merge very often (i.e. I'll do something else after that final push with all the approvals in place) and think the extra time may help our culture of trying to give everyone a chance to weigh in before merge. Maybe it makes development a little slower if you don't see problem until much later. I don't have strong feelings though, so if it makes your life better to have it this way fine to leave too

doesn't affect my life either way :) so if there's even a slight preference, let's go with the thorough testing and revisit later if we change our minds.  will update

Great, thanks! @tlambert03 you can merge once all the tests pass :-)

well that's interesting!  both of the windows pyqt5 had failures in one of the animation tests, and they both seemed to have crashed in the screenshot test.  let me try to reproduce those locally to see what's going on

well, I was excited when I thought maybe it had caught a new bug, but now I think this has more to do with virtualization and the CI environment.  I am unable to reproduce this on a real windows system with pyqt5 (or pyside2).  the screenshots always work, (all tests do).
but the screenshot tests are consistently failing on CI, with a stack overflow (Exit status:<PHONE_NUMBER>)... so I'm not sure what to do about it.  and not sure why it's only pyqt5.
signing off for the night though...  feel free to look into it, push to the PR, and/or tell me what you want to do tomorrow

Hmm ok, I think we can wait to tomorrow to look into this. Maybe @ttung, @shanaxel42, or @kne42 might have an idea

I played around with this today and i'm pretty sure now these are just the same issues we've always had with windows testing, it just appears that pyqt5 and pyside2 fail in different ways:

If I test on a remote windows computer using remote desktop, I get the same stack overflow crash that we see on CI, during the same tests
If I test on windows with RDP using pyside2, it doesn't crash, but all of the tests fail (with that old LooseVersion error message) unless I add that -o qt_no_exception_capture=1 that we have on there.  So either way, testing gl-dependent stuff on a virtualized windows machine isn't working.  It's just a hard crash on PyQt5 and a soft crash on Pyside2

given that this is really just a different manifestation of an old (and bigger) issue, I propose we roll back to just testing PyQt5 on linux and mac as this began.

ahhh yes, makes sense - let's roll back and then and get this in

Ok, we’re back to the minimal change that @kne42 approved and I think @jni was ok with as well.  It will at least catch api differences between the two backends. Merging


Give exception to user on pipeline __init__() when stride and window_size are incompatible

Feature request
This is tricky because we need to know in advance how many special tokens the tokenizer will add to each window. This may be impossible
@wigwit will experiment with figuring out how valid max_length and stride is validated when running the GPT tokenizer
Motivation
It would be nice to give users an error message as early as possible
Your contribution
@wigwit will work on this for now.

I tested GPT2 tokenizer and how it would perform with the stride argument. There are two differences:

GPT2 does not have special tokens and padding tokens. Therefore, it does not make a difference for gpt2 whether to set adding_special_tokens =True or False.  This might incur inconsistency when averaging the logits since BERT has [CLS] and [SEP] at the beginning and end of the sentence.
GPT2 would require the user to explicitly assign a padding token since the tokenizer it self does not contain one. Other than that, the stride feature functions the same in GPT2 as other BERT based tokenizers.

After reviewing the updated code I think (1) should not be a problem as we used special_token_mask to filter out the duplicated [CLS] and [SEP], and it should not mess with GPT2.
For (2), I am thinking if we could hardcode a pad token, or maybe there will a workaround on that. Here are the list of special tokens for GPT2 and BERT for reference:
GP2:
            {'bos_token': '<|endoftext|>',
             'eos_token': '<|endoftext|>',
             'unk_token': '<|endoftext|>'}

BERT:
            {'unk_token': '[UNK]',
             'sep_token': '[SEP]',
             'pad_token': '[PAD]',
             'cls_token': '[CLS]',
             'mask_token': '[MASK]'}

So, let's say I want to make a BERT-based token classification pipeline:
token_classifier = pipeline('token-classification-sliding-window', model='dslim/bert-base-NER', aggregation_strategy='FIRST', window_length=5, stride=3)

right now, our code lets me do this. But when I try to use the token_classifier I'll realize that this is an invalid combination of window_length and stride, because the BERT tokenizer inputs two special tokens for each window. Therefore if I run this:
token_classifier("Hello, world, my name is Connor Boyle and I am working on this project with Qingxia Guo")

I get a big stacktrace ending with this error message:
PanicException: assertion failed: stride < max_len

I want to get a warning/error message telling me not to do this, already when I try to initialize a pipeline with a bad combination of window_length and stride.

@wigwit Note that this varies per model, since GPT-based tokenizers (as you helpfully confirmed) don't seem to add any special tokens, so if I did the same thing with a GPT-based model:
token_classifier = pipeline('token-classification-sliding-window', model='nguyenkhoa2407/gpt2-NER-favsbot', aggregation_strategy='FIRST', window_length=5, stride=3)

then I can actually run this just fine:
token_classifier("Hello, world, my name is Connor Boyle and I am working on this project with Qingxia Guo")

Which outputs:
[{'entity_group': 'LABEL_3', 'score': 0.3464192765350073, 'word': ' Hello,', 'start': 0, 'end': 6}, {'entity_group': 'LABEL_0', 'score': 0.40267269932127137, 'word': ' world, my name is', 'start': 6, 'end': 24}, {'entity_group': 'LABEL_3', 'score': 0.2950232108138331, 'word': ' Connor Boyle', 'start': 24, 'end': 37}, {'entity_group': 'LABEL_0', 'score': 0.4543407750912735, 'word': ' and I am working on this project with Qingxia', 'start': 37, 'end': 83}, {'entity_group': 'LABEL_4', 'score': 0.37418330750861495, 'word': ' Guo', 'start': 83, 'end': 87}]


[1.3] Use build stages in .travis.yml

Q
A

Type
improvement

BC Break
no

Fixed issues

Summary

PHP 5.6 will be tested as part of the build against lowest dependencies
Tests the currently supported MongoDB versions (3.4, 3.6, 4.0)
Adds PHP 7.3 and 7.4snapshot (with allowed failures) to the build matrix

This change allows us to add more build stages for 1.3, e.g. the upcoming performance test suite 🎉

FWIW something seems to be off with sharding

Yep, need to investigate, just haven't gotten to it. Setting it up locally is always a pain 😉


SentimentResponse is missing SentimentBatchResult type

Package Name: @azure/cognitiveservices-textanalytics
Package Version: 4.0.0
Operating system: Mac OS
[x] nodejs

version: v12.13.0

[x] typescript

version: v3.3.3

Describe the bug
SentimentResponse type is missing the SentimentBatchResult type like the other types have.
The SentimentResponse type:
https://github.com/Azure/azure-sdk-for-js/blob/878dfff792a0477b4c07df377ed0389433ff3b57/sdk/cognitiveservices/cognitiveservices-textanalytics/src/models/index.ts#L502
The DetectLanguageResponse type:
https://github.com/Azure/azure-sdk-for-js/blob/878dfff792a0477b4c07df377ed0389433ff3b57/sdk/cognitiveservices/cognitiveservices-textanalytics/src/models/index.ts#L442
The SentimentBatchResult type already exists - just needs to be added to the SentimentResponse type definition.
Should I submit a pr?

The text analytics library is currently auto-generated based on a swagger spec.
@HanLiMS looping you in here as you had authored the last update to the swagger spec for Text Analytics. Can you shed some light on this issue when using v2 apis for Text Analytics?

The text analytics library is currently auto-generated based on a swagger spec.
@HanLiMS looping you in here as you had authored the last update to the swagger spec for Text Analytics. Can you shed some light on this issue when using v2 apis for Text Analytics?

@ramya-rao-a Ramya Rao Which version are we talking here?
For V2.0/V2.1, SentimentBatchResult should be there.
For V3.0-Preview, we only have SentimentResponse.

@HanLiMS The current package uses v2.1 api and was last regenerated on Sep 11, 2019. See PR #4955.
When was the last time the swagger for this version updated?
@joheredi, can we regenerate this package and see if there is any difference?

I regenerated the package against the stable version at the start of my track 2 work and didn't see any differences.
I see SentimentBatchResult defined in the package and used as the body mapper, but @Ethan-Arrowood is correct that SentimentResponse is not written correctly. I'm not sure why, maybe a bug in autorest? I keep looking at the swagger and it looks correct: https://github.com/Azure/azure-rest-api-specs/blob/master/specification/cognitiveservices/data-plane/TextAnalytics/stable/v2.1/TextAnalytics.json

@HanLiMS The current package uses v2.1 api and was last regenerated on Sep 11, 2019. See PR #4955.
When was the last time the swagger for this version updated?
@joheredi, can we regenerate this package and see if there is any difference?

@ramya-rao-a The last time v2.1 version got updated was Feb 2019.
https://github.com/Azure/azure-rest-api-specs/commit/5df40b9b08b77fe7b78f90f6b7c470600b8ea063#diff-ca45a0e48722614fce123ceea154351a

Thanks @xirzec and @HanLiMS
@joheredi This will need us to debug the code generator to see if we are missing something

@HanLiMs looking at the swagger file, I realized that /sentiment is the only operation that doesn't define a "default" response for the error case, instead, it defines it as "500" . This seems to be causing autorest to handle SentimentResponse differently (I'm still investigating why). I verified that having the error case defined as "default" generates the expected SentientResponse type.
Also looking at version 2.0, /sentiment defines "default", was this change intentional?

@HanLiMS looking at the swagger file, I realized that /sentiment is the only operation that doesn't define a "default" response for the error case, instead, it defines it as "500" . This seems to be causing autorest to handle SentimentResponse differently (I'm still investigating why). I verified that having the error case defined as "default" generates the expected SentientResponse type.
Also looking at version 2.0, /sentiment defines "default", was this change intentional?

That's interesting. No I don't think so. Let me follow up with my team member who made this change. By the way, if that is the case, we can update the swagger to define "default" as needed.

Thanks, @HanLiMS! If you update the swagger switching 500 -> default SentimentBatchResult would be picked up by autorest
For reference, we have an issue in autorest which describes how response errors are handled and some suggested improvements we can make to handle specific errors here /cc @ramya-rao-a

Thanks, @HanLiMS! If you update the swagger switching 500 -> default SentimentBatchResult would be picked up by autorest
For reference, we have an issue in autorest which describes how response errors are handled and some suggested improvements we can make to handle specific errors here /cc @ramya-rao-a

Hi @joheredi @ramya-rao-a  , I just sent out a PR for a quick fix of this: https://github.com/Azure/azure-rest-api-specs/pull/7970. Could you take a review and let me know if there is more needed? Thanks!

@Ethan-Arrowood To give you an update on this issue, we figured out and fixed the root cause, but since we're in the middle of releasing a rewrite of the text analytics library, I'm not sure if we want to release a new version of the old package before the holidays.
As you mentioned, the API shape is in the package, just not connected to the result, so you should be able to work around this in your code with something like the following:
const result = (await client.sentiment(options)) as TextAnalyticsModels.SentimentBatchResult;

  console.log("The result is:");
  for (const document of result.documents!) {
    console.log(`Id: ${document.id}`);
    console.log(`Sentiment score: ${document.score}`);
  }

Does that unblock you for now until we can get a new package out?

Thanks for the update @xirzec
Yes I used something similar as a work around. I'm glad I was able to point this out to y'all so it can be fixed for other users.
Thank you and have a happy holidays 😁☃

Thanks everyone!
Closing this issue as there are no action items here for the current version of the package.
@Ethan-Arrowood You check for announcements at https://azure.github.io/azure-sdk/ for when we have a new version of this package ready for a preview.