[BUG] Unable to load weights of embeds_token_info module of the pre-trained model in the case of multiformer

Describe the bug
https://github.com/clessig/atmorep/blob/3efcb122842b7cd285cb3074f84b98ef9665a306/atmorep/core/atmorep_model.py#L234-L259
Whenever my loading weights from a pre-trained multiformer model, the loading of weights fail
To Reproduce
Steps to reproduce the behavior:

Activate '...'
Change '....'
Run : Whenever model.create function is used i.e. with cf.fields for each variable containing the model_id in fields[1][4][0] variable
See error

0:   mloaded = torch.load( get_model_filename( name, field_info[1][4][0], field_info[1][4][1]))
0: Traceback (most recent call last):
0:   File "/p/project1/deepacf/atmo-rep/patnala1/atmorep/atmorep_downscaling_train_10699250/train.py", line 215, in <module>
0:     train()
0:   File "/p/project1/deepacf/atmo-rep/patnala1/atmorep/atmorep_downscaling_train_10699250/train.py", line 208, in train
0:     trainer = Trainer_Downscaling( cf, device).create()
0:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0:   File "/p/project1/deepacf/atmo-rep/patnala1/atmorep/atmorep/applications/downscaling/trainer.py", line 61, in create
0:     self.model.create( self.devices)
0:   File "/p/project1/deepacf/atmo-rep/patnala1/atmorep/atmorep/applications/downscaling/atmorep_downscaling.py", line 81, in create
0:     self.net = self.net.create( devices, load_pretrained)
0:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0:   File "/p/project1/deepacf/atmo-rep/patnala1/atmorep/atmorep/applications/downscaling/atmorep_downscaling.py", line 153, in create
0:     self.backbone = AtmoRep(cf).create(devices, load_pretrained=load_pretrained)
0:                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0:   File "/p/project1/deepacf/atmo-rep/patnala1/atmorep/atmorep/core/atmorep_model.py", line 254, in create
0:     self.embeds_token_info[-1].load_state_dict( mloaded)
0:   File "/p/project/deepacf/atmo-rep/patnala1/atmorep/jsc_scripts/virtual_envs/venv_jwb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2215, in load_state_dict
0:     raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
0: RuntimeError: Error(s) in loading state_dict for Linear:
0: 	Unexpected key(s) in state_dict: "1.weight", "1.bias", "2.weight", "2.bias", "3.weight", "3.bias", "4.weight", "4.bias", "5.weight", "5.bias".
0: [rank0]:[W1218 15:56:07.722048163 ProcessGroupNCCL.cpp:1168] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present,  but this warning has only been added since PyTorch 2.4 (function operator())

Expected behavior
It should load pretrained weights of embeds_token_info corresponding to all the fields ( 6 in this case)
Screenshots/Explanations
if you see line 249 and line 250,
https://github.com/clessig/atmorep/blob/3efcb122842b7cd285cb3074f84b98ef9665a306/atmorep/core/atmorep_model.py#L249-L250
'0' is being used which refers to first field only.
https://github.com/clessig/atmorep/blob/3efcb122842b7cd285cb3074f84b98ef9665a306/atmorep/core/atmorep_model.py#L251-L252
Once it is deleted, it is not able to find again.
The current code will only work if fieldinfo[0][1][4][0] contains model_id of singleformer.
Hardware and environment:

System: JSC
Runtime: sbatch
Special branch: branch corresponds to  issue#76 ankitpatnala/ankit_downscaling/issue_076-feature-development-of-perceiver-model-for-downscaling
Python Version: [e.g. 3.11]

Additional context
I am trying to load a pre-trained multiformer for downscaling. So, I replaced the fieldinfo[1][4][0] to the model_id corresponding to the multiformer.
Note: We do not need multiple embed_token_info for each field as the 8 variables of token_infos remains same for all the field variable.

Yes, the limitation with that you have to have singleformers for the embedding nets is true. But you have these, not?

Yes! I have them but I want to make use of updated weights of multiformer, the same id will be used for updated weights of encoders and decoder modules too.

From the singleformer only the embedding net weights should be loaded so this shouldn't be an issue.

Hi, is the multiformer you are trying to load one of the old ones from the paper or the new ones I gave you a few weeks ago? Because in the second case it should work out of the box, I call continue_training with no problem and it uses the same functions.
With the old runs is a bit more tricky as they were also trained with a different precision and no flash attention, right? Isn't maybe easier if you just pull an older version of the code, like tag v0.0?

I use the new one i.e. the second case. It works maybe for two reasons
-> In the json file, the model_ids are still from singleformers.
-> Continue training, simple loads from the model and does not use this nlock of code.

   self.embeds_token_info = torch.nn.ModuleList()
    for ifield, field_info in enumerate( cf.fields) :

      self.embeds_token_info.append( torch.nn.Linear( cf.size_token_info, cf.size_token_info_net))

      if not load_pretrained :
          # initalization
          torch.nn.init.constant_( self.embeds_token_info[-1].weight, 0.0)
          self.embeds_token_info[-1].bias.data.fill_(0.0)

    if load_pretrained:
        name = 'AtmoRep' + '_embeds_token_info'
        if not os.path.exists(get_model_filename( name, cf.fields[0][1][4][0], cf.fields[0][1][4][1])):
            name = 'AtmoRep' + '_embed_token_info'

        mloaded = torch.load( get_model_filename( name, cf.fields[0][1][4][0], cf.fields[0][1][4][1]))
        self.embeds_token_info.load_state_dict( mloaded)
        print( 'Loaded embed_token_info from id = {}.'.format( cf.fields[0][1][4][0] ) )

I wrote this line of code to make it work for my current case. Later realized this would not work for singleformers case. All we can do is give a check number of key in the embeds_token_info module. If it is twice the number of fields, we can be certain this is the model_id of the multiformer.  and use fidx instead of 0 else when it is not multiformer, we can use 0 as it is currently now.


Placeholders level/experience

Hey, I would like to ask is there any option to display the level, experience of a player in a given job regardless of whether he is in this job?

    - '  &#C6AB79&lˣ &7Poziom: %jobsr_user_jlevel_digger%   %jobsr_user_jmaxlvl_digger%'
    - '  &#C6AB79&lˣ &7Doświadczenie:   %jobsr_user_jexp_digger%   %jobsr_user_jmaxexp_digger%'

    I have something like that now

It is possible that with other placeholders I can do something like this, but despite the combination I could not get this effect

By default we are showing 0 for level and experience if player isint part of defined job. Can you elaborate more on issue you are having here?

Is it possible to show just the data of the jobs? even to the players is not in the job?


fix(identity-trust): disable spring security for identityMinusTrust endpoint

Description
disable spring security for identityMinusTrust endpoint
Pre-review checks

[X] DEPENDENCIES are up-to-date. Dash license tool. Committers can open IP issues for restricted libs.
[X] Copyright and license header are present on all affected files

Dominik Pinsel<EMAIL_ADDRESS>Mercedes-Benz Tech Innovation GmbH, legal info/Impressum

@DominikPinsel
Please rebase and resolve conflicts

@nitin-vavdiya Done :-)


Dmatrix optimization2

Observed #5530 slowdown in DMatrix creation was fixed.
Performance improvements:

DMatrix create time,s
Master
Optimized
Speed up
Optimized without specialization
Speed up

airline-ohe contiguous numpy
6.6
1.8
3.67
1.99
3.32

msrank10k
3.9
1.16
3.36
1.61
2.42

mortgage1Q
4.06
1.1
3.69
2.93
1.39

Can we close this one?


Empty string should be treated as if "no value provided" from the validation point of view

What happened:
With the actual implementation we are able to build object with properties populated by empty string (without any validation error)
{
    "name": "",
    "inputCollection": "",
    "iterationParam": "",
    "workflowId": "",
    "end": true,
    "type": "foreach",
    "usedForCompensation": false
}

What you expected to happen:
How to reproduce it:
The following code should pass, instead is failing
import {foreachstateBuilder} from '../../../src';

describe('xxx', () => {

	it('xxxxx', () => {

		expect(() => foreachstateBuilder()
			.name('')
			.inputCollection('')
			.iterationParam('')
			.workflowId('')
			.end(true)
			.build()).toThrowError();

	});
});

Anything else we need to know?:
Environment:

@antmendoza  It's unfortunatly "a feature, not a bug". required just means that the property must exist on the object, it doesn't restrict the value:
6.5.3.  required
...
   An object instance is valid against this keyword if every item in the
   array is the name of a property in the instance.

https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-6.5.3
I guess we should add a constraint in the schema such as minLength.

Hi @JBBianchi it is not unfortunate, it is as it is... thank you.
I am going to create a pr for the specification..
Closing this.


Combine sass (or less)?

Hello,
The docs speaks of the mix.combine() method, that allows to combine files. But it would seems that it only combine files that are already compiled (I might be wrong, if I am, sorry!).
With Elixir, I could combine multiple sass and less files in one big compiled css file like so:
mix.sass([
        'app.scss',
        'controllers.scss'
    ], 'public/assets/css');

Trying the same with laravel-mix results in an error Path must be a string, so that's clearely not working. Is it impossible at the moment to do so? Or did I miss something entirely?

You should have your main entrypoint Sass file import any other files that it needs to be combined with.

It... Totally makes sense. Thank you for that, I don't really know why I stuck with what I did before actually ! :D

me too i wasn't smart..
thanks @JeffreyWay you're my Hero


Actually listen to new directories with the file change section

Fixed in https://github.com/SimonBiggs/scriptedforms/commit/0c8ea66307c3d565f6bef3a9c551a090685a7e23


Add $update method to non-destructively modify data

It could be that I'm misunderstanding the docs, but it seems like if I have
user = { name: 'nick', github: 'ncammarata' } ($firebase object)

and I use
user.$set({ name: 'nick c' });

The github property is now gone.
How would I set and save only a single property? Right now I'm manually $seting and $saveing multiple properties, which is tedious.

is this anywhere in the docs?
Btw, great talk at ng-conf!

Thanks for the compliment! Glad you are giving AngularFire a try.
This issue is over 14 months old and the API has undergone two large revisions since. You can find the current docs here:

Quickstart
Guide
API Reference

Thanks for the quick response! At least one of those revisions has happened since I started this side project on nights and weekends months and months ago, so I think it's back to the drawing board.

I think you will find the recently-released 1.0.0 API to be much easier to use! Feel free to open a new issue or post to the AngularFire Google Group if you have questions.

@jwngr @anantn I am using Angularfire v1.1.3 and there is no $update() method for $firebaseObjects. Will that be coming soon? It really would help when trying to modify a user account without destroying the whole thing and recreating it!

Or is there any other way to  $save non-destructively?

Not supported. You're probably looking for $firebaseArray.


Middleware - Hawkular Server - Shown as Mutable

Hawkular Server in latest MiQ is shown as Mutable, i.e. Power and etc.. toolbar operations are visible.
However their execution is failing (which is correct).
Expected behavior: They should be hidden, as Hawkular Server is Immutable.
MiQ Version: master.20170428012901
@abonas

@miq-bot add_label providers/hawkular

@hhovsepy it seems it's an old issue that in the meantime was fixed. is this still reproducible?

issue is fixed, closing it


Release 27.0.0

BREAKING: Rename extra_links to extra_details (image card) (PR #2300)
Create devolved nations component (PR #2280)
Fix document list children (PR #2296)
Reset margins for govspeak images (PR #2297)
Fix global banner wrapper in public layout (PR #2294)

👍
Can you also update the PR title please?

Thanks, yep, that's all updated now


Fix feature flag use for new Seed::scalar API

I can't push to your PR, thus I have to create a PR against your PR :(

@cloudhead I think you need to merge it since it is against your branch


hubblestack_nova directory structure design

Wanted to put some more thought and discussion into the directory structure of the hubblestack_nova folder. Should we organize sub-directories by module (so that we can include a rendered README)? Should profiles be collected into sub-directories for organization? etc.

I do think directories would be helpful.  if the .py and .yaml files are all specific to one distribution (e.g. CentOS Linux-7), then grouping them by distro would be very useful.  What's a little confusing right now is having the CIS.yaml file and the pkg.yaml file.  They both have top-level 'pkg' keys.  If they have a different or conflicting whitelist or blacklist of files (e.g. pkgA in the whitelist for pkg.yaml, but in the blacklist for CIS.yaml), which file takes precedence?

@dgmorrisjr, pkg.py will search for the pkg top-level key in each yaml file found in the hubblestack_nova directory and it will merge all the data in a single dictionary.
For a better understanding of the process, see _get_yaml and _merge_yaml functions. You can find them in any of the nova plugins.

Thinking about this more after editing the CIS.yaml file for a bit.   Many organizations have their systems/servers designated with criticality levels based on the sensitivity of the information they contain/process.  As a result of this, a low-sensitivity set of hosts (e.g. user workstations) may need to be scanned against a profile (e.g. CIS-low.yaml), wheres a high-sensitivity set of hosts (e.g. Servers that support PCI, HIPPA, or R&D info/processes) may need to be scanned against a different profile (e.g. CIS-high.yaml).   Another instance of this is when using the DISA STIGS.  They evaluate controls associated with Confidentiality, Integrity, and Availability.  Within each of these three categories, they have three criticality levels: High, Medium, and Low.   Systems are then rated with their criticality for each control group, for instance, a HML system, would be High Confidentiality, Medium Integrity, and Low Availability.   Each of the criticality levels specifies a different set of "baseline controls".   Therefore, one system in an enterprise may be categorized as a HMM system, whereas another system within the enterprise might be MML.  Lastly, in addition to these differing levels of baseline controls, NIST 800-53 also provides for "control overlays" which allows for further hardening/strengthening of special purpose systems.  For example, they have control overlays for systems that process PII (Personally Identifiable Information) and other categories of special purpose systems.
So then, if at the hubble.nova command line, if we could have an argument such as subfolder=stig-hmm this would instruct the hubble.nova to restrict the import of .yaml files to the names sub-folder.   These sub-folders could be arbitrarily named so that each enterprise can tailor how they organize/customize their different files.
The last question that I would have here is regarding the .py files in the hubblestack-nova folder.  if there were to be multiple sub-folders for the various .yaml profiles, would there just be one set of .py files that live in the top-level hubblestack-nova folder?  or should the .py files live in a sub-folder under the _modules directory (e.g. _modules/nova/)

@cedwards and I talked at length about these issues today. I think we're going to make it official that all nova modules will require accompanying yaml. For checks that don't actually require configuration (like the pkgng_audit.py), the yaml will basically just be a top-level key.
We're then going to move the yaml loading into hubble.py and add the ability to target by yaml file using hubble.audit.
All of these changes are building blocks to get us to the biggest change, which is the introduction of a topfile-like structure for targeting audits to minions. I plan to steal the topfile-parsing code so we can support all the match types. Basically, you'll be able to add audits to minions via yaml filename, with an optional tag filter.
Relevant issues: #68 #69

how will that structure then allow for different profiles of inspections, such as CIS Level 1 vs CIS level 2, or STIG HML vs HMM vs MHL?  We you add additional Tags in the yaml for that further categorization?  I also think that this structure will be more difficult for review by security evaluators.  If the yaml files are organized in a way that is more consistent with they way the security standards are organized, it's easier for validation of coverage, etc...  it's organizing the profiles based on security users perspectives vs sw utility perspective.

I had a discussion with @dgmorrisjr in Slack today. I think we're on the same page now. I'd like to see something like benchmark-osfinger-level-(un)scored.yaml. I think this aligns with how we want to be able to target these using the top-file idea.

Initial work toward this restructuring is in #73.

Restructure: https://github.com/HubbleStack/Nova/pull/184


Update broken links

Description
Update broken links
Issues Resolved
List any issues this PR will resolve, e.g. Closes [...].
Checklist

[X] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Updated in #7030


android GpuDelegate: even the simplest tflite converted models seems to run on CPU

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): tf docker images
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: oneplus5, pixel3, galaxy s10
TensorFlow installed from (source or binary): various
TensorFlow version (use command below): 1.12.0, 1.13.1, 1.14.0, 1.15.0-dev20190628, r1.14 source build
Python version: 3.x
Bazel version (if compiling from source): 0.25.2
GCC/Compiler version (if compiling from source): 7.4.0
CUDA/cuDNN version:
GPU model and memory:

Describe the current behavior
tflite converted models seems to always run on cpu despite using gpudelegate, for example the simplest model from https://github.com/tensorflow/tensorflow/issues/30311 runs for hundreds of milliseconds per inference no matter whether cpu or gpudelegate is used, while stock mobilenet_v1 (no quantization, no manual tflite conversion, downloaded tflite model) runs ~80ms per inference on this device when using gpudelegate and 150-300ms on cpu.
Describe the expected behavior
tflite to actually use GPU when running conv2d networks
Code to reproduce the issue
attached NHW3 model (and script to generate pb and tflite with optimization options, which do not have any effect), this is basically NHW3 input, single tf.layers.conv2d() constant-initialized layer and reduce_sum() for output.
Any hint on debugging this issue?
I tested both jcenter downloaded (0.0.0-nightly branches) and manually built from r1.14 branch tensorflow-lite and tensorflow-lite-gpu.
I used 1.12.0, 1.13.1, 1.14.0, 1.15.0-dev20190628, r1.14 source build tensorflow/tflite with different optimization options with no success.
Could you please show how exactly you generated mobilenet_v1 tflite model found in your tutorials, since it works like a charm with gpudelegate: ~80ms per single-image inference on gpu and several hundreds seconds on cpu on sufficiently recent android. Or any hints on how to debug this gpudelegate issue.
test_model_3channels.tar.gz

@bioothod
Thanks for attaching the model.

ERROR: Next operations are not supported by GPU delegate:
SUM: Operation is not supported.
First 1 operations will run on the GPU, and the remaining 1 on the CPU.

Well, we don't have a SUM operation, and MobileNet v1 doesn't have a SUM at the end either.  It's not clear where the confusion is coming from.  Can you elaborate?

@bioothod
Oh and a couple of things.

By default, TFLite gpu delegate options has allow_precision_loss to 0 (false), to be on the safe side.  You may want to turn that on for speed up.
Your network needs to be deep enough.  There is additional cost you have to eat up (memcpy, GPU/CPU sync etc.) that needs to be compensated by GPU's faster execution.  So if you have a network with just a single CONV_2D, of course it's going to be slower than on the CPU only.

https://www.tensorflow.org/lite/guide/ops_compatibility states that tf.reduce_sum will be removed from the graph, probably will be replaced with ADD_N operation. Or does it mean it should be removed by the user? I've attached different mobilenet_v1 which doesn't work as fast as google-converted tflite either, it doesn't have SUM operator.
Also. I'm quite sure I saw gpudelegate exception about unsupported SUM operator with some other models (do not remember, what it was though), and now there are none, model just works slow.
I used precision loss without any effect.
This conv2d network is slower than the whole mobilenet_v1 on cpu, well, its because of large convolutions, but still.
model.ckpt-1044627-frozen.tflite.gz

N.B. maybe not slower, after 100-400 ms variation is very high and I never measured mean and anyway it is unacceptably high for 2 operations or for the whole network like in the example above. And since there is no exception in gpudelegate log I suppose it runs on GPU and should be, well, not slower than google-converted mobilenet_v1

More confusion.
SUM operator works and produces correct results (maybe running on CPU - any chance to know that graph works on GPU?):
images = tf.placeholder(tf.float32, shape=[None, 32, 32, num_channels], name='input/tensor')
x = tf.layers.conv2d(images, 1, kernel_size=[3, 3], kernel_initializer=tf.ones_initializer(), use_bias=False)
sum_pixels = tf.reduce_sum(x, axis=[1, 2], name='output/sum_pixels')

test_conv2d_3channels_sum_works.tflite.gz
SUM operator crashes with unsupported op exception:
images = tf.placeholder(tf.float32, shape=[None, 32, 32, num_channels], name='input/tensor')
x = tf.layers.conv2d(images, 1, kernel_size=[32, 32], kernel_initializer=tf.ones_initializer(), use_bias=False)
sum_pixels = tf.reduce_sum(x, axis=[1, 2], name='output/sum_pixels')

test_conv2d_3channels_sum_crashes.tflite.gz
GpuDelegate crashes with:
I/Adreno: Error: Uniform offsets cannot find a suitable location/component.
    Error: Linking failed.
W/ImageReader_JNI: Unable to acquire a buffer item, very likely client tried to acquire more than maxImages buffers
E/AndroidRuntime: FATAL EXCEPTION: inference
    Process: org.tensorflow.lite.examples.classification, PID: 30676
    java.lang.IllegalArgumentException: Internal error: Failed to apply delegate: TfLiteGpuDelegate Prepare: Program is not properly linked: Error: Uniform offsets cannot find a suitable location/component.
    Error: Linking failed.Node number 2 (TfLiteGpuDelegate) failed to prepare.

        at org.tensorflow.lite.NativeInterpreterWrapper.applyDelegate(Native Method)
        at org.tensorflow.lite.NativeInterpreterWrapper.init(NativeInterpreterWrapper.java:83)
        at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:60)
        at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:224)

images = tf.placeholder(tf.float32, shape=[None, 32, 32, num_channels], name='input/tensor')
x = tf.layers.conv2d(images, 1, kernel_size=[32, 32], kernel_initializer=tf.ones_initializer(), use_bias=False)
sum_pixels = tf.reshape(x, [1, 1], name='output/sum_pixels')

test_conv2d_3channels_crash_uniform_offsets.tflite.gz

delegates/gpu/common/model_builder.cc has a function called NewOperationParser().  That's the set of operations supported by GPU.

delegates/gpu/common/model_builder.cc has a function called NewOperationParser(). That's the set of operations supported by GPU.

It is probably related to SUM operation. Well, then why does it work in the first graph? The only difference is input shape 1,1,1,1 vs 1,1,1,1

In all cases, SUM runs on the CPU.
What is succeeding on the first case and failing on the second case is CONV_2D on the GPU.

Things are getting more and more weird, isn't conv_2d supposed to always succeed, at least as it is written in documentation, or there are some constraints? Why doesn't delegate throw the same exception explaining (at least something)...
Are you on this issue and do you need any additional details from me, or you think everything is fine?
I will dig into delegate code in a meantime (ouch, there are no debug helpers) and try to find out more

Apparently, both failing models fail because GL can not compile conv_2d kernel for 32x32 reception field, which is very large for real life anyway, and for smaller kernels it silently replaces sum or squeeze wtih CPU implementations, just like you've said.
Still curious how to determine which operations run on gpu, since the only way to run fast enough is not to touch any property of tflite converter, even not setting converter.optimizations = [tf.lite.Optimize.DEFAULT]. Model runs fast with plain
converter = tf.lite.TFLiteConverter.from_session(sess, [images], output_nodes)
#converter.optimizations = [tf.lite.Optimize.DEFAULT]
#converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_LATENCY]
#converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
converter = tf.lite.TFLiteConverter.from_session(sess, [images], output_nodes)

If any optimization property has been touched to any commented above, model greatly shrinks in size, but starts executing only on CPU.

Again...
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/gpu/common/model_builder.cc#L1879-L1957
this is the list of supported ops on the GPU.
If it crashes, that's a bug, and we should look into it, but we cannot accommodate all hypothetical single op test cases.  After all, this is an open source project, and you can contribute by fixing bugs too ;)

Hi There,
We are checking to see if you still need help on this, as you are using an older version of tensorflow which is officially considered end of life . We recommend that you upgrade to the latest 2.x version and let us know if the issue still persists in newer versions. Please open a new issue for any help you need against 2.x, and we will get you the right help.
This issue will be closed automatically 7 days from now. If you still need help with this issue, please provide us with more information.


#VULCAN-274/ Group reports in a page and minimise reports functionality

Group Reports and Minimize Reports Functionality
Features

Group one or more reports and we can specify a border and border colour to those grouped reports.
Minimize button for reports.
Added an option in table report config called compact canvas. when the query status is NO_DATA the card height is set to default 210px.

Note: This implementation can be only be done through JSON for now. If you are okay with this implementation. Then I can create some component through which user can configure the group reports.
Sample JSON
{
    "title": "My dashboard",
    "version": "2.3",
    .......
    "pages": [
      {
        "title": "Main Page",
        "groups": [
          {
            "groupId": 1, // Group Id
            "border": "5px solid",
            "borderColor": "red"
          }
        ],
        "reports": [
          {
            "id": "322e3326-29f9-42d5-972b-e0b0b78ca581",
            "groupId": 1, // Group Id
            "groupOrder": 1, // Group order is used to achieve horizontal compaction
         .....
          },{
            "id": "483908d-29f9-42d5-972b-e0b0b78ca581",
            "groupId": 1, // Group Id
            "groupOrder": 2, // Group order is used to achieve horizontal compaction and it is incremental
         .....
          },
     ]
}

NOTICE
The program was tested solely for our own use cases, which might differ from yours.
Author Info: Monish<EMAIL_ADDRESS>on behalf of Mercedes-Benz Research and Development India
https://github.com/mercedes-benz/mercedes-benz-foss/blob/master/PROVIDER_INFORMATION.md
Sample JSON
sample-moview-json.txt
Signed off: Monish<EMAIL_ADDRESS>

Thank you for your work, could you please:

[ ] add cypress test for this functionality
[ ] attach a video of how it works?

@alfredorubin96 , Added a GIF for your reference. This feature helps us to group two or more reports in a page.
// This is a sample dashboard json payload

{
  "title": "My dashboard",
  "version": "2.4",
  "settings": {
    "pagenumber": 0,
    "editable": true,
    "fullscreenEnabled": false,
    "parameters": {},
    "theme": "light"
  },
  "pages": [
    {
      "title": "Main Page",
      "groups": [
        {
          "groupId": 1,
          "border": "5px solid",
          "borderColor": "red"
        }
      ],
      "reports": [
        {
          "id": "50bbc9f4-dbec-4e41-8471-62b5a091b883",
          "title": "Not a group report",
          "query": "MATCH (n)-[e]->(m) RETURN n,e,m LIMIT 20\n\n\n",
          "width": 6,
          "height": 4,
          "x": 0,
          "y": 4,
          "type": "graph",
          "selection": {
            "Person": "name",
            "ACTED_IN": "(label)",
            "Movie": "title",
            "DIRECTED": "(label)",
            "PRODUCED": "(label)"
          },
          "settings": {},
          "schema": [
            [
              "Person",
              "born",
              "name"
            ],
            [
              "ACTED_IN",
              "roles"
            ],
            [
              "Movie",
              "tagline",
              "title",
              "released"
            ],
            [
              "DIRECTED"
            ],
            [
              "PRODUCED"
            ]
          ]
        },
        {
          "id": "6060fb1d-978b-429a-a020-3de2634cc711",
          "title": "Some random report",
          "query": "MATCH (n)-[e]->(m) RETURN n,e,m LIMIT 20\n\n\n",
          "width": 6,
          "height": 4,
          "x": 6,
          "y": 4,
          "type": "graph",
          "selection": {
            "Person": "name",
            "ACTED_IN": "(label)",
            "Movie": "title",
            "DIRECTED": "(label)",
            "PRODUCED": "(label)"
          },
          "settings": {
            "minimizable": true
          },
          "schema": [
            [
              "Person",
              "born",
              "name"
            ],
            [
              "ACTED_IN",
              "roles"
            ],
            [
              "Movie",
              "tagline",
              "title",
              "released"
            ],
            [
              "DIRECTED"
            ],
            [
              "PRODUCED"
            ]
          ]
        },
        {
          "id": "6a68555a-f891-493b-851a-dc15a34d948d",
          "groupId": 2,
          "groupOrder": 1,
          "title": "Group report 2 - A",
          "query": "MATCH (p:Person)-[:ACTED_IN]->(m:Movie)\nRETURN p, m.title",
          "width": 6,
          "height": 4,
          "x": 12,
          "y": 0,
          "type": "graph",
          "selection": {
            "Person": "name"
          },
          "settings": {
            "minimizable": true
          },
          "schema": [
            [
              "Person",
              "born",
              "name"
            ]
          ]
        },
        {
          "id": "8e632236-445b-483e-9540-627d3c8d34b7",
          "groupId": 2,
          "groupOrder": 2,
          "title": "Group report 2 - B",
          "query": "MATCH (p:Person)-[:ACTED_IN]->(m:Movie)\nRETURN p, collect(m.title)\n\n",
          "width": 6,
          "height": 4,
          "x": 18,
          "y": 0,
          "type": "graph",
          "selection": {
            "Person": "name"
          },
          "settings": {
            "minimizable": true
          },
          "schema": [
            [
              "Person",
              "born",
              "name"
            ]
          ]
        },
        {
          "id": "322e3326-29f9-42d5-972b-e0b0b78ca581",
          "groupId": 1,
          "groupOrder": 1,
          "title": "Group report 1 - A",
          "query": "**This is your first dashboard!** \n \nYou can click (⋮) to edit this report, or add a new report to get started. You can run any Cypher query directly from each report and render data in a variety of formats. \n \nTip: try _renaming_ this report by editing the title text. You can also edit the dashboard header at the top of the screen.\n\n\n",
          "width": 6,
          "height": 4,
          "x": 0,
          "y": 0,
          "type": "text",
          "selection": {},
          "settings": {
            "minimizable": true
          }
        },
        {
          "id": "c753770c-e1f7-41a7-b4d2-ddc418fcefb8",
          "groupId": 1,
          "groupOrder": 2,
          "title": "Group report 1 - B",
          "query": "MATCH (n)-[e]->(m) RETURN n,e,m LIMIT 20\n\n\n",
          "width": 6,
          "height": 4,
          "x": 6,
          "y": 0,
          "type": "graph",
          "selection": {
            "Person": "name",
            "ACTED_IN": "(label)",
            "Movie": "title",
            "DIRECTED": "(label)",
            "PRODUCED": "(label)"
          },
          "settings": {
            "minimizable": true
          },
          "schema": [
            [
              "Person",
              "born",
              "name"
            ],
            [
              "ACTED_IN",
              "roles"
            ],
            [
              "Movie",
              "tagline",
              "title",
              "released"
            ],
            [
              "DIRECTED"
            ],
            [
              "PRODUCED"
            ]
          ]
        }
      ],
      "toolbox": []
    }
  ],
  "parameters": {},
  "extensions": {
    "active": true,
    "activeReducers": []
  },
  "uuid": "bc58c8bf-d023-4528-8fc3-335d0d6d2d96"
}

I have added a GIF for your reference. I have not implemented a settings for this. For now I am configuring it in JSON.
NOTICE
The program was tested solely for our own use cases, which might differ from yours.
Author Info: Monish<EMAIL_ADDRESS>on behalf of Mercedes-Benz Research and Development India
https://github.com/mercedes-benz/mercedes-benz-foss/blob/master/PROVIDER_INFORMATION.md
Sample JSON
sample-moview-json.txt
Signed off: Monish<EMAIL_ADDRESS>


Missing packages?

Hi,
Is there some threshold or rule for inclusion in the stats?  The package I'm looking for but can't find is conda-forge/arcticdb.
https://anaconda.org/conda-forge/arcticdb
The package page says 50k downloads but I can't find it in the monthly parquet files.
Thanks,

You are probably hitting #45. arcticdb never had .tar.bz2 artifacts.

Thanks @jaimergp.

Just posted a status update on https://github.com/ContinuumIO/anaconda-package-data/issues/45#issuecomment-1877517341

We're working on finalizing a new pipeline that will source this public data set and include .conda packages moving forward. We expect to have it ready by the end of March 2024 and will post an update here when it is available.

Ah, thank you.  That would explain it.  I'll close this.


Pre commit release

Summary of the changes / Why this is an improvement
Checklist

[ ] CLA is signed

Codecov Report

Merging #17 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master       #17   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           14        14
  Lines          635       635
  Branches        43        43
=========================================
  Hits           635       635

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 31163d5...f3f898e. Read the comment docs.


Scheduled CIS Scan option is disabled on the UI when a user edits a cluster

What kind of request is this (question/bug/enhancement/feature request): bug
Steps to reproduce (least amount of steps as possible):

Deploy a cluster through API
Wait for the cluster to come up successfully.
Do an Edit on the cluster
The Scheduled CIS Scan option is disabled on the UI

Expected Result:
Scheduled CIS Scan option should be enabled
Other details that may be helpful:
Note: - This is not always reproducible.
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): 2.4.5-rc6
Installation option (single install/HA): single

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported): custom RKE
Kubernetes version (use kubectl version):

1.18.3

The cluster in question was a windows cluster. CIS is disabled for windows clusters.


Added/updated features

Added:   custom prefix functionality, set a custom prefix to logs
Added:   SetOutput functionality, add optional io.Writer output
Updated: SetLogLevel parameters, accepts int/Level type

merging now.
a follow up PR can be done to add "SetDefaultPrefixer()" function


Spawning language server with cmd: rust-analyzer failed. The language server is either not installed, missing from PATH, or not executable.

I found a little edge case and I am not sure how to solve it.
Error:

Spawning language server with cmd: rust-analyzer failed. The language server is either not installed, missing from PATH, or not executable.

My setup:
rust-analyzer is in fact NOT in path, BUT is being passed directly to lsp cmd attribute as path:
nvim_lsp.rust_analyzer.setup({
  cmd = [ '/path/to/rust-analzyer']
  ...
}

I'm not sure how you have the rest of your configs but the problem might be that if you really are using rust-tools you need to pass the config into the rust-tools setup function.
local rust_opts = {
    server = {
        cmd = ['/path/to-rust-analyzer']
    },
}
require('rust-tools').setup(rust_opts)

Also please don't call lspconfig's setup, as rust-tools does it automatically and it causes conflicts if called twice


Fixup VirtualMachine and CloudService instances

The constructor requested information which is not always
needed depending on the called instance methods. Therefore
the class requests operational information like container
names, region name, etc only when needed but not directly
at instance creation time. Fixes #158

LGTM


Add imagePullSecrets for AMCE http01 solver pod

Pull Request Motivation

We are able to specify service account name in acme http solver pod spec to pull solver image from private repo but there are use cases when users managing cert-manager don't have access to create service accounts in other namespaces where solver pod is running. This PR allows to specify imagePullSecrets in solver pod spec.
Related to #3853, #3817, #3913
Kind

feature
Release Note

NONE

/ok-to-test
/kind-cleanup

/kind cleanup

/kind feature


WebSocket is closed before the connection is established.

Reverb Version
1.0
Laravel Version
11.9
PHP Version
8.2.0
Description
I tried to integrate a one to one chat application on our application. Locally, its running fine. When i am integrating it on a server, i am unable to make it work
Getting this error :: WebSocket connection to 'wss://.....' failed: WebSocket is closed before the connection is established.
tried to changed port and all, could not make it work. Any help is appreciated.
Steps To Reproduce
Working fine locally, on server something is broken.

Hey there,
While this may be a legitimate issue, can you first try posting your problem or question on one of the support channels below? If this issue can be definitively identified as a bug, feel free to open up a new issue with a link to the original one and we'll gladly help you out.

Laracasts Forums
Laravel.io Forums
StackOverflow
Discord
Larachat
IRC

Thanks!


add share functionality

Signed-off-by: MUzairS15<EMAIL_ADDRESS>Notes for Reviewers
This PR fixes #
Signed commits

[ ] Yes, I signed my commits.

Codecov Report
Base: 16.46% // Head: 12.23% // Decreases project coverage by -4.23% :warning:

Coverage data is based on head (cf9b616) compared to base (4e8b756).
Patch coverage: 0.00% of modified lines in pull request are covered.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6950      +/-   ##
==========================================
- Coverage   16.46%   12.23%   -4.23%
==========================================
  Files         117      117
  Lines       16225    16235      +10
==========================================
- Hits         2671     1987     -684
- Misses      13073    13929     +856
+ Partials      481      319     -162

Flag
Coverage Δ

gointegrationtests
?

unittests
12.23% <0.00%> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

server/handlers/user_handler.go
0.00% <0.00%> (ø)

server/router/server.go
0.00% <0.00%> (ø)

mesheryctl/internal/cli/root/system/start.go
1.56% <0.00%> (-54.07%)
:arrow_down:

mesheryctl/internal/cli/root/system/status.go
1.51% <0.00%> (-53.79%)
:arrow_down:

mesheryctl/internal/cli/root/system/stop.go
2.00% <0.00%> (-46.50%)
:arrow_down:

mesheryctl/internal/cli/root/system/update.go
1.73% <0.00%> (-43.48%)
:arrow_down:

mesheryctl/internal/cli/root/system/logs.go
4.00% <0.00%> (-42.67%)
:arrow_down:

mesheryctl/internal/cli/root/system/restart.go
2.66% <0.00%> (-42.67%)
:arrow_down:

mesheryctl/internal/cli/root/root.go
25.49% <0.00%> (-40.20%)
:arrow_down:

mesheryctl/internal/cli/root/version.go
0.00% <0.00%> (-34.24%)
:arrow_down:

... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

🚀 Preview for commit bb7bd30cb4aa34ea18910bd2f0749007a8e576d3 at: https://63d99439378f37006f959085--meshery-docs-preview.netlify.app

🚀 Preview for commit a5b3ff1ec5b3cc7f9736450d5e7edfe7d140f1c3 at: https://63da38567c52a2741db6716b--meshery-docs-preview.netlify.app

🚀 Preview for commit a6d24f8a1a994f5273e54a02897270c892373583 at: https://63da4f9f5f0d800637044df3--meshery-docs-preview.netlify.app


[Question] Is it possible that sometimes the same delayed_work record is processed more than once?

Description
Is it possible that sometimes the same delayed_work record is processed more than once?
From the testing we've been able to do so far, it seems like this is an intermittent issue.
Also, I haven't encountered a scenario where the same record was processed more than twice.
I have created a sample Spring Boot project here https://github.com/RMISCG/queue which you can use for testing purposes.
The project has a README.md file in the parent directory for more details.
Additional Information

Framework Version:
Java: 11.0.4
Spring Boot: 2.2.5.RELEASE
Gradle: 5.6

Platform:
macOS Mojave Version 10.14.6

Command/Work You Tried to Run:

cd into the parent directory.
./gradlew clean
./gradlew build
./gradlew bootRun

Any Output you Recieved (record with state 09 processed twice):

2020-03-03 12:58:35.846  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Starting QueueApplication on ml0179620 with PID 34280 (/Users/rmis/dev/apps-external/queue/build/classes/java/main started by rmis in /Users/rmis/dev/apps-external/queue)
2020-03-03 12:58:35.849  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : No active profile set, falling back to default profiles: default
2020-03-03 12:58:36.413  INFO 34280 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2020-03-03 12:58:36.993  INFO 34280 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2020-03-03 12:58:37.499  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 01 and result 01
2020-03-03 12:58:37.610  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 02 and result 02
2020-03-03 12:58:37.722  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 03 and result 03
2020-03-03 12:58:37.834  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 04 and result 04
2020-03-03 12:58:37.949  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 05 and result 05
2020-03-03 12:58:38.060  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 06 and result 06
2020-03-03 12:58:38.172  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 07 and result 07
2020-03-03 12:58:38.282  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 08 and result 08
2020-03-03 12:58:38.393  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 09 and result 09
2020-03-03 12:58:38.504  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 10 and result 10
2020-03-03 12:58:38.616  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 11 and result 11
2020-03-03 12:58:38.729  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 12 and result 12
2020-03-03 12:58:38.840  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 13 and result 13
2020-03-03 12:58:38.951  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 14 and result 14
2020-03-03 12:58:39.065  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 15 and result 15
2020-03-03 12:58:39.177  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 16 and result 16
2020-03-03 12:58:39.361  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 17 and result 17
2020-03-03 12:58:39.478  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 18 and result 18
2020-03-03 12:58:39.588  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 19 and result 19
2020-03-03 12:58:39.700  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 20 and result 20
2020-03-03 12:58:39.813  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 21 and result 21
2020-03-03 12:58:39.926  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 22 and result 22
2020-03-03 12:58:40.038  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 23 and result 23
2020-03-03 12:58:40.151  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 24 and result 24
2020-03-03 12:58:40.262  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 25 and result 25
2020-03-03 12:58:40.373  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 26 and result 26
2020-03-03 12:58:40.485  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 27 and result 27
2020-03-03 12:58:40.599  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 28 and result 28
2020-03-03 12:58:40.712  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 29 and result 29
2020-03-03 12:58:40.832  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 30 and result 30
2020-03-03 12:58:40.947  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 31 and result 31
2020-03-03 12:58:41.061  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 32 and result 32
2020-03-03 12:58:41.172  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 33 and result 33
2020-03-03 12:58:41.283  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 34 and result 34
2020-03-03 12:58:41.395  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 35 and result 35
2020-03-03 12:58:41.506  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 36 and result 36
2020-03-03 12:58:41.620  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 37 and result 37
2020-03-03 12:58:41.737  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 38 and result 38
2020-03-03 12:58:41.849  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 39 and result 39
2020-03-03 12:58:41.963  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 40 and result 40
2020-03-03 12:58:42.075  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 41 and result 41
2020-03-03 12:58:42.186  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 42 and result 42
2020-03-03 12:58:42.299  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 43 and result 43
2020-03-03 12:58:42.411  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 44 and result 44
2020-03-03 12:58:42.523  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 45 and result 45
2020-03-03 12:58:42.634  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 46 and result 46
2020-03-03 12:58:42.746  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 47 and result 47
2020-03-03 12:58:42.865  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 48 and result 48
2020-03-03 12:58:42.975  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 49 and result 49
2020-03-03 12:58:43.088  INFO 34280 --- [           main] com.rss.queue.QueueApplication           : Created com.rss.queue.job.EchoJob with state 50 and result 50
2020-03-03 12:58:43.366  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 01
2020-03-03 12:58:43.442  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 50
2020-03-03 12:58:43.519  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 49
2020-03-03 12:58:43.596  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 48
2020-03-03 12:58:43.673  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 47
2020-03-03 12:58:44.118  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 46
2020-03-03 12:58:44.192  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 45
2020-03-03 12:58:44.266  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 44
2020-03-03 12:58:44.341  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 43
2020-03-03 12:58:44.421  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 42
2020-03-03 12:58:44.867  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 41
2020-03-03 12:58:44.944  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 40
2020-03-03 12:58:45.020  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 39
2020-03-03 12:58:45.105  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 38
2020-03-03 12:58:45.188  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 37
2020-03-03 12:58:45.696  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 36
2020-03-03 12:58:45.771  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 35
2020-03-03 12:58:45.846  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 34
2020-03-03 12:58:45.922  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 33
2020-03-03 12:58:45.996  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 32
2020-03-03 12:58:46.510  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 31
2020-03-03 12:58:46.587  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 30
2020-03-03 12:58:46.665  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 29
2020-03-03 12:58:46.741  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 28
2020-03-03 12:58:46.819  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 27
2020-03-03 12:58:47.269  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 26
2020-03-03 12:58:47.344  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 25
2020-03-03 12:58:47.418  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 24
2020-03-03 12:58:47.493  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 23
2020-03-03 12:58:47.569  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 22
2020-03-03 12:58:48.103  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 21
2020-03-03 12:58:48.179  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 20
2020-03-03 12:58:48.253  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 19
2020-03-03 12:58:48.327  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 18
2020-03-03 12:58:48.400  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 17
2020-03-03 12:58:48.931  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 16
2020-03-03 12:58:49.005  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 15
2020-03-03 12:58:49.079  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 14
2020-03-03 12:58:49.153  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 13
2020-03-03 12:58:49.232  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 12
2020-03-03 12:58:49.668  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 11
2020-03-03 12:58:49.751  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 10
2020-03-03 12:58:49.827  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 09
2020-03-03 12:58:49.904  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 08
2020-03-03 12:58:49.980  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 07
2020-03-03 12:58:50.495  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 06
2020-03-03 12:58:50.570  INFO 34280 --- [pool-1-thread-2] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 05
2020-03-03 12:58:50.648  INFO 34280 --- [pool-1-thread-3] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 04
2020-03-03 12:58:50.726  INFO 34280 --- [pool-1-thread-4] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 03
2020-03-03 12:58:50.800  INFO 34280 --- [pool-1-thread-5] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 02
2020-03-03 12:58:51.836  INFO 34280 --- [pool-1-thread-1] com.rss.queue.job.EchoJob                : Running com.rss.queue.job.EchoJob with state 09
Disconnected from the target VM, address: '<IP_ADDRESS>:59884', transport: 'socket'

Process finished with exit code 130 (interrupted by signal 2: SIGINT)

Hey,
It is certainly possible that a job can be worked twice. Although, I don't think this is a case where it should be. The rework should only happen in a failure case where a job failed to report it's status to the DB. I'll take a bit of a deeper look here as to why a job is getting queued multiple times.

I've run it 10 times, and haven't been able to replicate. Could you perhaps turn on the log level filtering off (I realize that'll be a lot of logs), and post there somewhere like gist.github.com so I can view the logs to see why it's being picked up again?

here's a gist snippet: https://gist.github.com/RMISCG/90780e69d8e517d80b794ca3231860b1
you'll notice that record with state 07 was processed twice. search for string "Running com.rss.queue.job.EchoJob with state 07"
also, if you want to do this yourself, show all coworker logs, simply remove the following block from application.yml:
logging:
  level:
    io:
      kungfury:
        coworker: WARN

Thanks for this! I do know how to change the logs, I just can't reproduce it unfortunately :smile:
So the logs actually show that the job gets released back to the pool:

https://gist.github.com/RMISCG/90780e69d8e517d80b794ca3231860b1#file-queuecoworkerlogs-txt-L226

The only places ReleaseToPool are called:

When a node first comes online it releases it's work that it previously had incase a job gets restarted while it was working: https://github.com/SecurityInsanity/Coworker/blob/d9e6aabb6d/src/main/kotlin/io/kungfury/coworker/CoworkerManager.kt#L104

If a service-checker is passed, and it reports a node as offline (this can't be called since the default service checker never reports any nodes as offline since it has no way of knowing): https://github.com/SecurityInsanity/Coworker/blob/d9e6aabb6d/src/main/kotlin/io/kungfury/coworker/CoworkerManager.kt#L121

If the future that is handling the work is cancelled: https://github.com/SecurityInsanity/Coworker/blob/d9e6aabb6d/src/main/kotlin/io/kungfury/coworker/CoworkerManager.kt#L219

If the work is marked as finished without finishWork being called: https://github.com/SecurityInsanity/Coworker/blob/d9e6aabb6d/src/main/kotlin/io/kungfury/coworker/CoworkerManager.kt#L235

If the NStrand is at maximum: https://github.com/SecurityInsanity/Coworker/blob/d9e6aabb6d/src/main/kotlin/io/kungfury/coworker/CoworkerManager.kt#L378

My biggest thought is that the future is cancelled. As none of the others make sense (and I can't seem to reproduce). Is there anything you can think of that would kill the thread/runnable? Perhaps using too much memory? Something along those lines?

I'm not really sure why you're unable to reproduce this. Maybe you can:

increase the number of delayed_work records - increase NUMBER_OF_JOBS in QueueApplication.
AND/OR
increase the number of coworker threads - increase THREADS in CoworkerConfiguration.

Hi,
Any updates on this?
Thanks

Hey,
I'm still not able to replicate this on my end. Even after increasing NUMBER_OF_JOBS && THREADS. Though I still think the future is getting cancelled (somehow). If you'd like debug further yourself I recommend adding in some debug lines on the line I pointed out where a job gets released to the pool for a future getting cancelled (as well as the others) to see where exactly the job is getting released back to the pool.


🐞 Dummy data incorrect

After #165 the created dummy data (universities' main pages) is missing. The main pages are now different.

Also, the universities visible at the beginning are different

@WiktorSzymanski do sprawdzenia


What if instruction+text merged?

Is it possible to apply somehow for instruction prompts without instruction/text separation?

Hi, Thanks a lot for your interest in the INSTRUCTOR!
It is possible to merge the instruction and text as one sentence, and put all the contents in the text body, leaving the instruction as an empty string, but this may be misaligned with the training format.

Feel free to re-open the issue if you have any further questions or comments!


defaultShippingAddress and defaultBillingAddress are not part of user addresses and they fail to fetch by id

What I'm trying to achieve
I am trying to fetch a user's defaultShippingAddress by id. That query returns null
Steps to reproduce the problem

Just save a defaultShippingAddress
Save other user addresses
Try to fetch the defaultShippingAddress by id, and it will return null. It's also not part of the addresses query response.

What I expected to happen
I expect the defaultShippingAddress and defaultBillingAddress are part of the user addresses array and I am able to query them by id's just like regular addresses.
Screenshots and logs

System information

Saleor version:

[ ] dev (current master)
[x] 3.0
[ ] 2.11
[ ] 2.10

Operating system:

[ ] Windows
[ ] Linux
[x] MacOS
[ ] Other

@cipriancaba Could you check in current master? Recently i've patched exact same issue on user creation in https://github.com/mirumee/saleor/pull/7327

If it won;t work on master please provide which mutation you used in order to set the default address.


assertObject: where is it defined?

Hi,
I recently updated to version 0.0.58. I use Traceur to parse Node.js modules. Ever since the update, whenever I use object/array destructuring I keep seeing $traceurRuntime.assertObject in my compiled codes, whose definition I can't find anywhere in the compiled codes. Where is it defined?

no  my compiled code and runtime library are in sync are you sure we dont need to add anymore packages other than traceur?

Are you including traceur-runtine.js?

yes I can share my package.json,bower.json and grunt task

Thanks @arv for getting back. Can you pull request from this link https://github.com/sid0818/Ecma6ToEcma5?


All types used as an Error or Warning should have a nice to_display_text function

This task is automatically imported from the old Task Issue Board and it was originally created by James Dunkerley.
Original issue is here.

All warning types attached by the APIs will be rendered to the user via the to_display_text.
This ticket is to review the display format and ensure they have helpful messages for each error.
Additionally, we should add tests to ensure regressions do not occur on this.

We should probably ensure our unit tests run to_display_text of each error type, to ensure that the error visualizations work correctly.
This will allow to avoid regressions like to_display_text breaking due to changed field names.


1004 Error

I am trying to download file using 1.2.4 version. Sometimes it is giving Error Message: IOException: Failed reading response. Error Code is 1004. Can you give me some tips why is it coming?

This occurs when there is an exception reading from the network connection stream. I suspect you are disconnecting from WiFi during the download.

@TapanM Let me know if this issue persists for you.

Anyone ever figure this one out? I am experiencing this error but there are no wifi or connection errors.

Hello,,
Many users complains from this error
i don't know why?
anyone have explain for this
i attached print screen from google analytic
http://prntscr.com/fx5po3


how to get html of the stream of the file or butter after generating

Can I get the stream or buffer from the generated pdf without setting pdf path?

I've updated convertToPdf to always return a Buffer. You can still specify outputPath to save the PDF, but it's optional.


ReadOnlyReplicationHelperCLI: do not rely on down node

When building a list of partitions to be copied between nodes
to restore a down node, don't expect to be able to fetch any
useful metadata from the down node.
v2: better method naming per feedback from athirupathi

looks good, please merge it in.


Next Version

This PR serves as the basis for an entire rewrite of this package. The previous version has been around for a very long time and is quite long in the tooth.
Key improvements over the older version are:

The overall architecture in the older version leads to a lot of duplication in tests and our tests are in fact often not really testing anything meaningful.
After re-reading the spec, there are a couple of areas where we currently don't correctly address creating an OAuth signature. Obviously this works for most folks, however there's some edge cases which we don't account for. Upon searching the web for other OAuth 1 packages, it appears nobody (that I've seen) accounts for these. An example is where the same parameter is found in both the request body and the query string. Both instances should be used for generating the OAuth signature. The new version accounts for this.
The creation of additional providers which customise functionality was painful in the previous iteration. By utilising PSR-7, PSR-17 and PSR-18, providers have a lot more flexibility to setup unique requests if they need to instead of overriding a lot of functionality.
Again, with the addition of PSR, we have decoupled from requiring specific Guzzle versions. We suggest Guzzle as a perfect partner for this package, however our only dependency is an implementation of the relevant PSRs as well as a Guzzle support package for handling RFC 3986 encoding/decoding. The final iteration of this package may remove that dependency.
The rewrite express a Generic Provider, which any OAuth1-compliant server should work with. We will likely remove all additional providers by extracting them into external packages (up for debate) and supply only the Generic Provider out of the box.
The rewrite has extensive unit test coverage with many scenarios in key areas, such as signature generation and OAuth flow. We have taken examples right out the OAuth spec and plugged them into our tests to ensure that we catch the edge cases exposed by the spec. We also have 100% code coverage, obviously 😉.

To do:

[ ] Decide if we extract providers out and ship with only a Generic Provider, or do we ship with the same providers as the old version? Thoughts: maybe we ship with the most common ones. Some less common ones like Magento 1 introduce the requirement of XML-based PHP extensions being dependencies…
[ ] Write up detailed documentation with usage examples.
[ ] Add a handy Guzzle middleware to sign authenticated OAuth 1 requests, or recommend the usage of the official OAuth subscriber.
[ ] Decide what properties to officially support on the User object. At the time of writing, we support id, email, username and all other data is provider-specific and is accessible via the user's metadata array.
[ ] Explore adding integrations/bridges (likely in separate repositories) for common frameworks such as Laravel/Symfony.

💥
😄

💥
😄

@mfn not so much anymore!

Add a handy Guzzle middleware to sign authenticated OAuth 1 requests, or recommend the usage of the official OAuth subscriber.

I'm not familiar with this implementation (so many changes 🙀) and I'm not sure if this it the proper forum to bring this up.
guzzle/oauth-subscriber works by storing the token credentials on the class in the constructor. This means once instantiated, this "state" persists to all requests performed with it.
I've a use-case where I want to pass these credentials on a per request basis (by allowing credentials being passed via Guzzles $options). This frees the Guzzle client from being tied to a specific set of credentials and allows to be more conservative with resources and logic:

a single Guzzle client instance can be used for all instances, as it's not tied to a set of credentials
every call explicitly can pass a different set of credentials

This is useful when interacting with multiple identities but not requiring to have multiple HTTP clients for this.

I searched for an opportunity to mention this somewhere in the hopes there are peers recognizing this as a valid use-case. Right now, to solve this, I've had to copy OAuth1 into a private source tree and adapt it accordingly.

I see.
Why don’t you want to create a separate client instance for each account you’re acting on behalf of?
Having said that, I can see your use-case being solved with this new version pretty much out of the box.
There’s a new $client->executeAuthenticatedRequest() method now. We take a PSR-7 request (guzzle satisfies this requirement) and also a PSR-18 HTTP client (guzzle also satisfies this).
I’ll put some more thought into it and come back with an example soon.
Sent from my iPhone

On 3 Sep 2020, at 2:29 pm, Markus Podar<EMAIL_ADDRESS>wrote:

Add a handy Guzzle middleware to sign authenticated OAuth 1 requests, or recommend the usage of the official OAuth subscriber.
I'm not familiar with this implementation (so many changes 🙀) and I'm not sure if this it the proper forum to bring this up.
guzzle/oauth-subscriber works by storing the token credentials on the class in the constructor. This means once instantiated, this "state" persists to all requests performed with it.
I've a use-case where I want to pass these credentials on a per request basis (by allowing credentials being passed via Guzzles $options). This frees the Guzzle client from being tied to a specific set of credentials and allows to be more conservative with resources and logic:
a single Guzzle client instance can be used for all instances, as it's not tied to a set of credentials
every call explicitly can pass a different set of credentials
This is useful when interacting with multiple identities but not requiring to have multiple HTTP clients for this.
I searched for an opportunity to mention this somewhere in the hopes there are peers recognizing this as a valid use-case. Right now, to solve this, I've had to copy OAuth1 into a private source tree and adapt it accordingly.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Why don’t you want to create a separate client instance for each account you’re acting on behalf of?

Persistent background worker job connecting to the same remote OAuth1 system on behalf of identities likely different for each job.
To conserve memory and reduce GC, I try to make objects "stay" if possible, make them state-less so they can be shared like this.
Thanks!

Why don’t you want to create a separate client instance for each account you’re acting on behalf of?

Persistent background worker job connecting to the same remote OAuth1 system on behalf of identities likely different for each job.
To conserve memory and reduce GC, I try to make objects "stay" if possible, make them state-less so they can be shared like this.
Thanks!

Hi @mfn, here's a snippet of how I would solve your problem. I've included all of the boilerplate as well as I'd like to know your thoughts on it overall:
<?php

require_once __DIR__.'/vendor/autoload.php';

// Boilerplate to create a Twitter provider
$twitter = new League\OAuth1\Client\Provider\Twitter(
    new League\OAuth1\Client\Credentials\ClientCredentials(
    'app-id',
    'app-secret',
    'https://api.client.com/callback',
    )
);

// Create an OAuth client, giving a provider, PSR-17 HTTP request factory and PSR-18 client.
//
// You can use:
// 1.  `http-interop/http-factory-guzzle` for the PSR-17 HTTP request factory
// 2a. `guzzlehttp/guzzle` (v7 for PHP 7.2+) or
// 2b. `php-http/guzzle6-adapter` (for PHP 7.1)
//
// I'm currently testing the package in PHP 7.1, so I'm using 1 and 2b from above
$oauthClient = new League\OAuth1\Client\Client(
    $twitter,
    new Http\Factory\Guzzle\RequestFactory(),
    new Http\Adapter\Guzzle6\Client()
);

// Now you need a PSR-7 request, I use `guzzlehttp\psr7` for this
$request = new GuzzleHttp\Psr7\Request('GET', 'https://api.example.com/me');

// Here's your token credentials / access tokens
$tokenCredentials = new League\OAuth1\Client\Credentials\Credentials('token-id', 'token-secret');

// Now, you could either set this with the client for continual use:
//
// $oauthClient->setTokenCredentials($tokenCredentials);
// $response = $oauthClient->executeAuthenticatedRequest($request);
//
// However you won't want to do this as this sets the token credentials for the client overall.
// You might want to send unique token credentials every time:

try {
    $response = $oauthClient->executeAuthenticatedRequest($request, $tokenCredentials);
} catch (Psr\Http\Client\ClientExceptionInterface $e) {
    //
} catch (Throwable $e) {
    //
}

@mfn out of curiosity - are you integrating with a known OAuth 1 provider? I'm looking through what still uses OAuth 1 and there isn't a lot!

@bencorlett The code in general looks fine to me and decouples the token credentials, seems exactly what I was talking about.

are you integrating with a known OAuth 1 provider?

Yes, Twitter ;)
Currently using guzzle/oauth-subscriber absorbed in the code base and adapted to serve the use-case.
Once a (dev) release is available, I can try integrating it and see how it works out. At first the code looks boilerplate-y though, but I might judge this wrong. I'm just used to directly use Guzzle 6 and not having to deal with the abstractions too much.
Thanks!

Yeah the cost of decoupling from dependency implementations is you must supply said implementations.
Having said that, we’ll likely ship with a Guzzle 6/7 factory for convenience.
Sent from my iPhone

On 4 Sep 2020, at 4:18 pm, Markus Podar<EMAIL_ADDRESS>wrote:

@bencorlett The code in general looks fine to me and decouples the token credentials, seems exactly what I was talking about.
are you integrating with a known OAuth 1 provider?
Yes, Twitter ;)
Currently using guzzle/oauth-subscriber absorbed in the code base and adapted to serve the use-case.
Once a (dev) release is available, I can try integrating it and see how it works out. At first the code looks boilerplate-y though, but I might judge this wrong. I'm just used to directly use Guzzle 6 and not having to deal with the abstractions too much.
Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Will open additional PRs for further work. Develop will point to the next version from now on.


Installation security alert

This app is not safe and not updated (many packages are not supported anymore)
found 90 vulnerabilities (58 low, 11 moderate, 19 high, 2 critical) in 15883 scanned packages   runnpm audit fix to fix 32 of them.   38 vulnerabilities require semver-major dependency updates.   20 vulnerabilities require manual review. See the full report for details.

Hi @swina ,
Apologies for the inconvenience caused. We are working on updating the outdated packages which can resolve the vulnerability issues as well. We will publish the updated code within two business days.

Hi @swina ,
We have updated the outdated packages. Please download the latest files and try again.


500 error code in sp.audio_features() reported as 429

Describe the bug
500 error code in sp.audio_features() reported as 429
Your code
For this particular ID, https://developer.spotify.com/console/get-audio-features-several-tracks/ returns 500.
audio_features = sp.audio_features(tracks=["1tkkrLY3upIwNfLs4ei8Bh"])

Spotipy returns:
 "exception": "SpotifyException(429, -1, '/v1/audio-features/?ids=1tkkrLY3upIwNfLs4ei8Bh:\\n Max Retries')"

Expected behavior
I would expect Spotipy to show me that it is a 500 error code. So I could debug and verify which ID is causing the issue on Spotify's side. When I am getting 429 I am assuming it is a limit error.
Environment:

OS: Mac, Docker
Python version 3.10.2
spotipy version spotipy = "==2.19.0"
VS Code

Additional context
The request exception is caught here: https://github.com/plamere/spotipy/blob/9a627e88f422927822ce39ae9919cc7ab9813dde/spotipy/client.py#L274
And then it is raising 429 here: https://github.com/plamere/spotipy/blob/9a627e88f422927822ce39ae9919cc7ab9813dde/spotipy/client.py#L281

What makes you so sure that 429 is the wrong error? How do you know that your requests aren't getting rate-limited by Spotify?
Have you used the debugger to determine that spotipy returns a rate-limited error even when the raw network response from the requests module contains a 500 status code?

Hey. I checked what error code is returned when requesting same url using same token using Spotify’s console and then also using Postman. It returns 500 in both cases.
Spotipy returns 429 (same token, same url, before and after I tested in console).
It’s super easy to check using code samples and IDs I provided.

I got the same issue and error (max retries) when feeding a list of songs, and when searching for it (by link in browser and the URI ofc.) it said it was unavailable, however it was not a local song and I got the song info for it when requesting a playlist of songs. My thought is it might just not be available in your market.
My solution after trying multiple fixes is to just if you want to also fetch other songs, do a try except, and just ignore the one that doesn't work.
It's not rate limiting, at least not in my case, because all songs except some few ones don't work, and the try except allows me to fetch multiple other songs and just skip these that are retrying.

I was unable to reproduce the rate-limited error with any of the ids, although I did get a 500 error with all of these ids. I don't know why this error is returned for certain tracks, but it's clear that this is not due to an issue on spotipy's end because I get the same error when I make the request in Postman. You'll have to ask about this on the Spotify Developer Forums.

Thank you for your answers, I appreciate your time.

You'll have to ask about this on the Spotify Developer Forums.

Yes. But I don't mind 500s from Spotify for some IDs - it happens, it is ok. But I would like Spotipy to show me it was a 500 on Spotify's end and not 429.
This code (assuming you have your env SPOTIPY_CLIENT_ID and SPOTIPY_CLIENT_SECRET)
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials

sp = spotipy.Spotify(auth_manager=SpotifyClientCredentials())
result = sp.audio_features(tracks=["1tkkrLY3upIwNfLs4ei8Bh"])

Results in urllib3 exception, caught and passed byrequests exception and then caught by Spotipy and then reported as spotipy.exceptions.SpotifyException: http status: 429. Which (in my opinion) is not correct, becase my rate limits are OK, and Spotify returns 500 for this particular request (albo same market etc.). Full Traceback (Pastebin for convenience, line 42, https://pastebin.com/1SHAQJbH):
(app) /app # python 500_test.py
Max Retries reached
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/connectionpool.py", line 876, in urlopen
    return self.urlopen(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/connectionpool.py", line 876, in urlopen
    return self.urlopen(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/connectionpool.py", line 876, in urlopen
    return self.urlopen(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/connectionpool.py", line 866, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/audio-features/?ids=1tkkrLY3upIwNfLs4ei8Bh (Caused by ResponseError('too many 500 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/spotipy/client.py", line 240, in _internal_call
    response = self._session.request(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/requests/adapters.py", line 510, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/audio-features/?ids=1tkkrLY3upIwNfLs4ei8Bh (Caused by ResponseError('too many 500 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/500_test.py", line 5, in <module>
    result = sp.audio_features(tracks=["1tkkrLY3upIwNfLs4ei8Bh"])
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/spotipy/client.py", line 1683, in audio_features
    results = self._get("audio-features/?ids=" + ",".join(tlist))
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/spotipy/client.py", line 297, in _get
    return self._internal_call("GET", url, payload, kwargs)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/spotipy/client.py", line 281, in _internal_call
    raise SpotifyException(
spotipy.exceptions.SpotifyException: http status: 429, code:-1 - /v1/audio-features/?ids=1tkkrLY3upIwNfLs4ei8Bh:
 Max Retries, reason: too many 500 error responses
(app) /app #

@Peter-Schorn where you able to reproduce it? I am willing to help.


Fix OpenMetadata entities link

OpenMetadata has released a new version and it seems that they have introduced backward compatibility problems with the links used in our Readme.
The following link provided by the Readme does not work any more:
https://docs.open-metadata.org/openmetadata/schemas/entities/tagcategory
https://docs.open-metadata.org/openmetadata/schemas/entities/table#tabledata
https://docs.open-metadata.org/openmetadata/schemas/entities/table#tableprofile

I tried to fix all the broken links and add the CUE validation for OpenMetadata structures.
I think we can close this issue.


[Mobile App] Uninstalled mobile app still displayed in App list on UI for Android

Pre-Condition:-

IoTS should be up and running
A device should be enrolled
The enrolled device should be Android 4.4.2

Install WSO2 con app on device
Manually uninstall the app from device
Make sure the device is synched with the IoTS after uninstalling the app
Go to device management UI and check installed apps

Expected Results:-

Uninstalled apps should not be displayed on Applications list

Actual Results:-

Uninstalled app (WSO2 con) is being displayed on applications list (screenshot attached)

@rasika @chamaraanu : Labels: 3.1.0-RC2 , Medium

This issue is no longer reproducible.


Skip manifest which have no annotation for tags

What type of PR is this?
feature

Which issue does this PR fix:
Closes https://github.com/project-zot/zot/issues/704
What does this PR do / Why do we need it:
It skips manifests of a repo that don't have an annotation tag.
If an issue # is not available please add repro steps and logs showing the issue:
Testing done on this change:

tests add a manifest that doesn't have a tag as a manifest.Annotations field and expect this manifest will be skipped
on calls such as ExpandedRepoInfo
Automation added to e2e:

Will this break upgrades or downgrades?
Does this PR introduce any user-facing change?:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

skopeo --insecure-policy copy --format=oci --dest-tls-verify=false --multi-arch=all \         docker://public.ecr.aws/docker/library/busybox:latest \         docker://<IP_ADDRESS>:8080/busybox:latest
can you also do this and make sure the image is returned.
This path is different because '--multi-arch' creates a Manifest Index (instead of just a Manifest) that in turn refers to other manifests, and let's make sure this is returned.


Test Rhat for the Gibbs kernels

This consists of multiple tests that follow the same basic structure:

Take a single sample from an HIRM
Incorporate that sample into n new HIRMs
Call the relevant transition method multiple times on each HIRM
Use R-hat (#131) to test stationarity/sampling efficiency

Test each of these components

[ ] Transitioning entity cluster assignments, using the mean number of clusters as a metric
[ ] Transitioning relation cluster assignments, using the mean number of clusters as a metric
[ ] Transitioning latent values

The tests should be applied both to a case where the individual Gibbs kernel is transitioned repeatedly (with everything else held constant) and to a case where the whole model is transitioned.

This sounds ultimately good to have but maybe not highest-priority. (I have rarely used these diagnostics in the early stages of developing a model / inference algorithm, but it's possible there are good ways of doing so -- so I don't mean to discourage if we think it would be useful!)


Watch on SQL view not updating

I’m using a .drift file to define a view and in DAO I'm accessing this view to create a select statement. The query works and returns the expected result, but when I try to watch the query no updates are pushed to the stream. The changes are made to the underlying table of the view.
The underlying table is defined in a dart file:
@DataClassName('ChatMessageEntry')
class ChatMessageEntries extends Table {
  IntColumn get id => integer().autoIncrement()();

  TextColumn get uuid => text()();
  TextColumn get participant => text()();

  IntColumn get timestamp => integer()();
  TextColumn get content => text()();

  TextColumn get serverId => text().nullable()();
  IntColumn get serverTimestamp => integer().nullable()();

  BoolColumn get outgoing => boolean()();

  @override
  List<String> get customConstraints => [
        'UNIQUE (server_id)',
        'CHECK (server_timestamp != NULL OR outgoing)',
      ];
}

The view in the .drift file:
import '../models/chat/chat_message.dart';

CREATE VIEW IF NOT EXISTS chat_message_display_view AS
SELECT *, CASE WHEN outgoing THEN (
    CASE WHEN server_timestamp IS NULL THEN (
        timestamp
    ) ELSE (
        CASE WHEN timestamp < server_timestamp THEN (
            timestamp
        ) ELSE (
            server_timestamp
        )
        END
    )
    END
) ELSE server_timestamp END AS display_timestamp
FROM chat_message_entries;

The DAO:
@DriftAccessor(tables: [ChatMessageEntries])
class ChatMessageEntriesDao extends DatabaseAccessor<Database> with _$ChatMessageEntriesDaoMixin {
  ChatMessageDisplayView get chatMessageDisplayView => attachedDatabase.chatMessageDisplayView;

  ChatMessageEntriesDao(Database db) : super(db);

  // Here are the queries
}

The insert query:
return into(chatMessageEntries).insertReturning(ChatMessageEntriesCompanion.insert(
    uuid: uuid,
    timestamp: Encodings.encodeDate(clock.now()),
    content: content,
    participant: participant,
    outgoing: true,
  ));

The select query:
final statement = select(chatMessageDisplayView)
   ..where((tbl) => tbl.uuid.equals(uuid) & tbl.participant.equals(participant));
return statement.watch();

I'm using drift 1.1.1 and drift_dev 1.1.1

Thanks for the report! Basically, the problem here is that we only track updates on tables at runtime, so views didn't work properly. When selecting views in a .drift-defined query, we can find all the tables that view reads from and use that information to work around that problem. So far, no such mechanism existed for the Dart API. I have added the necessary information and handling in 0bad799b8254fe763f73b2ab8c71af52e9001003.

Thanks a lot for this extremely fast fix. Can confirm that your fix works for my use case :)
And btw, it’s such a joy working with your library :)


LastPass app docs

Purpose of this pull request
This pull request is to create app docs for LastPass.
Select the type of change

[ ] Minor Changes - Typos, formatting, slight revisions
[ ] Update Content - Revisions, updating sections
[x] New Content - New features, sections, pages, tutorials
[ ] Site and Tools - .clabot, version updates, maintenance, dependencies, new packages for the site (Docusaurus, Gatsby, React, etc.)

Ticket (if applicable)
Create public doc for LastPass App

@JV0812, I'm merging this PR at the request of @ankitgoelcmu.


Mail subject utf8 error

Subject	=?utf-8?q?I=27some_test?=

Hi,
I'm facing a similar issue when trying to fetch mails from the Api (v2).
Performing a query on /api/v2/search using containing on the subject of a mail gives an empty result.
For instance I search: 'containing' => 'Your free product', but in fact the subject appears as Your free =?utf-8?Q?product=0A?= when fetching all mails.
Could it be possible to do something for clean up the subject as it is done for the web interface, in order to allow performing containing queries on?
Thank you


Workaround to allow autocompletion labels containing spaces

Note to self: the documentation for textadept.editing.autocompleters says that spaces are used by default, so autocompleter functions may not be written with using buffer.auto_c_separator in mind.

I've committed what I think is proper fix here: https://github.com/orbitalquark/textadept-lsp/commit/78e72c28acdd6d693cdc1c547f3803769a632d15


Modularize Styles

At the moment styles are in a big file index.css which should really I guess be for global styles.

going to combine with #9


Use explicit gen_scalars.exe dependency

This follows the convention used elsewhere in the codebase.
Explain how you tested your changes:

Mina project builds successfully

Checklist:

[x] Dependency versions are unchanged

Notify Velocity team if dependencies must change in CI

[ ] Modified the current draft of release notes with details on what is completed or incomplete within this project
[x] Document code purpose, how to use it

Mention expected invariants, implicit constraints

[x] Tests were added for the new behavior

Document test purpose, significance of failures
Test names should reflect their purpose

[x] All tests pass (CI will check this if you didn't)
[x] Serialized types are in stable-versioned modules
[x] Does this close issues? None

!ci-build-me


Update reqoption.go

add WithAppAccessToken()

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


update vanilla server to <IP_ADDRESS>

My little contribution. I was playing with my GF and when she restarted the game she couldn't log in anymore

@bskrtich could you push the new image please? :)

Done


windows 11's IE mode work differently then win 10's?

we test edge ie mode on windows 11.
some DOM click event is not working.
(on windows 10's ie mode, it works fine)
I found IE version is different.
win10 : 11.0.19041
win11 : 11.0.22000
i think, dynamic element's event is ignored.

@dodream8 Thanks for reaching out!  Normally these GitHub issues are specifically for Microsoft Edge Enterprise Documentation.
In this case it might be best to reach out to Support (https://microsoftedgesupport.microsoft.com/hc/en-us) they should be able to work with you directly and investigate the issues you experiencing.


suppress most of the pyethereum 1.6 UserWarnings

What was wrong?
I was tired of having to scroll through all the warnings
How was it fixed?
I suppressed the warnings with matching text in most of the modules that create the warning.  I left a few in there as a reminder to get it taken care of.
Cute Animal Picture

Thanks for doing this.  I'm evidently too lazy to figure out how to silence these and I've been slowly conditioned to ignore them.

Thanks for doing this.  I'm evidently too lazy to figure out how to silence these and I've been slowly conditioned to ignore them.


Feature Request: Remove jQuery dependency

I have an idea for a new feature for jQuery Terminal
Removing jQuery as a dependency could improve the loading speed of jquery.terminal.

I don't think it will ever happen, but I can check if you can use some jQuery alternative like Zepto or Cash.
Also, note that jQuery Terminal is twice as big as jQuery, it would be much bigger without jQuery. The code is much shorter because of using jQuery as a dependency, so it would not load faster because it will probably be twice as big. Also jQuery Terminal without jQuery would require rebranding. Maybe Vanilla Terminal. I don't think that this is something I would like to invest my time in.
The alternative is creating a Vanilla Adapter based on jQuery that uses native DOM API.
In short, I don't see myself rewriting the whole library, just to get rid of jQuery.

I expected something like that. Probably jQuery is a helpful dependency in this case. Anyway, thank you very much for your library!

I was playing with using cash library it says it's 8kB, but in fact, the library is 15kB. I've created a clone of jQuery because I needed $.Callbacks that is used in jQuery Terminal. The output file is as small as 21.7kB.
https://github.com/jcubic/jquery/blob/main/cash.min.js
But still, the library doesn't work because jQuery Terminal uses jQuery only CSS selectors like :visible, it may take a while to fix all the missing pieces to use Cash. Note that jQuery is only 36kB.
I don't this that 36kB is that much larger to make it worth spending much time working on this.

I now also think like that. Thank you for your efforts!


get current time: Time.now, Time.utc_now

The following snippets work.
puts Time.now
puts Time.utc_now

simulate TZ using $ TZ="America/New_York" wine ./foo.exe
Note: this does not handle daylight savings.

This didn't make travis happy :-( hold a bit please.

Travis is happy again. Ready to go if you think the path taken is good.
I think it would be great if you can enable travis on this fork and enable builds for PR. I hope other will join forces :-)

Travis has been enabled! :tada:
I've made some review comments, but I'm not sure how it notifies you (after the review system changed). Did you see them?

I see no review comments. After writing them yo need to click on the "review changes" button in the top right to send them all.