Error occured while getting decoy outputs

MacOS Venture, MyMonero 1.3.0

I also have this issue. I am on Linux.
It happened immediately after the balance finally showed up following the hard fork related bug/defect. I assume this is related...

Same issue on Windows 10. I also get "Error: count cannot be 11"

MacOS Venture, MyMonero 1.3.0

Hi @keinsell @KittyKattyKat @hack-r @eugeneotto @charleyhine, MyMonero are currently working on the latest Monero hardfork patch. Whilst we are scanning wallets to the current block height where you are able to view your xmr, we are working on the decoy selection issue which relates to send transactions not confirming. We will update you on when the spending function will be enabled. Will keep you updated with progress here.

Meanwhile I've used my mnemonic in Cake Wallet and have sent my funds, so it's no longer critical issue for me.

It's same on Windows 10. also get "Error: count cannot be 11",MyMonero 1.3.0

Yes all OSs are affected. The MyMonero company replied that they were joking previously about the software being fixed and hopefully it should be fixed soon.
No word on if they are paying us interest on the time-value of money.

Here is an official announcement from MyMonero:
We are happy to announce that the MyMonero wallet is fully operational again.
We have successfully upgraded the MyMonero software to fall in line with the Monero hard fork.
This is true for all our web-wallet, MacOS and Linux users, as well as those operating on the Android and iOS mobile apps.
You will need to update to the latest software in order to fully operate on the new software.
If you are unsure about this, you can refer to our website - mymonero.com for the latest software versions.
You can now view your XMR balances and transactions, and you can also send funds again.
An important note to our Windows users:
We have successfully completed the hard fork upgrades on the Windows software and although we are very eager to release it, we are awaiting a certificate that will allow us to release this software publicly.
This process is a priority for us and we will announce once our Windows platforms are available again.
Feel free to reach out to us if you have any questions or concerns.
Thank you for your patience.


Add Copyright Date to Footer

Description
Add a dynamic copyright date to the footer.  Should appear in the left column, immediately below "A LEANSTACKS Solution".

Code complete.


Make Swift linter stricter

Pull request
Description
See associated issue.
Changes made

Updated rules and code.

Checklist

[x] Your branch has been rebased onto the main branch.
[x] APIs have been properly documented (if relevant).
[x] The documentation has been updated (if relevant).
[x] New unit tests have been written (if relevant).
[x] The demo has been updated (if relevant).
[x] The playground has been updated (if relevant).

🌙 Available in TestFlight nightlies
The most recent state can be tested in the following builds:

Platform
Version

iOS
-

tvOS
1.0.0 (137)


partial_dependence with ensembles errors

Currently, trying to run partial_dependence with a stacked ensemble pipeline will raise the following error:

PartialDependenceError                    Traceback (most recent call last)
<ipython-input-12-4f356ba3e6ad> in <module>
      3     partial_dependence,
      4 )
----> 5 graph_partial_dependence(ensemble, X, 'Sex')

~/Desktop/evalml/evalml/model_understanding/graphs.py in graph_partial_dependence(pipeline, X, features, class_label, grid_resolution, kind)
    996
    997     part_dep = partial_dependence(
--> 998         pipeline, X, features=features, grid_resolution=grid_resolution, kind=kind
    999     )
   1000

~/Desktop/evalml/evalml/model_understanding/graphs.py in partial_dependence(pipeline, X, features, percentiles, grid_resolution, kind)
    856             raise PartialDependenceError(
    857                 str(e), PartialDependenceErrorCode.ALL_OTHER_ERRORS
--> 858             ) from e
    859
    860

PartialDependenceError: feature_importance is not implemented for SklearnStackedEnsembleClassifier and SklearnStackedEnsembleRegressor

This is because our ensemble estimator wraps a scikit-learn estimator using scikit_learn_wrapped_estimator. Within our wrapper classes, we call:
        if pipeline._is_fitted:
            self.feature_importances_ = pipeline.feature_importance
            self._is_fitted = True
            self.classes_ = pipeline.classes_

For a fitted ensemble, calling self.feature_importances_ = pipeline.feature_importance will error out since our ensemble does not support feature_importance (calling ensemble_pipeline.feature_importance throws an error).
As a fix, we should not try to set this attribute for ensemble components! Instead, however, we will need another attribute that ends with an underscore, such as attribute_; this is because partial dependence can only be calculated on fitted components, and scikit-learn considers a component that ends in a underscore as a fitted component.
This issue tracks adding a test for calculating partial dependence with ensembles and resolving this bug :)

Is this a dupe of #2121 ?


Consider adding empty lines between groups of using statements, or respect existing empty lines.

The initial version of sorting usings (#661) did not retain existing empty lines.
If we try to retain existing empty lines then

What happens when usings are moved?
Can the user easily correct any misplaced empty lines?

If we add empty lines automatically

Should they only appear between groups that are sorted? Or between the first part of the namespace?

// between sorted groups
using System.IO.Abstractions;
using System.Text.Json;
using System.Text.Json.Serialization;

using Microsoft.Extensions.Logging;
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.NamingConventions;

// by first part of namespace
using System.IO.Abstractions;
using System.Text.Json;
using System.Text.Json.Serialization;

using Microsoft.Extensions.Logging;

using YamlDotNet.Serialization;
using YamlDotNet.Serialization.NamingConventions;

I'd be happy if it worked like this:

Record which pairs of imports have a space between them.
Sort as is currently done.
For any of the pairs that still exist in the sorted output, reinsert the space between them.

Or, perhaps,

Group by first part of the namespace
Put System first
Put the group that matches the namespace used in the file last
Sort the rest alphabetically.


Changes badge-pill-border-radius variable from 10px to 50 percent to ensure ro…

Hi! Variables.scss reads:
// Use a higher than normal value to ensure completely rounded edges when
// customizing padding or font-size on labels.
$badge-pill-border-radius:    10rem !default;

If we want to ensure completely rounded edges, that value should be 50%. This PR changes it to:
$badge-pill-border-radius:    50% !default;

What do you think?

Doesn't work like that: https://codepen.io/anon/pen/vJezWQ


tracing: add generic infrastructure for collection of tracing data

add generic infrastructure for collection of tracing data.

add samples for tracing framework with the help of ctf format.

add script to receive the ctf data which can be decoded by trace compass.

add usb backend.

Signed-off-by: Wentong Wu<EMAIL_ADDRESS>

To clarify, the situation hierarchically:
Zephyr Trace API
 |-> Segger SystemView
 \-> CTF top. Can compile-select between several bottoms:
      |-> IO bottom: posix (using fwrite)
      |-> IO bottom: buffered (this PR). This buffered CTF bottom is Generic in the sense that further IO bottoms can be implemented (i.e. implementing tracing_backend_api)
      |    \-> tracing_backend_uart.
      \-> IO bottom: Oticon proprietary (memory mapped, unbuffered)

I'd propose changing this PR's name to: "tracing: Add new CTF bottom: Buffered and extensible, optionally formatted".

A welcome change. Good to see another CTF backend. The first public non-posix ctf bottom +1
This PR is not a Framework: This PR plugs into the existing CTF bottom API (which itself is plugged into the Zephyr trace API).
This PR is not 'Generic': It provides for different hardware backends, yes, and comes with a default one (CDC UART). But the common denominator for this PR's CTF bottom is that it is buffered.
So I propose to change 'Generic' to 'Buffered'.

Thanks for the review. @mped-oticon
This tracing framework is not designed for CTF using only, it's used for collecting many kinds of tracing format(front-end), I think CTF is one kind of front-ends, I use the existing CTF format to verify this infrastructure can work well. Maybe later we can add other front-ends on top of it.
For the unbuffered concerns, actually I also consider it before coding(I called it synchronous method), but I'm not sure if it will impact the real time of the system if back-end is slow, so I post the code first to review, if synchronous method is also needed, I will add it also.
And a USB back-end is also under developing.

Also, could this also be used as a LOG or PRINTK backend, instead of being limited to TRACING?

@fnde-ot sure, it can be used as printk or log backend, see tracing_format_string

@fnde-ot sure, it can be used as printk or log backend, see tracing_format_string

Awesome! Then I would suggest to place this in a new subsystem or as a library so it can be used by more than just tracing.

@fnde-ot sure, it can be used as printk or log backend, see tracing_format_string

Awesome! Then I would suggest to place this in a new subsystem or as a library so it can be used by more than just tracing.

@fnde-ot if anything wrong, I apologize for that. I know there is huge improvements needed for the current code, I just want to focus on tracing here, yes, I just try it on CTF on which I have not so much experience, but also I have plan to try percepio tracealyzer, not sure so far CTF can cover the format of tracealyzer and others', so that's the reason why I want to put the code in same level path as ctf.

@fnde-ot if anything wrong, I apologize for that. I know there is huge improvements needed for the current code, I just want to focus on tracing here, yes, I just try it on CTF on which I have not so much experience, but also I have plan to try percepio tracealyzer, not sure so far CTF can cover the format of tracealyzer and others', so that's the reason why I want to put the code in same level path as ctf.

No need to apologize, this is a great contribution and maybe it could stay in the tracing subsystem for now as a CTF bottom (we should still definitely consider it for other subsystems requiring buffered serial output).
As far as percepio tracealyzer goes, I believe the plan was to use CTF output format for that tool, as well as others (tracecompass, kernelshark, ...). See the TSC discussion topic here:
https://lists.zephyrproject.org/g/tsc/topic/trace_visualization/31546422
Maybe @nashif or someone from the TSC should take a look at this PR.

@fnde-ot if anything wrong, I apologize for that. I know there is huge improvements needed for the current code, I just want to focus on tracing here, yes, I just try it on CTF on which I have not so much experience, but also I have plan to try percepio tracealyzer, not sure so far CTF can cover the format of tracealyzer and others', so that's the reason why I want to put the code in same level path as ctf.

No need to apologize, this is a great contribution and maybe it could stay in the tracing subsystem for now as a CTF bottom (we should still definitely consider it for other subsystems requiring buffered serial output).
As far as percepio tracealyzer goes, I believe the plan was to use CTF output format for that tool, as well as others (tracecompass, kernelshark, ...). See the TSC discussion topic here:
lists.zephyrproject.org/g/tsc/topic/trace_visualization/31546422
Maybe @nashif or someone from the TSC should take a look at this PR.

@fnde-ot @mped-oticon if the decision to using CTF to support all kinds of tracing tools has made, I'm fine with your proposals and I will change the place where we put the code, but could you please help review the code again?

and the CI failed because of https://github.com/zephyrproject-rtos/zephyr/issues/21048, not related to this change

@fnde-ot @mped-oticon could you please review this PR again? thanks. I will submit USB backend after this PR is merged.

@fnde-ot @mped-oticon could you please review this PR again? thanks. I will submit USB backend after this PR is merged.

Yep, will do -- but will first happen for me early next week. X-mas parties are imposing themselves ;-)

To clarify, the situation hierarchically:
Zephyr Trace API
 |-> Segger SystemView
 \-> CTF top. Can compile-select between several bottoms:
      |-> IO bottom: posix (using fwrite)
      |-> IO bottom: buffered (this PR). This buffered CTF bottom is Generic in the sense that further IO bottoms can be implemented (i.e. implementing tracing_backend_api)
      |    \-> tracing_backend_uart.
      \-> IO bottom: Oticon proprietary (memory mapped, unbuffered)

I'd propose changing this PR's name to: "tracing: Add new CTF bottom: Buffered and extensible, optionally formatted"

Hmm, unless I am missing something, this hierarchy assumes CTF is the center of the world and every backend we need to support needs to be a CTF bottom?
The intention of this PR was/is to provide a common infrastructure for pushing tracing data to the host, so instead of putting all of this under CTF I would suggest to move the current bottoms (in this case posix to be another backend of this generic infrastructure and not make new backends CTF specific.
Why something as generic as tracing_backend_uart.c or tracing_core.c need to be under CTF and depend on CTF?
I propose to make the native_posix fwrite so called bottom a backend of this new infrastructure and avoid code duplication and make any existing backends available to any new tracing formats we might add.

@nashif

Hmm, unless I am missing something, this hierarchy assumes CTF is the center of the world and every backend we need to support needs to be a CTF bottom?

If you looked at my ASCII tree, it should be clear that isn't the case. Namely, Segger sits next to CTF. So CTF is not "center of the world" (but IMHO it should be). Why would we not consolidate around an {open, efficient, ...} standard like CTF, with a growing ecosystem of tools around it.

The intention of this PR was/is to provide a common infrastructure for pushing tracing data to the host, so instead of putting all of this under CTF I would suggest to move the current bottoms (in this case posix to be another backend of this generic infrastructure and not make new backends CTF specific.

Disagree. Explanation:

Oticon is already pushing data to the host. No buffering needed.
This PR is buffered. We at Oticon do not want that. We're happy with our prioprietary  synchronous memory mapped CTF backend. What this PR is doing is essentially replicating what was done with the logging framework; buffering stuff up until a slow UART has some air to breathe. Not efficient (more cycles wasted per trace), complex (overflow policy anyone?), and has dubious timestamping.
The current CTF infrastructure in Zephyr is already sufficient and efficient, why do you want to wrap it again? The map macro and much of this PR has actually been copied already from the CTF backend.
The current CTF infrastructure in Zephyr is backend agnostic and goes to great lengths to permit any kind of backend. This PR being a good example, even though it replicates some existing macros and idioms.
It does make sense to wrap it again, if one wants to buffer. But not everybody wants that.

Why something as generic as tracing_backend_uart.c or tracing_core.c need to be under CTF and depend on CTF?

Because it would be nice to consolidate on one format-to-rule-them-all? I.e. CTF. And retain one TSDL for that definition along with one CTF top that defines what gets emitted.
But I don't want them there if merely CTF is kconfig chosen, I want them there if CTF_BUFFERED is there.
Secondly, tracing_core.c does sound generic, except it isn't. It assumes specifically a buffered backend. Seems like POSIX fwrite CTF bottom does fine without it.

I propose to make the native_posix fwrite so called bottom a backend of this new infrastructure and avoid code duplication and make any existing backends available to any new tracing formats we might add.

That would mean buffered tracing becomes unavoidable. No thanks.
The current CTF implementation has very little overhead, almost none if you implement it correctly -- compared with this buffered PR.
I don't want to allocate a whole new thread or increase stack size for doing my very efficient synchronous tracing emit. As a result, I regard this PR as imposing itself a bit too much, making assumptions while calling itself 'generic'. Frankly, overruling the thoughts that led to the generality already exhibited by CTF; you can read more about these thoughts in the documentation: https://docs.zephyrproject.org/latest/guides/tracing/ctf.html#i-o-taxonomy

Oticon is already pushing data to the host. No buffering needed.
This PR is buffered. We at Oticon do not want that. We're happy with our prioprietary  synchronous memory mapped CTF backend. What this PR is doing is essentially replicating what was done with the logging framework; buffering stuff up until a slow UART has some air to breathe. Not efficient (more cycles wasted per trace), complex (overflow policy anyone?), and has dubious timestamping.
The current CTF infrastructure in Zephyr is already sufficient and efficient, why do you want to wrap it again? The map macro and much of this PR has actually been copied already from the CTF backend.

@mped-oticon For the MAP marco, yes, I get the idea from you (subsys/debug/tracing/ctf/bottoms/posix/ctf_bottom.h) when try CTF on this infrastructure
, sorry for wrap it, I will put THE MARCO under subsys/debug/tracing/ctf/bottoms/generic/
, but what do you mean for and much of this PR has actually been copied already from the CTF backend ?

That would mean buffered tracing becomes unavoidable. No thanks.
The current CTF implementation has very little overhead, almost none if you implement it correctly -- compared with this buffered PR.

@mped-oticon I agree it's buffered. But not every platform has RTT or your mentioned prioprietary synchronous memory mapped CTF backend, instead most platforms have UART and USB, etc, and I would like to see your very little/none overhead implementation with current code for UART and USB.

That would mean buffered tracing becomes unavoidable. No thanks.
The current CTF implementation has very little overhead, almost none if you implement it correctly -- compared with this buffered PR.

@mped-oticon I agree it's buffered. But not every platform has RTT or your mentioned prioprietary synchronous memory mapped CTF backend, instead most platforms have UART and USB, etc, and I would like to see your very little/none overhead implementation with current code for UART and USB.

No offense meant. I like this PR, thanks for it. It is definitely needed for UART etc.
I just don't want the roles swapped in the future as @nashif indicates, as that would be a shame.
This PR will operate fine being a regular CTF bottom.

@mped-oticon For the MAP marco, yes, I get the idea from you (subsys/debug/tracing/ctf/bottoms/posix/ctf_bottom.h) when try CTF on this infrastructure
, sorry for wrap it, I will put THE MARCO under subsys/debug/tracing/ctf/bottoms/generic/
, but what do you mean for and much of this PR has actually been copied already from the CTF backend ?

I'm fine this PR. Copying is also fine. I was just saying some infrastructure could perhaps be the same, and some copying could have been avoided.

If you looked at my ASCII tree, it should be clear that isn't the case. Namely, Segger sits next to CTF. So CTF is not "center of the world" (but IMHO it should be). Why would we not consolidate around an {open, efficient, ...} standard like CTF, with a growing ecosystem of tools around it.

I am referring to the bottom half and the transport, not the for payload format, why would want to we have all the backends (which are generic and should be payload agnostic) under CTF?
I should be able to create a dummy backend that just uses printf and push the data via UART or whatever, why do I have to go via  the CTF IO bottom ?

Oticon is already pushing data to the host. No buffering needed.

This PR is buffered. We at Oticon do not want that. We're happy with our prioprietary  synchronous memory mapped CTF backend. What this PR is doing is essentially replicating what was done with the logging framework; buffering stuff up until a slow UART has some air to breathe. Not efficient (more cycles wasted per trace), complex (overflow policy anyone?), and has dubious timestamping.

This PR was not trying to replace the existing non-buffered transport, it was merely adding additional backends. We are aware that the buffered approach is not effecient, but on some systems this is the only way possible and is provided as the last resort when no other more effecient backends are available. Where applicable, more efficient backends can be added.

5\. It does make sense to wrap it again, if one wants to **buffer**. But not everybody wants that.

and if you do not want buffering, you are free not to use it and stick with you own implementation, I am not sure what the issue is.

That would mean buffered tracing becomes unavoidable. No thanks.
The current CTF implementation has very little overhead, almost none if you implement it correctly -- compared with this buffered PR.

Why would a native posix fwrite be buffered? This should something that is defined by the backend, you could have backends that would need buffering, others that do not, i.e. write directly to some memory or whatever.
to sum it up, CTF is a tracing format, not a transport, there are other formats, some are less efficient and some maybe more efficient that CTF, there are propitiatory and open standards and making the CTF the one and only is a NO go. Thats why I think it makes lots of sense to seperate transport from format and support different types of both formats and transports, both buffered and non-buffered.


docs(unstable): Point stable-unstable docs to nightly docs

I ran into this in trying to find the "cargo script" docs.  Unless its the rare case of RUSTC_BOOTRAP=1 cargo +stable ..., someone reading about features on +nightly likely would want to see the latest version of this document, so we should help people find it.

@bors r=weihanglo

:pushpin: Commit 8a9bfc05a16073c6eb2b6ecc7dc7cf056773ff98 has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit 8a9bfc05a16073c6eb2b6ecc7dc7cf056773ff98 with merge 52b1ffb915c3c26ce60776d12f9227f4be7f857d...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing 52b1ffb915c3c26ce60776d12f9227f4be7f857d to master...


Implement global state management

Enhancement Description
As discussed in the previous meeting, it would be great to replace the use of flow context variables with global state variables managed by an external source file.
Usage
It can be used to track the state of quantum registers and quantum circuit initialisation, it can also potentially be used to record the states of classical registers and change how measurement nodes work.
Potential Issues

If we manage to implement such global state management, it would be very very useful for the following aspects of the project:

registers initialisation
qc initialisation
classical register node communication
measurement node communication
simulator node 'waiting' phase

Gonna take a look at this.


Don't let RuboCop blow up when removing .rubocop_todo.yml

Running bundle exec rubocop --auto-gen-config will edit .rubocop.yml to inherit from .rubocop_todo.yml. If the latter is removed, subsequently running bundle exec rubocop will result in an unhandled exception.
I think we should emit a warning instead, or worst case exit with an appropriate message.

Expected behavior
Warning: inherited configuration `.rubocop_todo.yml` not found.

(RuboCop continues to run.)
Actual behavior
No such file or directory @ rb_sysopen - /Users/drenmi/Workspace/Rubocop/rubocop/.rubocop_todo.yml

Steps to reproduce the problem

Run bundle exec rubocop --auto-gen-config
Delete the generated .rubocop_todo.yml
Run bundle exec rubocop

RuboCop version
Include the output of rubocop -V:
$ rubocop -V
0.42.0 (using Parser <IP_ADDRESS>, running on ruby 2.3.0 x86_64-darwin15)

Maybe still make it fail with a more clear error message unless the user is running with --auto-gen-config? Only issuing a warning will probably result in more confusion if all of the sudden rubocop is complaining about a bunch of violations that were previously suppressed

Yeah. I'm okay with that too. 😀

I couldn't reproduce the problem.
Rubocop Version
0.42.0 (using Parser <IP_ADDRESS>, running on ruby 2.2.4 x86_64-darwin15)
Steps Followed

Run bundle exec rubocop --auto-gen-config
Delete the generated .rubocop_todo.yml
Run bundle exec rubocop

Tried with ruby 2.3.0 as well.

@absk1317: Something has changed on current master that requires you to manually add the inherit_from to the configuration file.
As evidenced by the message when running bundle exec rubocop --auto-gen-config:

Run rubocop --config .rubocop_todo.yml, or add inherit_from: .rubocop_todo.yml in a .rubocop.yml file.

Something has changed on current master that requires you to manually add the inherit_from to the configuration file.

As far as I know, it's always been like that.
:+1: on failing with an error message pointing to the unsuccessful inherit_form.

Given that this has been approved by Jonas, I'd be happy to work on it, unless anyone else already is?

@jaredbeck: Feel free to have a go at it. Would love to see this in place. 🙂

Fixed by https://github.com/bbatsov/rubocop/pull/4972


Make MiniAccumuloCluster default to using RawLocalFileSystem when not using MiniDFSCluster

RawLocalFileSystem does not do checksumming and sync actually works which makes it a much more reliable testing FileSystem over LocalFileSystem (in which sync is not guaranteed to work as we want it).
We should default to using RawLocalFileSystem (configured via "fs.file.impl" in core-site.xml) and let the user override it if they so choose. This would alleviate having to manually set it in new unit tests/ITs
(originally reported at ACCUMULO-2478)

Greetings, I am a new developer working on Ivan Bella 's team. Please assign this issue to me.

Hi @ron-rico. I will assign the issue to you, to document your interest in working on this task.
However, please be aware that outside affiliations do not grant any special roles or privileges within the community. From the perspective of the Apache Software Foundation, all contributions to Foundation projects are done on an individual voluntary basis. Acceptance of contributions and recognition by the community is done on the merit-worthiness of the contributions, rather than on any personal or professional affiliations a contributor may have.
This is all to say that it is sufficient to express an interest in contributing to a task in order to have an issue assigned to you, regardless of who you work for.
Feel free to ask any questions about how the Foundation, or this project, operates on the developer mailing list.

@ron-rico - take a look at the AccumuloClusterHarness.java class on line 362.
Maybe changing changing the function to:
@Override
  public void configureMiniCluster(MiniAccumuloConfigImpl cfg, Configuration hadoopCoreSite) {
    hadoopCoreSite.set("fs.file.impl", RawLocalFileSystem.class.getName());
  }

might work.

Hi @ctubbsii ,
How would you suggest I test this issue once I integrate my code changes to default to RawLocalFileSystem? Sync is mentioned. Is there a unit test touches this sync function?
Thanks.

DurabilityIT is one such IT. It currently overrides the LocalFileSystem to be RawLocalFileSystem, as do several other tests where it might matter. As long as those ITs pass, that should be sufficient. You need not wait until the ITs pass for you before you submit a PR. I can test the ITs once the PR is created and the change is reviewed.

The class ConfigurableMacBase is extended by many IT classes.
The class ConfigurableMacBase has a protected method called configure that passes properties
for MiniAccumulo configuration and hadoop configuration .
In many instances the IT classes that extend ConfigurableMacBase override the configure method
to define the FileSystem type as RawLocalFileSystem (the hadoop configuration) with the point being that
RawLocalFileSystem had desired test characteristics over LocalFileSystem.(checksum vs no checksum)
It is proposed that defining the ConfigurableMacBase.configure method to use RawLocalFileSystem would
allow any new classes that extend ConfigurableMacBase to use RawLocalFileSystem by default without a need to define an
overriding configure method. Any extending class that prefers LocalFileSystem can create an overriding configure method.

@ron-rico You shouldn't have to touch any code in the test/ module to implement the change, only in the minicluster module. After doing so, it may be possible to remove some redundant configuration that sets it to RawLocalFileSystem in the test/ module, but it shouldn't be necessary to implement.
That said, this one might be more complicated and annoying that I had realized at first:
The constructor for MiniAccumuloClusterImpl takes a MiniAccumuloConfigImpl. In there, there is setup of config files, depending on .useMiniDFS(). I had initially thought it would be simple to add in the fs.file.impl property (similar to @jzgithub1 said above) if the user hadn't already overridden it.
However, I now realize that the way we create the Hadoop configuration, override the config files with additional properties, and retrieve the FileSystem for use by callers of MiniAccumuloCluster, is done inconsistently at various points in time, and this isn't as simple as I had initially thought.
Specifically, in MiniAccumuloClusterImpl's constructor, it doesn't even appear to write out a config file at all for the relevant case (when !useMiniDFS() && !useExistingInstance()), and when getFileSystem() is called, it doesn't use the config files at all, but uses the value of dfsUri variable with a new Configuration(false) (a new empty configuration), which doesn't seem right.
Additionally, when certain properties are overridden, such as in ConfigurableMacBase, it does so by taking the current contents of the config file (if it exists) and clobbers the explicitly set test properties if they have the same name (that's how .addResource() works), rather than the other way around.
So, the code related to this issue is a lot more complicated (and convoluted) than I first believed. It's probably that way to support the various use cases for MiniAccumuloCluster that have evolved over time, but it makes this task harder.
If you'd like to continue trying to implement this one, please feel free to do so... but I'm going to go ahead and remove the clearly incorrect "good first issue" label. :smiley_cat:


Better logging when token generation fails

@alexeykazakov snapshot fabric8-wit image is available for testing.  docker pull docker.io/fabric8/fabric8-wit:SNAPSHOT-PR-1519-1

@sbose78 updated. Take a look.

@alexeykazakov snapshot fabric8-wit image is available for testing.  docker pull docker.io/fabric8/fabric8-wit:SNAPSHOT-PR-1519-2

@alexeykazakov snapshot fabric8-wit image is available for testing.  docker pull docker.io/fabric8/fabric8-wit:SNAPSHOT-PR-1519-3

[test]


docker

@azure-devops-cloud Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.


Hud does not appear

Issue of HUD not appearing is still an issue for me
SWADE 4.2.0
Argon (CORE): 3.1.0
Argon (SWADE): 1.0.4
Error on attempting to open:
foundry.js:655 Error: An error occurred while rendering CoreHUD 38. Cannot read properties of undefined (reading 'value')
[Detected 2 packages: enhancedcombathud-swade(1.0.4), enhancedcombathud(3.1.0)]
at Hooks.onError (foundry.js:654:24)
at 🎁Hooks.onError#0 (libWrapper-wrapper.js:188:54)
at foundry.js:5795:13
Caused by: TypeError: Cannot read properties of undefined (reading 'value')
[Detected 2 packages: enhancedcombathud-swade(1.0.4), enhancedcombathud(3.1.0)]
at get movementMax (argo-swade.min.js:1:23211)
at get movementColor (movementHud.js:26:74)
at SWADEMovementHud.updateMovement (movementHud.js:50:32)
at SWADEMovementHud.render (movementHud.js:82:10)
at async Promise.all (index 1)
at async CoreHUD._renderInner (hud.js:282:9)
at async CoreHUD._render (foundry.js:5844:19)

Dug into the problem. It appears that get movementMax fails based on an actor not having a defined speed. Quick hack fix:
get movementMax() {
//Check to see if this.actor.system.stats.speed.value is undefined and catch it without letting an error occur
if (this.actor.system.stats?.speed?.value == undefined) {
return 0
} else {
return 0 != o ? o / canvas.scene.dimensions.distance : this.actor.system.stats
?.speed?.value / canvas.scene.dimensions.distance
}
}

I cloned the repository and made the changed suggested above and got the HUD to appear.
However, some things don't work. When I click Powers for characters that have them nothing happens. I assume that the powers should pop up like the spells do in the D&D version of the HUD. The console displays the following error when I select a token that has powers:
Uncaught (in promise) TypeError: this.system.range.match is not a function
[Detected 3 packages: system:swade(4.2.0), enhancedcombathud-swade(1.0.4), enhancedcombathud(3.1.0)]
at get range (SwadeItem.ts:115:37)
at get quantity (config.js:723:29)
at SWADEItemButton.getData (itemButton.js:76:27)
at SWADEItemButton._renderInner (component.js:136:33)
at SWADEItemButton._renderInner (itemButton.js:148:17)
at SWADEItemButton.render (component.js:126:20)
at SWADEItemButton.render (config.js:704:29)
at buttonPanel.js:63:63
at Array.map ()
at ButtonPanel._renderInner (buttonPanel.js:63:40)
at async ButtonPanel.render (component.js:126:9)
at async SWADEButtonPanelButton._renderInner (buttonPanelButton.js:55:5)
at async SWADEButtonPanelButton.render (component.js:126:9)

The SWADE system has had some recent changes that might be the reason for that. I'll be back in front of my computer in the third week of January to look into this issue. I'm assuming the Token Action HUD for SWADE is having similar issues that will need some code review and fix.

Yes, you're right. I usually use Token Action HUD and it completely ignores the SWADE Tools versions of the actions. Argon uses the SWADE Tools actions, but the Powers aren't popping up when I click the Powers button.
And thaks for looking at this!


iterative machine

[WIP] This is to see if our concept with IterativeMachine will work. The PR is not yet complete.

I like the mixin, and also there wont be any of the instantiation problems (like we have for trees) because Perceptron is actually not templated..
For now, we left iteration() without parameters (as of yet, labels and features are stored in CMachine and CLinearMachine anyways). Once we refactor things, the design patterns can be applied here.
Asking for a review @vigsterkr @lisitsyn @iglesias

Implementing the IterativeMachine concept with a mix-in seems like a reasonably idea. Nice job!

Cheers !


Analytics feature doesn't work by default (closed 5000, 5601 ports)

Some users complained that it is necessary to open a couple of ports to work with analytics component. At the same time it should work without opening extra ports. At list our internal server doesn't open any ports and the feature works fine. Need to investigate and fix.
Aaron N. Brock @AaronNBrock 11:34
Hey, how do I access the kibana dashboard? Port 5000 doesn't seem to be open? Do I need another docker-compose.override.yml?

Bill Heffelfinger @HeffRaleigh 13:55
@nmanovic Thanks Nikita!
Hey, how do I access the kibana dashboard? Port 5000 doesn't seem to be open? Do I need another docker-compose.override.yml?
I had to open port 9200. Worked fine when I did that.

Bill Heffelfinger @HeffRaleigh 14:52
Hey, how do I access the kibana dashboard? Port 5000 doesn't seem to be open? Do I need another docker-compose.override.yml?
I had to open port 9200. Worked fine when I did that.
Also...had to open 5601 in another install I'm helping with.

Nikita Manovich @nmanovic 15:24
@AaronNBrock , @HeffRaleigh , you don't need to open additional ports. In dashboard you should have a button for analytics. Use it to open kibana app. https://github.com/opencv/cvat/tree/develop/components/analytics

Bill Heffelfinger @HeffRaleigh 15:38
@AaronNBrock , @HeffRaleigh , you don't need to open additional ports. In dashboard you should have a button for analytics. Use it to open kibana app. https://github.com/opencv/cvat/tree/develop/components/analytics
@nmanovic True that Analytics button will be present, but data will not load without errors until you open ports. Happy to take this off-thread and hop on a video conference to demonstrate if you like.

Nikita Manovich @nmanovic 15:44
@HeffRaleigh , it is quite interesting. I have checked on our internal server and we don't open 5601 and 9200 ports. It works in our case. Could somebody post here errors which you can see when ports are closed.

Bill Heffelfinger @HeffRaleigh 15:49

Nothing loads after a long time for Kibana to start up.
This is when 5000 and 5601 aren't open.
Also...this is CVAT on EC2 Linux. Could have something to do with it.
When 9200 isn't open Kibana complains about not being able to access the service from what I recall. When I have time, I'll close 9200, start Kibana, and grab the error message.

Bill Heffelfinger @HeffRaleigh 16:09
Error when 9200 is shut down:
<EMAIL_ADDRESS>Unable to connect to Elasticsearch at http://elasticsearch:9200/.

Adding some color to the above thread.
I had no issues running Analytics up to the point where I went on PTO on 14 June.  I shut down my dev server at that point - which was a t2-medium EC2 instance running Ubunutu 16.0.4.
When I returned to work on 20 June and started my EC2 instance it came up fine but when I went to start CVAT with Analytics, I got an insufficient memory error.  This should have been my first indicator that something had changed with CVAT.  I didn't address it and instead changed my EC2 instance to a t2-large.  CVAT with Analytics started just fine.
While working with a client that is using CVAT, we had problems getting Kibana to run.  I tried opening port 9200.  No luck.  We then opened ports 5000 and 5601.  Analytics started working for them.
I saw the post to Gitter from another community member about Analytics not working for them which started me down this path.
To test, I removed my inbound port rules for ports 9200, 5601 and 5000.  Ran Kibana which opened but couldn't find any data (see screen shot shared previously).  I added port 9200.  Same result.  I added ports 5000 and 5601.  Interestingly, Kibana started and data was present but I had a port 9200 error message - even though it was open to inbound traffic.  I removed it and added it back.  Now Kibana was back to reporting it couldn't find any data.  Tried a server reboot.  No difference.  This is when Nikita filed the bug.
Let me know what I can do to help troubleshoot.  I love Kibana and would like to get it back to a stable state.

Quick update....I made a copy of an AMI I made of my CVAT with Analytics VM prior to leaving for PTO and launched it.  When I started CVAT on that image, it started fine and Analytics, when I clicked on it, started up but I'm still getting this message<EMAIL_ADDRESS>| Unable to connect to Elasticsearch at http://elasticsearch:9200/.
I refreshed Kibana and now I'm not getting any data.  Same behavior as in the original VM post PTO.
Hope this helps!

@HeffRaleigh Could you please post a output from docker info and docker-compose -f <YOUR SET> config commands?

Another update.  One of the clients I work with that uses CVAT is also experiencing this same behavior with Kibana.
Do we have an ETA on the bug fix?

First of all we need to reproduce the issue on our side. I will run a clean version of CVAT and see what is going on. If you can help here it will be awesome.

Docker Info:
Containers: 13
 Running: 6
 Paused: 0
 Stopped: 7
Images: 166
Server Version: 18.09.5
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-1085-aws
Operating System: Ubuntu 16.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.858GiB
Name: ip-172-31-39-116
ID: WKYR:ORBA:7KEV:MED2:5F5B:JLSV:EG5O:7QU7:VHCH:AJ55:YJUX:I3UJ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: heffelw
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support

@HeffRaleigh Could you please post a output from docker info and docker-compose -f <YOUR SET> config commands?

@nmanovic What do you mean by <YOUR SET>

As an aside...the client I'm working with is running an EC2 instance that is Ubuntu 18.0.4.

@HeffRaleigh Could you please post a output from docker info and docker-compose -f <YOUR SET> config commands?

@nmanovic What do you mean by <YOUR SET>

I mean the set of docker-compose files used to build CVAT, for example docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml config

@HeffRaleigh Could you please post a output from docker info and docker-compose -f <YOUR SET> config commands?

@nmanovic What do you mean by <YOUR SET>

I mean the set of docker-compose files used to build CVAT, for example docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml config

Got it.  Heading off to grab that now.  Thx!

Here you go.  Output from docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml -f docker-compose.override.yml config:
WARNING: The no_proxy variable is not set. Defaulting to a blank string.
WARNING: The https_proxy variable is not set. Defaulting to a blank string.
WARNING: The http_proxy variable is not set. Defaulting to a blank string.
services:
  cvat:
    build:
      args:
        DJANGO_CONFIGURATION: production
        TF_ANNOTATION: "no"
        USER: django
        WITH_TESTS: "no"
        http_proxy: ''
        https_proxy: ''
        no_proxy: ''
        socks_proxy: ''
      context: /home/cvat
    container_name: cvat
    depends_on:
      cvat_db:
        condition: service_started
      cvat_redis:
        condition: service_started
    environment:
      ALLOWED_HOSTS: .amazonaws.com
      CVAT_SHARE_URL: Mounted from /mys3bucket
      DJANGO_LOG_SERVER_HOST: logstash
      DJANGO_LOG_SERVER_PORT: 5000
      DJANGO_LOG_VIEWER_HOST: kibana
      DJANGO_LOG_VIEWER_PORT: 5601
      DJANGO_MODWSGI_EXTRA_ARGS: ''
      no_proxy: kibana,logstash,
    image: cvat
    ports:
    - 8080:8080/tcp
    - 80:8080/tcp
    restart: always
    volumes:
    - cvat_data:/home/django/data:rw
    - cvat_keys:/home/django/keys:rw
    - cvat_logs:/home/django/logs:rw
    - cvat_models:/home/django/models:rw
    - cvat_share:/home/django/share:rw
  cvat_db:
    container_name: cvat_db
    environment:
      POSTGRES_DB: cvat
      POSTGRES_USER: root
    image: postgres:10.3-alpine
    networks:
      default:
        aliases:
        - db
    restart: always
    volumes:
    - cvat_db:/var/lib/postgresql/data:rw
  cvat_elasticsearch:
    build:
      args:
        ELK_VERSION: 6.4.0
      context: /home/cvat/components/analytics/elasticsearch
    container_name: cvat_elasticsearch
    image: cvat_elasticsearch
    networks:
      default:
        aliases:
        - elasticsearch
    restart: always
    volumes:
    - cvat_events:/usr/share/elasticsearch/data:rw
  cvat_kibana:
    build:
      args:
        ELK_VERSION: 6.4.0
      context: /home/cvat/components/analytics/kibana
    container_name: cvat_kibana
    depends_on:
      cvat_elasticsearch:
        condition: service_started
    image: cvat_kibana
    networks:
      default:
        aliases:
        - kibana
    restart: always
  cvat_kibana_setup:
    container_name: cvat_kibana_setup
    depends_on:
      cvat:
        condition: service_started
    entrypoint:
    - bash
    - wait-for-it.sh
    - elasticsearch:9200
    - -t
    - '0'
    - --
    - /bin/bash
    - wait-for-it.sh
    - kibana:5601
    - -t
    - '0'
    - --
    - /usr/bin/python3
    - kibana/setup.py
    - kibana/export.json
    environment:
      no_proxy: elasticsearch,kibana,
    image: cvat
    volumes:
    - /home/cvat/components/analytics/kibana:/home/django/kibana:ro
    working_dir: /home/django
  cvat_logstash:
    build:
      args:
        ELK_VERSION: 6.4.0
        http_proxy: ''
        https_proxy: ''
      context: /home/cvat/components/analytics/logstash
    container_name: cvat_logstash
    depends_on:
      cvat_elasticsearch:
        condition: service_started
    image: cvat_logstash
    networks:
      default:
        aliases:
        - logstash
    restart: always
  cvat_redis:
    container_name: cvat_redis
    image: redis:4.0.5-alpine
    networks:
      default:
        aliases:
        - redis
    restart: always
version: '2.3'
volumes:
  cvat_data: {}
  cvat_db: {}
  cvat_events: {}
  cvat_keys: {}
  cvat_logs: {}
  cvat_models: {}
  cvat_share:
    driver_opts:
      device: /mys3bucket
      o: bind
      type: none

Thanks!

You're welcome.  I noticed yesterday but didn't mention that I now know why 9200 had to be opened for my initial Kibana install to work - elasticsearch needed it for some reason.
Let me know if you want me to share my AMI with you.  Happy to make a copy and share it with your AWS account if you're willing to share your AWS account with me.

Hello all!  Any progress with this bug?  Thanks!

Hello @nmanovic !  Any progress on fixing this bug?  Thanks!

@HeffRaleigh , no progress so far. Our team is overloaded at the moment. Need help here for sure.

Thanks for the update @nmanovic
Success is good.  CVAT has made a big difference.  I hope you get some additional help soon!  I'm trying to do my part in helping on the forum.  Hope that isn't hurting more than it is helping.

@nmanovic Hello Nikita!  I've been away from CVAT for a while working with commercial tooling partners.  I'm back!
I just did a clean install of CVAT on EC2/Ubuntu 16.0.4 to see if I could figure out if the S3 mount was causing an issue.  So...out of the box without the S3 mount, Kibana is still hosed.

I saw the above image and thought it might mean something.

@nmanovic the problem still exists with cvat 1.2. Even after fresh install (following the instructions at https://github.com/openvinotoolkit/cvat/blob/develop/components/analytics/README.md) Kibana still shows nothing at all. No tasks, no users ,.... It says "No results found".

@fbrokanlou
I suppose there is cause of your issue:
cvat_kibana_setup    bash wait-for-it.sh elasti ...   Exit 1
Please attach logs from this container docker logs cvat_kibana_setup > logs.txt

i have the same problem
cvat_kibana_setup ... Exit 1
logs is
wait-for-it.sh: waiting for elasticsearch:9200 without a timeout wait-for-it.sh: elasticsearch:9200 is available after 0 seconds wait-for-it.sh: waiting for kibana:5601 without a timeout wait-for-it.sh: kibana:5601 is available after 0 seconds Updating dashboard named 7e8996e0-c23d-11e8-8e1b-758ef07f6de8 Updating visualization named d92524b0-c25c-11e8-8e1b-758ef07f6de8 Updating search named 9397f350-c23e-11e8-8e1b-758ef07f6de8 Updating visualization named 3ade53d0-c23e-11e8-8e1b-758ef07f6de8 Updating visualization named 1ec6a660-c244-11e8-8e1b-758ef07f6de8 Updating index-pattern named ec510550-c238-11e8-8e1b-758ef07f6de8 Updating visualization named 65918380-c244-11e8-8e1b-758ef07f6de8 Updating visualization named 543f6260-c25c-11e8-8e1b-758ef07f6de8 Updating visualization named 31ac2d60-c25b-11e8-8e1b-758ef07f6de8 Updating visualization named 7f637200-d068-11e8-9320-a3c87be2b433 Updating dashboard named 22250a40-c25d-11e8-8e1b-758ef07f6de8
i run
docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml up -d from develop branch e41c301

which version is stable in which the analytics works? can you give me commit?

@antonioomsk I don't see any errors in your log and it looks ok. d33daa1669925c42c789cf4b786fa9c4453337cb works for our instances

ok, i tested this commit, next command ran
docker-compose -f docker-compose.yml  -f .\components\analytics\docker-compose.analytics.yml down
docker-compose -f docker-compose.yml  -f .\components\analytics\docker-compose.analytics.yml up -d

cvat_kibana_stup container exited 0 logs looks ok as logs up.

but i didn't see any results

docker-compose ps looks ok
docker-compose ps
   Name                 Command               State          Ports
--------------------------------------------------------------------------
cvat         /usr/bin/supervisord             Up      8080/tcp
cvat_db      docker-entrypoint.sh postgres    Up      5432/tcp
cvat_proxy   /docker-entrypoint.sh /bin ...   Up      <IP_ADDRESS>:8080->80/tcp
cvat_redis   docker-entrypoint.sh redis ...   Up      6379/tcp
cvat_ui      /docker-entrypoint.sh ngin ...   Up      80/tcp

@antonioomsk I'll check on the clear Ubuntu

@azhavoro  i rebuilded last release 1.3
added build config in docker-compose because docker hub didn't find new version
docker-compose.yaml
  cvat:
    container_name: cvat
    image: openvino/cvat_server
    restart: always
    depends_on:
      - cvat_redis
      - cvat_db
    build:
        context: .
        args:
          http_proxy:
          https_proxy:
          no_proxy: nuclio,${no_proxy}
          socks_proxy:
          USER: 'django'
          DJANGO_CONFIGURATION: 'production'
          TZ: 'Etc/UTC'
          CLAM_AV: 'no'

them i checked analitycs , and i saw metrics
look like

i think problem was localy, i didn't know wich kind problem.
Can you push docker image in docker hub release 1.3 ?

I cant open kibana page. How can solve this problem?

The webpage at http://<IP_ADDRESS>:5061/ might be temporarily down or it may have moved permanently to a new web address.

@azhavoro I cant open kibana page. How can ı solve this problem?

The webpage at http://*****:5601/ might be temporarily down or it may have moved permanently to a new web address.

@arcibiyo  https://github.com/openvinotoolkit/cvat/blob/develop/cvat/apps/documentation/user_guide.md#analytics

Hi,
I have a similar problem.
After cvat upgrading to version 1.2 analytics doesn't work correctly.
Events that happened after the upgrading aren't displayed
What I tried to do:

fixing of cvat_libana_setup configuration https://github.com/openvinotoolkit/cvat/pull/2659 and then rebuilding
upgrading to 1.3 and develop versions (2021-04-29)

When I deployed CVAT 1.3 on the other host, this problem isn't appear.
Link to the logs and error screenshots:
https://drive.google.com/drive/folders/1SDDRHJpO1Rnvj4H_xjvRTIwr5v2RBNdf?usp=sharing

I don't see complains about analytics for a long time. I will close the issue as outdated.


feat: poetry build

Hi @dzmitry-lahoda , thanks for opening an issue! Can you elaborate a bit?

make this package to be poetry package, so can so poetry add, install.

The package is already pip installable through pypi. Does that not allow you to install it through poetry?

I do not use pypi. I build from scratch all things.

That seems like a really specific usecase, probably unique to you. I don't think that warrants us moving from setuptools to poetry.
Or are there other benefits of moving to poetry that I'm not seeing here?
Mind if I ask you why you want to build everything from scratch btw?

Poetry is like Cargo. Cargo is cool.
Poetry has best reproduce ability and repeat ability in python ecosystem, and yet flexible. These all people want. Poetry repacing setuptools, setuptools is legacy.
So Nix likes Poetry, and I like Nix. So all Poetry packages are very good for Nix. Nix builds from scratch.

Oh, Python gets Rusted fast, I guess ypur saw it. Cargo is same as Poetry. And also with open system  like blockchain, the is nor trust, so only option to make dependency graph to the end traceable. Blockchain likes Rust. Any supply chain attack costs millions.
So, Poetry is future of Python.

setuptools is dead tbh

@nanne-aben  thank you.  so i use poetry because it is the best out of build options in python. it is reproducible determinstic with integration of project with virtual env shell. that is default in rust, haskell, go. python get it, hope python people will get it too:)


cleanup cloudshell relevant resource on failed state

/kind feature

requeue depend on job update event
cleanup the cloudshell relevant resource on failed state

Signed-off-by: calvin0327<EMAIL_ADDRESS>

/lgtm


Schema Registry

Schema Registry
Introduction
The schema registry is a service to keep track of of the schemas that are hosted and the clients including their queries.
The schema registry has the following goals:

Prevent breaking schema changes to the currently used set of clients.

Prevent invalid queries from being deployed.

Manage environments and control which schema/client is deployed to which environment.

Define validation policies for schemas and clients per environment.

Control which queries are still in use and what parts of the schema are in use.

Control of which schemas a gateway is made up.

Prevent schema changes from breaking stitched schemas.

Cofigure white-listed queries through the schema registry.

Task 1 - Create the initial service
Create the initial service that lets you manage and explore environments.
## Task 2 - Manage Schemas
Add support for managing and exploring schemas.
Task 3 - Manage clients
Add support for managing and exploring clients and queries.
Task 4 - Introduce CLI to upload schemas and queries.
Introduction of a CLI tool that can be used in the CI pipeline to upload a schema/query to the schema registry. If a schema/client breaks the tool will fail.
Task 5 - Introduce environment policies.
Allow to set different policies and rules per environment.
Task 6 - Introduce whitelisting middleware
Introduce a new query middleware that whitelists queries based on the schema registry.
Task 7 - Add stitching support
Add gateway knowledge to the registry. The registry now can track which schema is made up of which domain schemas. Add rules that can pre-stitch incoming schemas and ensure that schema changes do not break the schema.

This article also has nice ideas that we should consider:
https://medium.com/@__xuorig__/the-rise-of-graphql-overambitious-api-gateways-2c6f054e80a1

We have moved this issue to the cloud repro.


No module named 'mmdet.ops'

Thank you for your sharing. I am studying this  project . My local mmdetection is the newest version. And there is no ops module in  mmdetection and this project. It occurred with error
from mmdet.ops import ContextBlock, GeneralizedAttention ModuleNotFoundError: No module named 'mmdet.ops . Any resolution is appreciated

The mmdetection version may be mmdetection-2.1.0

Have you solved the issue?


Deploy using git instead of Patch so that we get the git prereceive v…

…alidation back and have a longer timeout on the operation
Credit to @AskAlice for this git code, I just cleaned up some errors/testing mostly.

potentially, could we make it a flag to use the old deploy if customers run into issues?

potentially, could we make it a flag to use the old deploy if customers run into issues?

If they run into issues they could downgrade sectionctl. I'm hesitant to add the option because ideally, we don't want to support both. I've been able to deploy well from Linux, and assuming you have been testing via Windows I'm feeling like these are good changes.

interesting output from the git hook pre-receive.

theoretically we should also make sure to remove the repo from the temp folder when we're done with it.


gitlab: 12.5.5 -> 12.6.0

Motivation for this change
Update GitLab to the latest version.
Gitaly is problematic in this release for two reasons:

The version used in the gitlab repo is not tagged and as a result its commit id is used instead
A new permission check on hook files checks for write permission when only read and execute are needed.

I'll report these upstream when I have time, unless someone else does it before me ;)
Things done
Tried a few things manually in a VM:

Clone, commit and push to a repo
Remove and add ssh keys, making sure the executable path referenced in authorized_keys is correct

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @flokli @globin @roblabla @fpletz @krav @petabyteboy

@GrahamcOfBorg test gitlab

Updated to latest security update.
@GrahamcOfBorg test gitlab

Updated to the latest security update.

The issues are reported; I've updated the description with links.

Merged - especially considering the security fixes, let's not delay this any further.

backported to 19.09 in dbc79093be0, 9fc0570071a, 18469aca5a6, 7ce433cf062, c93501dc786, fd7ce831d4b, 7201d239204.


Add minimum size for a batch

If only 3 or fewer files would be in a batch, upload them individually to minimize server overhead.

Looks good besides the test failures.


Support domain names in URIs.

If the URI has a netlocation that is a domain name, we should resolve it and check if it's local.

Code is already in production.


Overhaul Windows UI

Right now, there's a few issues that lead to massive issues with the Windows UI. This will be a status update, with incremental fixes being displayed.

A few issues so far:

#11
#15
Close button in QTabWidget is pixelated on Windows
0.1ex borders and styles are ignored on Windows, and are ugly. Need to be increased to 0.2ex.
QProgressBar has poor alignment, and needs an increase in the right border.
QSpinBox icons themes are duplicated and cause competing styles when hovered.

Steps include:

Adding more examples to example.py, to catch these issues earlier.

Bug fixes that have been made so far:

QScrollBars now look nice and rounded at all resolutions, on all UIs.
Close Button has been renovated, uses a more rounded style to avoid pixelation.
QTreeView now has branching that works in a cross-platform manner.
QProgressBar has been enhanced to better scale, however, in a few dependent contexts, still has issues.
QSpinBox and QDoubleSpinBox now works as expected out-of-the-box
Padding for QComboBox is a lot better (but still slightly buggy on Windows).
Borders have been updated so they properly display on Windows.

Additional testing has been added by adding the single.py and example.py scripts with extensive options, to test a variety of resolutions, font size, Qt styles (such as Windows, Fusion, etc.) to try to test in a cross-platform manner with 1-variable optimizations.
Major enhancements are being made.

Additional bugs that have been fixed:

An unwanted blue outline on the bottom right when both scrollbars were displayed has been removed.
QComboBox issue with with the indicator overlapping the text has been fixed.
The QSlider has become more circular, and more responsive.

There's also a migration from ex, which is a relatively unused CSS unit, to em, which has been shown to reduce a lot of scaling bugs. This is incremental, and will take time.

The QTabView has been renovated and looks much better.

Closed, see the gallery for extensive, beautiful UIs on Windows.


[ARD:mediathek] individual videos and series download not working (Unable to download JSON metadata: HTTP Error 410: Gone)

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.07.06 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required

Region
Germany
Provide a description that is worded well enough to be understood
Downloading other videos and shows from the ARD Mediathek works as intended. This is the only series I found that causes this error for both downloading the whole playlist and individual videos. There was a similar issue, but the provided solution doesn't work here, so I opened a new one.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', 'https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ?isChildContent']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[b532a3481] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19042-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 6.0-full_build-www.gyan.dev (setts), ffprobe 6.0-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: 5ff3e702171a50175c34397494e2d18ce35d771c2110b1e59bd173ec2fb352aa
yt-dlp is up to date<EMAIL_ADDRESS>[debug] Using fake IP <IP_ADDRESS> (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ?isChildContent
[ARD:mediathek] Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ: Downloading webpage
[ARD:mediathek] Downloading media JSON
ERROR: [ARD:mediathek] None: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 710, in extract
  File "yt_dlp\extractor\ard.py", line 275, in _real_extract
  File "yt_dlp\extractor\ard.py", line 28, in _extract_media_info
  File "yt_dlp\extractor\common.py", line 1064, in download_content
  File "yt_dlp\extractor\common.py", line 1028, in download_handle
  File "yt_dlp\extractor\common.py", line 898, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 855, in _request_webpage

  File "yt_dlp\extractor\common.py", line 837, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3863, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 569, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone

I've just spotted some more links with the same error:

https://www.ardmediathek.de/video/serien/wapo-berlin/hr-fernsehen/YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm
https://www.ardmediathek.de/video/serien/in-aller-freundschaft-die-jungen-aerzte/hr-fernsehen/ZTdiOTY2NjktYWRlOS00NjFiLTgwYmEtNDIzZjNmYzdjNWNl
https://www.ardmediathek.de/video/serien/herr-hesselbach-und-das-juwel/hr-fernsehen/YWQ5Zjk4MzAtOWMxYy00ODAxLWFhZjktOGJhNmU1YWZjZTQ4
https://www.ardmediathek.de/video/serien/das-fest-6-oder-die-laubenpieper/hr-fernsehen/Y2RiMDJlZmYtOTkwNS00ZjAxLWI1MzYtNzI2ZTg1NjY4OGY2
...

They are all from https://www.ardmediathek.de/sendung/serien/Y3JpZDovL2hyLW9ubGluZS8zODIyMDEwOQ.
The easy visible difference:

[ARD:mediathek] extractor for those failing links.
[ARDBetaMediathek] extractor for all normally working links.

Die Ratgeber vom 24.07.2023
https://www.ardmediathek.de/video/die-ratgeber/die-ratgeber-vom-24-07-2023/hr-fernsehen/Y3JpZDovL2hyLW9ubGluZS8yMDc5NTM
$ curl 'https://api.ardmediathek.de/public-gateway' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.ardmediathek.de' --data-binary '{"query":"query { playerPage(client: \"ard\", clipId: \"Y3JpZDovL2hyLW9ubGluZS8yMDc5NTM\" ) { title broadcastedOn } }"}'
{"data":{"playerPage":{"title":"Die Ratgeber vom 24.07.2023","broadcastedOn":"2023-07-24T16:45:00Z"}}}

Good!
Broadcasted one day later:
Die Ratgeber vom 25.07.2023
https://www.ardmediathek.de/video/die-ratgeber/die-ratgeber-vom-25-07-2023/hr-fernsehen/NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl
$ curl 'https://api.ardmediathek.de/public-gateway' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.ardmediathek.de' --data-binary '{"query":"query { playerPage(client: \"ard\", clipId: \"NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl\" ) { title broadcastedOn } }"}'
{"errors":[{"message":"invalid clipId: NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl","locations":[{"line":1,"column":9}],"path":["playerPage"],"extensions":{"code":404}}],"data":{"playerPage":null}}

Bad.
So, ARD mediathek refuses to serve recently published videos through public-gateway.

Hi,
here the simular issue
https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fernsehen/OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw
~$ ./yt-dlp_linux  https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fernsehen/OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fer...MzUtMGY1NjNjMGM2ZTMw
[ARD:mediathek] OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw: Downloading webpage
[ARD:mediathek] 94636404: Downloading media JSON
ERROR: [ARD:mediathek] None: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

The original problem URL was a playlist page that should have been handled by the ARDBetaMediathek extractor but wasn't because the extractor expected a final / (problem 1). But then the playlist items aren't suitable() for the extractor (problem 2) -- they match ARDMediathek:
$ yt-dlp -v -F 'https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent'
[debug] Command-line config: ['-v', '-F', 'https://www.ardmediathek.de/sendung/g
arfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), erro
r utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version<EMAIL_ADDRESS>[812cdfa06] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: de4cf77ec
[debug] Python 3.9.16 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-g
libc2.23 (OpenSSL 1.1.1u  30 May 2023, glibc 2.23)
[debug] exe versions: ffmpeg 4.3, ffprobe 4.3
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2019.11.28, secretstorage
-3.2.0, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[debug] Using fake IP <IP_ADDRESS> (DE) as X-Forwarded-For
[ARDBetaMediathek] Extracting URL: https://www.ardmediathek.de/sendung/garfield/
Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent
[ARDBetaMediathek] [Playlist] garfield: Downloading JSON metadata
[ARDBetaMediathek] [Playlist] garfield: Downloading JSON metadata
[download] Downloading playlist: garfield
[ARDBetaMediathek] Playlist garfield: Downloading 96 items of 96
[download] Downloading item 1 of 96
ERROR: No suitable extractor (ARDBetaMediathek) found for URL https://www.ardmed
iathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk
...

Forcing ARDMediathek gives 404 (problem 3). Checking the playlist page, the URLs should have a penultimate /kika/:
-https://www.ardmediathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk
+https://www.ardmediathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/kika/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk

The corrected URL now gives 410 (problem 4).
Looking at the wapo-berlin page, the hydration JSON contains this URL https://api.ardmediathek.de/page-gateway/pages/ard/item/YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm?embedded=false&mcV6=true in this structure:
[
  [URL, {
    "url": URL,
    ...
    "data": {
      /* this  is the JSON object is that the URL returns */
    },
    ...
  }]
]

This is a new page and API structure that isn't handled by any existing ARD extractor module (problem 5).
If it's the case that pages handled by ARDMediathek expire regularly (there are no permanent test pages), that extractor can probably be removed or replaced.
Similarly, no test video pages work for ARDBetaMediathek. If no-one can find a valid video page for ARDBetaMediathek, both Mediathek extractors can be replaced with a new one.
Parsing the new page structure (trial yt-dl code) gives this (and similarly for the Garfield kika link):
[info] Available formats for YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm:
format code  extension  resolution note
SD           mp4        640x360    [de] 1200k
SD_480p      mp4        960x540    [de] 1600k
HD_Ready     mp4        1280x720   [de] 3200k
Full_HD      mp4        1920x1080  [de] 5000k
hls-543      mp4        480x270    [de]  543k , avc1.4d401e, 50.0fps, mp4a.40.2
hls-841      mp4        640x360    [de]  841k , avc1.4d401f, 50.0fps, mp4a.40.2
hls-1260     mp4        960x540    [de] 1260k , avc1.4d401f, 50.0fps, mp4a.40.2
hls-2154     mp4        1280x720   [de] 2154k , avc1.640020, 50.0fps, mp4a.40.2
hls-3456     mp4        1920x1080  [de] 3456k , avc1.64002a, 50.0fps, mp4a.40.2 (best)

The sort is dubious: probably tbr from the mp4 format URLs is a peak bit-rate while the HLS formats are showing AVERAGE-BANDWIDTH from the manifest.

I’ve just experienced the same issue with KIKA URL‘s.
https://www.kika.de/emmas-chatroom/videos/zwei-netzreisende-1
Maybe mediathekview can be helpful ?
https://github.com/mediathekview/MediathekView

It's not the same page structure. The quoted page is 404 but a valid page (https://www.kika.de/emmas-chatroom/videos/zwoelf-das-tagebuch-100, say) is still handled by the outstanding PR #5788 for Kika.

Can not download, which is only available for another 3 hours until midnight:
https://www.ardmediathek.de/video/filme/die-drei-tage-des-condor-oder-thriller/kika/MGUzZTE3ZGItYzQ4Yi00NzgxLWFjMDktODA0ZjNjNTM0ZDM4
(But https://www.zdf.de/filme/spielfilm-highlights/salt-102.html is possible, even if it is not late enough for the age restricted movie, I know ZDF is not ARD.)

arm Version
yt-dlp_linux_armv7l https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[debug] Command-line config: ['-v', 'https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>(linux_armv7l_exe)
[debug] Python 3.8.0 (CPython armv7l 32bit) - Linux-4.4.35-armv7l-with-glibc2.7 (OpenSSL 1.1.1  11 Sep 2018, glibc 2.37)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.22.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Using fake IP <IP_ADDRESS> (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[ARD:mediathek] ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5: Downloading webpage
[ARD:mediathek] 12822193: Downloading media JSON
ERROR: [ARD:mediathek] Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: Gone>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
File "yt_dlp/extractor/common.py", line 715, in extract
File "yt_dlp/extractor/ard.py", line 275, in _real_extract
File "yt_dlp/extractor/ard.py", line 28, in _extract_media_info
File "yt_dlp/extractor/common.py", line 1069, in download_content
File "yt_dlp/extractor/common.py", line 1033, in download_handle
File "yt_dlp/extractor/common.py", line 903, in _download_webpage_handle
File "yt_dlp/extractor/common.py", line 860, in _request_webpage
File "yt_dlp/networking/_urllib.py", line 410, in _send
File "urllib/request.py", line 531, in open
File "urllib/request.py", line 640, in http_response
File "urllib/request.py", line 569, in error
File "urllib/request.py", line 502, in _call_chain
File "urllib/request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "yt_dlp/YoutubeDL.py", line 4051, in urlopen
File "yt_dlp/networking/common.py", line 114, in send
File "yt_dlp/networking/_helper.py", line 204, in wrapper
File "yt_dlp/networking/common.py", line 325, in send
File "yt_dlp/networking/_urllib.py", line 415, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 410: Gone
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "yt_dlp/extractor/common.py", line 847, in _request_webpage
File "yt_dlp/YoutubeDL.py", line 4070, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 410: Gone

Intel Version:
./yt-dlp_linux -v https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[debug] Command-line config: ['-v', 'https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[b634ba742] (linux_exe)
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-6.1.0-13-amd64-x86_64-with-glibc2.36 (OpenSSL 3.1.3 19 Sep 2023, glibc 2.36)
[debug] exe versions: ffmpeg 5.1.3-1 (setts), ffprobe 5.1.3-1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.43.2, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Using fake IP <IP_ADDRESS> (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[ARD:mediathek] ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5: Downloading webpage
[ARD:mediathek] 12822193: Downloading media JSON
ERROR: [ARD:mediathek] Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: Gone>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
File "yt_dlp/extractor/common.py", line 715, in extract
File "yt_dlp/extractor/ard.py", line 275, in _real_extract
File "yt_dlp/extractor/ard.py", line 28, in _extract_media_info
File "yt_dlp/extractor/common.py", line 1069, in download_content
File "yt_dlp/extractor/common.py", line 1033, in download_handle
File "yt_dlp/extractor/common.py", line 903, in _download_webpage_handle
File "yt_dlp/extractor/common.py", line 860, in _request_webpage
File "yt_dlp/networking/_urllib.py", line 410, in _send
File "urllib/request.py", line 525, in open
File "urllib/request.py", line 634, in http_response
File "urllib/request.py", line 563, in error
File "urllib/request.py", line 496, in _call_chain
File "urllib/request.py", line 643, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "yt_dlp/YoutubeDL.py", line 4051, in urlopen
File "yt_dlp/networking/common.py", line 114, in send
File "yt_dlp/networking/_helper.py", line 204, in wrapper
File "yt_dlp/networking/common.py", line 325, in send
File "yt_dlp/networking/_urllib.py", line 415, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 410: Gone
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "yt_dlp/extractor/common.py", line 847, in _request_webpage
File "yt_dlp/YoutubeDL.py", line 4070, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 410: Gone


Added adoptlogo on the header

Fixed: #56
Signed-off-by: Yuehan-Lin<EMAIL_ADDRESS>

Please also change the header background to #152935


Sorting of property options is case sensitive

PHP Version
8.2
Shopware Version
6.4
Expected behaviour
The alphanumeric sorting should be case insensitive:

Alpha
beta
Gammar
delta

Actual behaviour
But is instead case sensitive:

Alpha
Gammar
beta
delta

How to reproduce
Add filterable group options with capital and non capital letters.
Fix by using sorting function strnatcasecmp instead.
Used:
https://github.com/shopware/platform/blob/8c941d78dc7d933a6ec977195abde3994254209b/src/Core/Content/Property/PropertyGroupCollection.php#L59
https://github.com/shopware/platform/blob/8c941d78dc7d933a6ec977195abde3994254209b/src/Core/Content/Product/SalesChannel/Detail/ProductConfiguratorLoader.php#L174

Hey @mitelg @alexanderschornberg
I want to work on this issue, plz assign this to me

Another thing: strnatcasecmp won't be enough tho, because its not sorting special chars, like umlauts. You need to convert special chars to its normal letter representation beforehand, with something like:
$nameA = iconv('utf-8', 'ascii//TRANSLIT', $a->getTranslation('name');
$nameB = iconv('utf-8', 'ascii//TRANSLIT', $b->getTranslation('name');
return strnatcmp($nameA, $nameB);

[public] Automated response: This issue is linked to the internal issue https://shopware.atlassian.net/browse/NEXT-37839.
[created from NEXT-37839, comment 488368]


Latent diffusion

latent diffusion training

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


AMQP-754 Expose AbstractRabbitListenerContainerFactory's adviceChain

This allows adding additional advice if another piece of code like
Boot's SimpleRabbitListenerContainerFactoryConfigurer already set the
advice chain.

Added a {{this}} ref and test assertion with slightly updated commit message: please inform me what, if anything, you're still missing in the message.
The factory API closely mirrors the actual listener container methods and doesn't add convenience methods anywhere else. Also, the use case is a bit esoteric as the current docs don't even inform developers to what they're advising when adding an advice chain, you have to read the source code for that, and even then you only need this when you have more than one place where you want to add advice using the factory. Therefore I figured adding an adder method here would probably be overkill and a diversion of the overall API style.

Yes, fully agree with your explanation.
Merging with simple polishing about your name to the @author list.
My main concern for commit message was about the header line length - not more than 50 symbols.
"Why?" is explained in the mentioned article.
Not critical, but lets to avoid similar comments and discussions in the future.

Merged as https://github.com/spring-projects/spring-amqp/commit/5c1c61f1a37313066fa90dc97a3bb596a58e23ac and cherry-picked to 1.7.x as https://github.com/spring-projects/spring-amqp/commit/ebb1dfe0a028806f30d7580aad891271d3b5fbd7
Thank you, @jkuipers , for contribution!
Looking forward for more!


feat: set the institutional domain

Purpose
Make it usable outside the alunos.uminho.pt domain.
What has been added
Swap will now look for an env variable called STUDENT_INSTITUTIONAL_MAIL_DOMAIN (didn't come up with a better name, looking forward to suggestions :smile:), and sets it as the e-mail address of students who register on the platform. By default, it will be set to what had been hard-coded until now (alunos.uminho.pt).
What could be added
The possibility to define different regex patterns to validate a student_number. It is currently hardcoded as /^(a|pg)[0-9]+$/i in app/Providers/AppServiceProvider.

Hi @alves-luis,
Thank you for your PR!
I see the relevance of this change.
I would like to keep tests, seeders and factories unchanged, since the configuration value is not important for tests case.
IMO, this should be more flexible than it is with a configuration variable.
For instance, by means of adding a setting property for collecting multiple valid email domains.
This should be implemented in the future. :+1:


IMU sensor values not updating

Hey,
I checked the point cloud and IMU using these commands. The point cloud seems to be working correctly, but the IMU values are not updating. What could be causing this issue?
Launch: ros2 launch unitree_lidar_ros2 launch.py
Check point cloud: rviz2 -d src/unitree_lidar_ros2/rviz/view.rviz
Check IMU: ros2 topic echo /unilidar/imu

Considering the high likelihood of a faulty product, I'll proceed with a return and address the issue further after that.


Allow support for custom ILogger during testing

Right now, the output of a custom logger is ignored in the textual trace dumped by the tester when a bug is found. Reported by @ankushdesai.

This is now supported in the latest release (https://github.com/p-org/PSharp/releases/tag/1.6.9), and this doc describes how the new logger works.


Added backfill

Added backfill action that allows lower priority jobs to run when higher priority jobs cannot run but still hold resources
Tested with added unit tests, e2e tests and local manual test

signed CLA

btw, please also open a PR on design doc.

and please also append the review comments last time and highlight how those comments were addressed.

@k82cn  Design doc of backfill and starvation prevention can be found in PR 821.

/ok-to-test

rebased is done.

still working on a few issues. the e2e tests from the previous rollback in the master branch (not related to this PR) wrecked a mess. attempting to fix it now at least for backfill.


<EMAIL_ADDRESS>has not been published

The latest version is still 2.0.4 but, even ignoring the tags, 3.0.0 is not available to install.

Done. The auto-publish from CI failed for some reason.


Font Change W20.css

The current font was rather difficult to read for people with visual impairments and hard to skim-read as well, so I proposed changing it to Linik Sans because it's simple, visually consistent, and clear.

Submission Checklist
Required

[x] The pull request title clearly contains the name of the sheet I am editing.
[x] The pull request title clearly states the type of change I am submitting (New Sheet/New Feature/Bugfix/etc.).
[x] The pull request makes changes to files in only one sub-folder.
[x] The pull request does not contain changes to any json files in the translations folder (translation.json is permitted)

New Sheet Details

The name of this game is: <   >  (i.e. Dungeons & Dragons 5th Edition, The Dresden Files RPG)

The publisher of this game is: <   > (i.e. Wizards of the Coast, Evil Hat)

The name of this game system/family is: <   > (i.e. Dungeons & Dragons, FATE)

[ ] I have followed the Character Sheets Standards when building this sheet.

[x] This sheet has been made on behalf of, or by, the game's publisher.
[ ] This game is not a traditionally published game, but a copy of the game rules can be purchased/downloaded/found at: <   >
[ ] This sheet is for an unofficial fan game, modification to an existing game, or a homebrew system.

Changes / Description

Character Sheet Info Roll20 Internal Use only.


fix: Update risk label second feature flag

There's a request for a second feature flag until the backend is sorted out.
The new feature flag is ocp-advisor-ui-upgrade-risks.
Should show the label only if it's enabled. To enable this feature both upgrade risks feature flags have to be enabled or in case of preview/beta env only this flag has to be enabled.

:tada: This PR is included in version 1.42.5 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


fix(dropdown): inside label full size when no option is selected

Same as previous PR but made to old main.
Describe pull-request
Dropdown with label inside is not rendering the label properly when no option is selected. Not changed for new Tegel components yet
Solving issue
Github issue https://github.com/scania-digital-design-system/sdds/issues/440 and Azure ticket AB#2321
How to test

Go to storybook
Check dropdown label inside and see if it looks correct

When i use label position inside and manually select an option it looks like this for me:
(tested in our storybook)

Great catch!


Updates yarn.lock

Description
Updates yarn.lock
Addresses CVE-2024-4068
Category
Maintenance
Check List

[ ] New functionality includes testing
[ ] New functionality has been documented
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@DarshitChanpura can we update the title/description of this PR that we are addressing the braces CVE?

@derek-ho Anything blocking this PR from getting merged?

@derek-ho Anything blocking this PR from getting merged?

Nope it looks good to me now


[WIP] Add deform_conv2d as implemented in torchvision

Add Deformable Convolution Layers v1/v2
Help would be welcome, I've been looking at the kernel so long I can no longer see my mistakes. Getting a few more pairs of eyes on it would be helpful.
Checklist

[x] Confirmed that run-checks all script has been executed.
[x] Made sure the book is up to date with changes in this PR.

Related Issues/PRs
A lot of resnet variants currently replace most conv2d layers with deform_conv2d. See torchvision docs
Changes
Adds the relatively popular deform_conv2d layer initially implemented in torchvision in 2021.
Testing

Tested with comparison to pytorch output
All tests passing

Currently blocked on atomics being added to cubecl

Finally done and ready to review


[BUG] Volumes stuck in Detached State after 1.5.1 to 1.6.0 upgrade

Describe the bug
After updating from 1.5.1 to 1.6.0 via Helm and now also updating the engine, none of my volumes work. Related to #7915?
To Reproduce
I updated from 1.5.1 to 1.6.0 and then updated all volume engines.
Expected behavior
Working volumes after the update.
Support bundle for troubleshooting

Sent via E-Mail.
Environment

Longhorn version: 1.6.0
Impacted volume (PV): all but for example pvc-e3100eff-8337-4c3a-b0d5-14d7202ab048
Installation method Helm
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: v1.27.6+k3s1

Number of control plane nodes in the cluster: 3
Number of worker nodes in the cluster: 3

Node config

OS type and version: Debian 12
Kernel version: 6.1.0-13-amd64
CPU per node: 6
Memory per node: 32GB
Disk type (e.g. SSD/NVMe/HDD): 1TB M.2 NVMe
Network bandwidth between the nodes (Gbps): 1Gbps

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): Baremetal
Number of Longhorn volumes in the cluster: 20

Additional context

That solved my problem: https://github.com/longhorn/longhorn/issues/7915#issuecomment-1947448942


chore: bdd-sbhg-1592568509 to 0.0.1

chore: Promote bdd-sbhg-1592568509 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


memory leak.

I use:
sanic 0.6
python-socket.io
aioredis
aiohttp client
sqlalchemy
As my users continue to enter, my thread memory continues to increase. Until the thread crashes.
When i use aiohttp +python-socket.io ,this problem does not appear.
@seemethere Can you help me?

yeah, memory leak is annoying, it also appears in tornado, aiohttp, or other python webframeworks. use gunicorn could solve your problem by recycling workers/processes.

@larry-dev just to confirm you're using websockets right? I could reproduce this with websockets but not without them.

@miguelgrinberg any chance you might be able to look into this? I added a line to log memory usage and number of python objects (class, dict, etc...) in the ws route to verify, but wasn't able to fix it with my attempts.

yes。
i use socket.io ,aioredis,aiomysql

I'll see if I can figure it out. Couple of questions:

What Python version has this been seen?
How many users does it take to crash?

@r0fls what did you do to see this leak? I'm using the websocket.py example. I added a memory report in the HTTP route and then connected to this app and hit refresh several times to generate a lot of connects and disconnects

Memory usage started at 19.5MB
After 10 refreshes, it was at 21.3MB
After 20 refreshes it was at 21.7MB
I did something like 50 more refreshes, at still got 21.7MB usage

This looks fairly normal to me and not indicative of a memory leak. Have you tested differently?

@miguelgrinberg   I run in production environment, with about more than 300 users.
my environment:

python 3.6.1
sanic 0.6

@larry-dev how did you determine there is a memory leak? The more users concurrently join your application, it is expected that memory will increase. There would be a leak only if when those 300 users are replaced by another 300 users memory continues to increase significantly.

@miguelgrinberg My project is constantly having users joining and exiting. Memory will continue to increase at this point. About 18 hours or so, the python thread collapsed.

@miguelgrinberg I did see memory seem to go up linearly, but admittedly didn't test as extensively (10 connections). What I did do that was pretty convincing was make a route something like this:
from sanic import Sanic
from sanic.response import file
from sanic.config import Config

app = Sanic(__name__)

@app.route('/')
async def index(request):
    return await file('websocket.html')

@app.websocket('/feed')
async def feed(request, ws):
    while True:
        objgraph.show_most_common_types()
        data = 'hello!'
        print('Sending: ' + data)
        await ws.send(data)
        data = await ws.recv()
        print('Received: ' + data)

if __name__ == '__main__':
    app.run(host="<IP_ADDRESS>", port=8000, debug=True)

As you close the browser or tab and reopen it (or however you're testing), you should see the number of some item (I think it was dict) continue to increase endlessly. I think it was going up 5, then removing one upon closed connection or something. Similar with some other object(s) ¯\(ツ)/¯

@ahopkins maybe time to close this issue ?

It might be preferable to avoid cyclic references between per-request objects. While this will be collected by the GC eventually, some might disable GC, and even if enabled, it is nicer to have that memory freed instantly when the request ends.
In recent years there have been no reports of leaks, so I am closing this with a note that directionality of object ownership should be observed by core developers, in an effort to avoid reliance on GC. Shared data should go in objects that do not refer back to their owners. AFAIK this is already pretty well handled in Sanic.


Update apache-ambari-usage.md

@sreekzz : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Closing this PR as forked the main repo.


Improve archive options

This gives three options for viewing archived clients, 'Non-archived' is the default, advisors can also choose to view all clients, as well as just archived clients.

Coverage decreased (-0.06%) to 91.889% when pulling 12ed2b3982654ba6fa0103b0469be6632571c95a on improve-archive-options into b76d69dfbcfafdb4946ee7adf704b91ae5c3b86e on staging.


Color for table rows is applied to some table-like environments.

Table row coloring is used in some environments like array or cases where no coloring is expected.
\begin{equation}
\ref{eq:pos-clamping-complete}
x_d \notin [x_{min}, x_{max}] \Rightarrow
\begin{cases}
& v_d \leftarrow 0\\
x_d < x_{min} \Rightarrow & x_d \leftarrow x_{min}\\
x_d > x_{max} \Rightarrow & x_d \leftarrow x_{max}\\
\end{cases}
\end{equation}

I noticed this problem, too.
When only using pmatrix environments, everything works fine, but as soon as I introduce a table (I'm converting from markdown to latex/pdf), it also colors the previously fine matrices.


Database connect details object - Not accepted anymore

In sqlAgent version '9.8.1' I was able to specify the connection details for the database in the form of an object like so:
{
  host: 'ipAddress'
  ,port: 1111
  ,user: 'username'
  ,password:''
}

However after updating to 9.9.1, pg mode_module is throwing an error, stating that it expects a string for the URL ("TypeError: Parameter "url" must be a string, not object"). Connection details is a string<EMAIL_ADDRESS>) is accepted.
Is this feature still supported? Is it unique to MySQL database connections?

Hi @Aidan-Chey, I have fixed it. Please try to download new version of SQL Agent from NPM 9.9.2. Give me a feedback.

Hi @petersirka I can confirm that this issue has been corrected. I can once again use objects to specify connection details.
Just a heads up I do have an additional (potentially much larger) bug i'm experiencing regarding dynamic database connections. However I didn't get the time to write a comprehensive issue report about it.


Fix Prometheus metrics

We were using the global Prometheus registry, but controller-runtime
(now?) uses its own, separate registry. This change uses
controller-runtime's registry instead.
This change also uses the most recent OpenCensus Prometheus exporter and
removes a bunch of dead code, including direct access of the Prometheus
golang client. As a result, there are a bunch of changes in the vendor
directory.
Tested: directly inspected the /metrics endpoint on the HNC pod via
wget.


Add complaint events to activity feed

closes TryGhost/Team#2248

Added ability to show spam emails in member activity feed

This is looking good @lenabaidakova and appears to work as expected!
I was curious about the "Unsubscribed from Ghost Inc." event that appears right after. Is that something we added or is it just a natural side effect of what happens when the email gets suppressed?

@moreofmorris , we added it previously. Member is unsubscribed from newsletters when we receive spam event from MG

https://github.com/TryGhost/Team/issues/2248#issuecomment-1318175125


Cannot debug 410 Server Gone errors

When I upload multiple files using uppy, sometimes it fails to upload some of the files with http error code 410. If I retry the failed files they upload successfully.
I'm still investigating the issue, might be caused by uppy but the problem with tus-php is, it's impossible to understand what's going on without looking at the source code since the actual reason of the error is not logged or returned back to the client.

I also try to deal with 410 (gone) errors. I hava an setup which worked fine on localhost using xampp. Then I moved it on a server into a docker container. At first everything worked fine, but after a few days now I can't upload any file at all and even retrying doesn't help. I use Uppy and it only shows that I have an 410 error.
I use just file based caching. What would be steps to check the cache for entries?

Never mind, I found the cache file in /vendor/ankitpokhrel/tus-php/.cache and it was not readable since I copied the /vendor folder

@Amerlander Glad you found the solution. Can we close this issue if you feel you have found your solution?

Yes, for me it can be closed