What should happen when an FD is closed while a thread is blocked on it?

For both epoll and eventfd, the question arises what happens when an FD is closed while a thread is blocked on it. Currently, we throw an "unsupported" error. That's not a great solution -- we should figure out what POSIX guarantees in that case, and implement that.
So, what does POSIX say?
Cc @tiif

POSIX says nothing on this. In theory it counts as a race condition, as you can't ever say another thread has blocked on this FD already (there's no general way to ensure that has happened yet, it may be right before the thread actually executes the blocking op).
In practice, once actually blocked, the thread just never wakes.
So... beyond going to reporting a race condition error instead of an unsupported error, I see no good course of action. We picked unsupported as it's unspecified what the behaviour is, and in practice it's definitely not "return the same result as if the FD were closed before the read/write call was done"

you can't ever say another thread has blocked on this FD already (there's no general way to ensure that has happened yet, it may be right before the thread actually executes the blocking op).

I can't parse this, what do you mean?

reporting a race condition error

That would mean making it UB. The question is, is it UB?

@the8472 do you know if there is any required behavior from POSIX here?
"Never wake up the thread" does seem like a reasonable implementation we could copy. Maybe we should issue some sort of warning though? We don't have much in the way of warnings in Miri currently, but after all this is almost certainly a bug.

I can't parse this, what do you mean?

When thread A is running on a code path that will inevitably block on FD, and thread B is closing FD, we are in an unclear state wrt what happens to thread A. Depending on the scheduler it

is already blocked
it will invoke the blocking op after thread B closes FD
it's already accessing FD internals while B is closing FD

It's unspecified if the third option can happen.
The first option leads to the thread staying blocked forever in practice (this may also depend on the type of FD, pipes seem in theory more likely to unblock than e.g. a file)
The second option is well specified

I decided we should pick unsupported as this is always a bug according to all stackoverflow and other forum answers I could find. These often state that it's a race condition, but without any references. On a more high level, a FD should only be closed if there are no outstanding uses of it. The docs of close say the word "race condition"
   Multithreaded processes and close()
   It is probably unwise to close file descriptors while they may be
   in use by system calls in other threads in the same process.
   Since a file descriptor may be reused, there are some obscure
   race conditions that may cause unintended side effects.

   Furthermore, consider the following scenario where two threads
   are performing operations on the same file descriptor:

   (1)  One thread is blocked in an I/O system call on the file
        descriptor.  For example, it is trying to write(2) to a pipe
        that is already full, or trying to read(2) from a stream
        socket which currently has no available data.

   (2)  Another thread closes the file descriptor.

   The behavior in this situation varies across systems.  On some
   systems, when the file descriptor is closed, the blocking system
   call returns immediately with an error.

   On Linux (and possibly some other systems), the behavior is
   different: the blocking I/O system call holds a reference to the
   underlying open file description, and this reference keeps the
   description open until the I/O system call completes.  (See
   open(2) for a discussion of open file descriptions.)  Thus, the
   blocking system call in the first thread may successfully
   complete after the close() in the second thread.

https://www.man7.org/linux/man-pages/man2/close.2.html

"unsupported" indicates a missing feature in Miri, not a bug in the program. The error text even explicitly says that. So according to what you said we should emit a different error.

The epoll man page says the following:

Will closing a file descriptor cause it to be removed from all epoll interest lists?
Yes, but be aware of the following point.  A file descriptor is a reference to an open file description (see open(2)).  Whenever a file descriptor is  duplicated  via  dup(2),  dup2(2),  fcntl(2)
F_DUPFD,  or  fork(2),  a  new file descriptor referring to the same open file description is created.  An open file description continues to exist until all file descriptors referring to it have
been closed.
A file descriptor is removed from an interest list only after all the file descriptors referring to the underlying open file description have been closed.  This means that even after a  file
descriptor that is part of an interest list has been closed, events may be reported for that file descriptor if other file descriptors referring to the same underlying file description remain open.
To prevent this happening, the file descriptor must be explicitly removed from the interest list (using epoll_ctl(2) EPOLL_CTL_DEL) before it is duplicated.  Alternatively, the  application  must
ensure that all file descriptors are closed (which may be difficult if file descriptors were duplicated behind the scenes by library functions that used dup(2) or fork(2)).

In other words the fd will be removed from the interest list if there is no dup of the fd still around. If it is the last fd in the interest list, epoll_wait will block until another thread inserts a new fd in the interest list and this new fd triggers an event I think, just like when calling epoll_wait when the interest list was empty from the start.

The epoll case I was talking about above is about what happens when the epoll FD itself is closed while someone blocks on it.
What happens when an FD in the interest list is closed, is a separate question, but I think that is already handled properly. epoll identifies interest by the pair of FD number and underlying file description object; if the object is closed then even if the number gets reused the interest will never be triggered.

@tiif do we have a test for this situation of "the FD registered with epoll got closed but there's a duplicate FD for the same file description so events are still reported"? I guess that means events are reported with the old FD number, which is actually quite confusing since it won't refer to the relevant object any more...

@the8472 do you know if there is any required behavior from POSIX here?

epoll is a linux-specific api. poll is the posix standard one, but I haven't seen any explicit documentation whether a wakeup will be triggered by a concurrent close or not.

The epoll case I was talking about above is about what happens when the epoll FD itself is closed while someone blocks on it.

File descriptions are refcounted resources. So I assume that closing the file descriptor will only decrease the count by one but the ongoing polling syscall will own another token and so the underlying description will remain open at least until the syscall returns.

I really don't want to replicate per-FD-kind behaviour on close while blocked, with platform dependent shenanigans. It's why I opted for the unsupported, since we do not support the (potentially guaranteed) platform or FD specific behaviour. The situations are always an edge case that you probably don't want to encounter anyway. We could add a new Undesirable error to clearly signal that difference to regular unsupported which usually mean we invite adding an impl for. A race condition error signals too much that this is UB, when it may be defined for some platforms and some FDs

I don't think we should have per-FD-type logic, either.
We could raise a "deadlock" error, since it does mean that this thread will definitely never be woken up.

So I assume that closing the file descriptor will only decrease the count by one but the ongoing polling syscall will own another token and so the underlying description will remain open at least until the syscall returns.

So that would mean even if the epoll FD is closed, future wakeups on threads waiting on that FD still happen, since the underlying data is still present. Makes sense. We should probably implement that, then.

do we have a test for this situation of "the FD registered with epoll got closed but there's a duplicate FD for the same file description so events are still reported"?

yes https://github.com/rust-lang/miri/blob/158973f0817dbcac3cb2c832086a4f341d5f2ad8/tests/pass-dep/libc/libc-epoll-no-blocking.rs#L339-L383

So that would mean even if the epoll FD is closed, future wakeups on threads waiting on that FD still happen, since the underlying data is still present. Makes sense. We should probably implement that, then.

If we plan to support "closing fd while blocking on it" operation for epoll, should we do the same for socketpair/pipe and eventfd?

Those are different situations though -- one is blocking on read/write, the other is blocking on a dedicated operation (epoll_wait) that merely uses the FD for bookkeeping. epoll_wait does not get woken up by actions on its own FD, it gets woken up by actions on other FDs.

epoll_wait does not get woken up by actions on its own FD, it gets woken up by actions on other FDs.

I expect it'd be similar for socketpair/pipe too. If the thread blocked because it tried to read from an empty socketpair/pipe, writing to it's peer fd would unblock it.

Yeah but we are not talking about a closed peer fd here.

The blocked fd could be closed too, the case that I am thinking of is:

thread1 blocks on read for one of the socketpair fd, fd1
thread2 tries to close fd1
thread3 writes to the peer fd of fd1, and this can unblock thread1

which of the following should happen here?

throw error for the close in step2
allow the close and allow thread1 to be unblocked
throw miri error as we don't want to support this behavior

Since in practice the thread will not be unblocked, we should not do 2. So my proposal would be to throw a "deadlocked" error either when the FD is closed with a non-empty wait list, or when the thread wakes up and finds that its FD is gone.

Sure, this makes sense. It would be nice if we could throw a more specific error than deadlock if we encounter this scenario. The reason is if someone manages to make this work on a certain platform in a real system, but it fails in miri with a deadlock error, it'd be helpful to know that it encountered this specific scenario. This way, we can rule out the possibility of it being a miri implementation bug early on.

Yeah ideally the error would point at both the "close", and the thread that can now no longer wake up.
And then for epoll - we should probably make the Weak ref into an Rc so that the unblock callback can definitely access the epoll instance?

And then for epoll - we should probably make the Weak ref into an Rc so that the unblock callback can definitely access the epoll instance?

Yea, and perhaps we need to close it again to properly destruct it from the global machine state after we finish using it.

I am currently implementing blocking unnamed socket, so I can take care of the pipe/socketpair part together with that. I opened #4065 for epoll.

@tiif figured out that these cases are unreachable anyway, so we should turn them into panics, not deadlock errors.


update json emitter to better handle errors

Resolves #3952 and relevant for #3947
This changes the json emitter to emit the full json output for the formatting results as part of emit_footer whereas the current approach incrementally emitted incomplete json snippets as each file was processed and used emit_header and emit_footer to try to sandwich it all together into valid json (which was not always viable as described in #3952).

Also, I like that we now have a newline at the end of the output.

Me too. I feel like one day it would be good to see if we can also have the errors emitted as part of the json/xml output instead of stderr when the user specifies the json or checkstyle emit mode


Improve test coverage

coverage: 89.726% (+23.3%) from 66.429%
when pulling 61a92e7fb915de558d878d39e480cb55b55eafb0 on tests
into c95a9ad3bb8efffe8cd3ccac3c60759b58cb242a on master.

coverage: 89.726% (+23.3%) from 66.429%
when pulling 61a92e7fb915de558d878d39e480cb55b55eafb0 on tests
into c95a9ad3bb8efffe8cd3ccac3c60759b58cb242a on master.


Issues with automatic discovery on GKE

After setting up permissions on GKE to enable the default service account and creating the necessary roles (I volunteer to add these steps to the readme file) I cannot get the automatic discovery to work.
The issue seems to lie in the fact that when the vernemq.sh script requests the PodList, the spec.hostname and spec.subdomain properties are missing from the Pod descriptions, and so kube_pod_names is always empty. Is there something I'm missing? Could we use spec.nodeName instead of spec.hostname?

@p-mendes thanks for opening the issue!
Do you think that the PodList should have the spec.hostname and spec.subdomain? I mean does it normally? Note, I'm asking as a non-expert in GKE and K8 ;)

As a non-expert myself, I was basing my assumption that the PodList should contain these fields by default, on the behaviour of the vernemq.sh script:
In order for the VERNEMQ_KUBERNETES_SUBDOMAIN and kube_pod_names variables to be correctly set, spec.hostname and spec.subdomain must  be present, otherwise they'll be set to null. I have tried manually setting spec.dnsPolicy to Default and the hostname, but with no success.
I do have spec.nodeName for all pods, and this resolves correctly to the corresponding pod IPs. Could this be an option? Still missing spec.subdomain, but perhaps setting it manually would be a solution.

Ok, let's see, then the problem is already clearer. Maybe rings a bell for @dergraf or @larshesel.
The Astarte folks are great Kubernetes experts , so comparing their scripts might give some insights. For instance this one:
https://github.com/astarte-platform/astarte_vmq_plugin/blob/master/docker/bin/vernemq.sh
Haven't diff'ed the files though.

After a quick overview of the script from Astarte, there don't seem to be any changes as far as the auto discovery mechanism is concerned. They added a big section which seems to add an SSL listener automatically - quite interesting, but for a separate topic :)

There's a small change for the K8 service account in the Astarte script:
https://github.com/astarte-platform/astarte_vmq_plugin/blob/master/docker/bin/vernemq.sh#L19
No idea if this matters here or not.
Hope we can figure this out all together :)
Next step for me is some reading to do on Kubernetes services and discovery.

That change doesn't affect this issue actually. In the meantime you can check this out: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pods-hostname-and-subdomain-fields
In this doc they say that both spec.hostname and spec.subdomain are optional settings that must be defined in the Pod config. However I haven't figured out how to apply these settings on a Deployment... If I set a template.spec.hostname on a Deployment manifest, all pods have the same hostname, and that doesn't make much sense...

Sorry pressed Close accidentally...

OK, I did an attempt using spec.nodeName instead of the FQDN and I got this error:

** System running to use fully qualified hostnames **
** Hostname gke-prod-cluster-0-default-pool-f2b42b87-qxtl is illegal **

This hostame (gke-prod-cluster-0-default-pool-f2b42b87-qxtl) resolves to the actual IP address of the pod in question, so I don't see why a FQDN is necessary?

This is the Erlang cluster complaining, as it is configured for FQDN. (node-name@full-hostname). Do you actually have a nodename? something like VerneMQ@gke-prod-cluster-0-default-pool-f2b42b87-qxtl

Yes, I have changed vernemq.sh to get the list of spec.nodeName for all pods in the cluster. I am using VerneMQ@${kube_pod_name}, where ${kube_pod_name} is a pod hostname, such as gke-prod-cluster-0-default-pool-f2b42b87-qxtl.
Is there a way of disabling the FQDN requirement? In the meantime could you point me in the direction of some example K8S Deployment for launching a VerneMQ Cluster? It would be a great help to see some manifest files that you guys use :grinning:

Seen a blog post on this lately: https://github.com/nmatsui/kubernetes-vernemq
Then the Astarte stuff I pointed at.

@p-mendes please be so kind and tell me about the ClusterRole and ClusterRoleBinding permissions in GKE / enable the default service account, I cannot seem to find the references in the Readme.

Hi @popaaaandrei I'm affraid there are no references. Here are the settings I have used:
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  # ClusterRoles not bound to namespace?
  # namespace: default
  name: pod-reader
rules:
  - apiGroups: [""] # "" indicates the core API group
    resources: ["pods"]
    verbs: ["get", "watch", "list"]
...
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: pod-reader-pod
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: pod-reader
subjects:
  - kind: ServiceAccount
    name: default
    namespace: default

Hope this helps!


Extend QG for tables to Table+Text

jaydesen created the following on Feb 01:
How to design a QG mechanism for Table+text setup i.e. generate questions which needs information from both modalities.
Original issue: https://github.ibm.com/ai-foundation/PrimeQA/issues/18

On Feb 08, jaydesen commented:
One possible baseline is to reduce the QG over table+text to QG over table by augmenting a given table with additional information coming from related text passages. A related paper is : https://aclanthology.org/2021.naacl-main.469.pdf
We are brainstorming over two main questions:
(1) Retrieval Aware QG: How to generate a question that has sufficient information to retrieve "unique" documents
(2) Has multi-hope QG been explored over text data e.g. QG for HotpotQA ?

On Feb 08, saneem-cg commented:
Table+Text question generation looks for a multi-hop question generation pipeline where hops can be both in tables and passages. Initial step is to look for existing methods in multi-hop reading comprehension question generation domain and find methods/tricks which can be borrowed to table+textQA setup.
This comment might be updated with more paper details.
Multi-hop question generation

Gupta et. al. Reinforced Multi-task Approach for Multi-hop Question Generation, COLING 2020 paper
Pan et. al. Semantic Graphs for Generating Deep Questions, ACL 2020 paper
Yu et. al. Low-Resource Generation of Multi-hop Reasoning Questions, ACL 2020 paper
Su et. al. Multi-hop Question Generation with Graph Convolutional Network, EMNLP Findings 2020 paper
Huang et. al. Latent Reasoning for Low-Resource Question Generation, ACL Findings 2021 paper

Table+Text Question Generation

Pan et. al. Unsupervised Multi-hop Question Answering by Question Generation, NAACL 2021 paper

This github page is a good source to find published question generation papers.

On Mar 15, jaydesen commented:
@saneem-cg : put your updates here

Hey @riyazbhat I’m using Zenhub in GitHub, click this link to join my workspace and see other features available in GitHub or download the Zenhub extension and sign up with your GitHub account.
Posted using Zenhub

Extended the QG over Hybrid context by adding

Path sampler for inference
Processor for training


IAR example project compilation errors

While using IAR 8.22.2 i am getting the following Linker Errors for compiling CppUTestTest:
Error[Li005]: no definition for "TestTestingFixture::lineOfCodeExecutedAfterCheck" [referenced from C:\Tools\cpputest-3.8\platforms\iar\Debug\Obj\UtestTest.o]
Error[Li005]: no definition for "TestTestingFixture::checkTestFailsWithProperTestLocation(char const *, char const *, int)" [referenced from C:\Tools\cpputest-3.8\platforms\iar\Debug\Obj\TestUTestMacro.o]
Error[Li005]: no definition for "TestTestingFixture::lineExecutedAfterCheck()" [referenced from C:\Tools\cpputest-3.8\platforms\iar\Debug\Obj\TestUTestMacro.o]
Error[Li005]: no definition for "TeamCityTestOutput::TeamCityTestOutput()" [referenced from CommandLineTestRunner.o(cpputest.a)]

Has this been resolved ?

Can not currently verify this, as i am not using iar 8.22 anymore.

Ok, closing it for now then. Not sure how to reproduce it.


Documentation for the KET 1.6.0 release [skip ci]

Documentation for the KET 1.6.0 release

[x] Cloud provider documentation
[x] Overriding component configuration
[x] Installing with HTTP proxies
[x] Seed registry documentation
[x] Upgrade to 1.6.0 documentation

[x] Etcd now requires block device when using direct-lvm
[x] Disconnected upgrade documentation (you must seed registry before, etc...)
[x] Swap memory and Kubelet

Updated the tag on the AWS cloud provider prereqs list.

LGTM!

Going to rebase and merge.


add parallelism to transmute

Logging is only done if num_jobs is 1 i.e. no parallelism is used. Otherwise it would lead to intermingled logs.


fix(ci): version bump check when new recipe and multiple commits

Fix the check of recipe version bump by the CI, which was failing when adding a new recipe with 2+ commits in the PR. (example: https://github.com/ferdium/ferdium-recipes/pull/359).
We are now using another gh-action for detected changed files, which seems to work ok.
Here are some tests:

Hi @Alphrag, if it seems go to you, you can merge it, I will be away for the week-end.
After the merge, rebasing #360 should resolve the failing CI.

This is exactly the action I thought of when I realised the previous one was not working as expected, thanks @nathanaelhoun for checking was changes were required and testing it.


Creation of website blocker

Aim

What is the objective of the Script
The scripts will block websites that the user wants. Basically there will be some files that will be running in the background of the system and preventing these sites to work.

Details

What the features will your script have
I will be implementing it in python using datetime library

@avinashkranjan will like to work on this as part of swoc

Sure go Ahead @Sloth-Panda


use "exit /b", not "exit"

"exit" terminates the cmd shell that invoked this script.
"exit /b" terminates only this bat script.

Changes look good.  I'll work on getting this in today.


Advanced validation and cleaning

Description

This issue merges some of the features from #195
This PR will allow for a pipelined approach while cleaning and validating a GTFS.
Motivation and Context

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

Test configuration details:

OS: Windows 10
Python version: 3.9.13
Java version: N/A
Python management system: MiniConda

Advice for reviewer

Checklist:

[x] My code follows the intended structure of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules

Additional comments

@r-leyshon Any thoughts on the failing test here?


Use provisional NWIS data?

From @lekoenig in #134:

I've set earliest_date and latest_date as 10/1/1979 and 12/31/2021, respectively. I chose this latest_date because I was under the impression that data records are meant to be approved within 120 days, however, I noticed there's still some provisional data starting in August 2021. We may ultimately just opt to omit provisional data to avoid what we saw in https://github.com/USGS-R/drb-do-ml/pull/78 that considerable chunks of data had been removed from NWIS in between data pulls for several of our sites.

Should we omit provisional data from the model input files?

There are 304 unique days in our dataset with DO records marked as "provisional:"
01473500: 55 days (10/7/2021 - 11/30/2021)
01475530: 87 days (9/7/2021 - 12/2/2021)
01475548: 98 days (8/27/2021 - 12/2/2021)
01480617: 45 days (10/18/2021 - 12/1/2021) now tagged as approved on NWIS
01481500: 19 days (12/13/2021 - 12/31/2021) now tagged as approved on NWIS
As of 11/2/2022, the time series underlying 64 of those records have now been approved but 240 are still listed as "provisional."  Since it's been nearly 12 months since most of those records were collected, I've contacted the PA WSC requesting more information.

I've been checking in on these data periodically, here's an update as of 3/1/2023:
01473500: 55 days (10/7/2021 - 11/30/2021)
01475530: 87 days (9/7/2021 - _12/2/2021) now tagged as approved on NWIS
01475548: 98 days (8/27/2021 - 12/2/2021) now tagged as approved on NWIS
01480617: 45 days (10/18/2021 - 12/1/2021) now tagged as approved on NWIS
01481500: 19 days (12/13/2021 - 12/31/2021) now tagged as approved on NWIS
# 1) 01473500
dat_01473500 <- dataRetrieval::readNWISuv(siteNumber = "01473500", parameterCd = "00300", startDate = "2021-10-07", endDate = "2021-11-30")
unique(dat_01473500$X_00300_00000_cd)
#> [1] "A" "P"
dat_01473500 %>% group_by(X_00300_00000_cd) %>% summarize(n = n())
#> # A tibble: 2 x 2
#>   X_00300_00000_cd     n
#>   <chr>            <int>
#> 1 A                   44
#> 2 P                 5232

# 2) 014755300
dat_01475530 <- dataRetrieval::readNWISuv(siteNumber = "01475530", parameterCd = "00300", startDate = "2021-09-05", endDate = "2021-12-05")
unique(dat_01475530$X_00300_00000_cd)
#> [1] "A"

# 3) 01475548
dat_01475548 <- dataRetrieval::readNWISuv(siteNumber = "01475548", parameterCd = "00300", startDate = "2021-08-25", endDate = "2021-12-31")
unique(dat_01475548$X_00300_00000_cd)
#> [1] "A"

# 4) 01480617
dat_01480617 <- dataRetrieval::readNWISuv(siteNumber = "01480617", parameterCd = "00300", startDate = "2021-10-05", endDate = "2021-12-05")
unique(dat_01480617$X_00300_00000_cd)
#> [1] "A"

#5) 01481500
dat_01481500 <- dataRetrieval::readNWISuv(siteNumber = "01481500", parameterCd = "00300", startDate = "2021-12-05", endDate = "2021-12-31")
unique(dat_01481500$X_00300_00000_cd)
#> [1] "A"

So it looks like 01473500 is the only data that still hasn't been approved.

 Good to know the extent of the provisional data. That said, I'm not sure it will affect us too much. In the lastest runs, I set the end of the test set to 2021-10-01. We could theoretically go through 2022-10-01, but then we'd need to update the gridmet data. The way things are set up, we have to go in 365-day increments, which means we can't end in summer 2022 which is where our current gridmet data ends. I don't know that adding one more year to our test set is really going to give us much anyway, when we already have 6 years of data.
Sorry for not looking at those dates earlier! I guess knowing that we were stopping at 2021-10-01 could have saved you the time to do the analysis.

Oh, OK thanks for confirming those dates! I hadn't realized until yesterday that the NWIS latest_date differed from the test_end_date.
You're right, if we stick with 2021-10-01 we avoid the data that is still tagged as provisional. Before yesterday, I hadn't re-pulled the DO data since 6/11/2022. In the meantime, it looks like some sites took down ~45-90 days worth of data from NWIS, including the sites mentioned within this issue. I guess some data was probably determined to be bad during the records approval process.


Importing the workspace JSON doesn't give the whole skill, dialog is missing.

So I cloned the repository and imported the workspace.json to the skill into my assistant, entities and intents are there, but the dialog has the greeting and anything_else nodes only. I tried to re-submit the workspace many times, same results.

The dialog only has "Welcome" and "anything_else" -- until you expand the Welcome node.  It is very nested.  There is a screenshot in the README showing it expanded.  It's a little old so I am pushing a new one, but the dialog hasn't changed just the UI around it.


add layout to assist infinite scrolling

am-layout="frame"
We have a good idea. Find a way to assist infinite scrolling in such a way to provide header, content, footer within Ambient. This means it needs to work anywhere in the semantic scaffolding.

View codepen example
included in 1.3.4


Add defaultProps in README

In order to make the documentation more clear, I thought that adding the defaultProps in the README could be a good idea, so people know what they don't need to call in the settings

Merged this PR
https://github.com/akiran/react-slick/pull/942


Ignore certain fields on serialization tests

When testing serialization and deserialization we don't serialize all of the properties. This adds a method in TestUtils to give back a list of ignored properties for a specific class.

Coverage remained the same at 61.283% when pulling 4f087429a54ffb40cdef728e66749f3b81b21210 on pbardea:ser-test-fix into 6339bc43b706d91ca1baae895a0ac5108e88fc2c on graphflow:master.

Coverage remained the same at 61.283% when pulling 4f087429a54ffb40cdef728e66749f3b81b21210 on pbardea:ser-test-fix into 6339bc43b706d91ca1baae895a0ac5108e88fc2c on graphflow:master.

Different fix with a different PR by resetting the PropertyIterator.


Segmentation fault in microCDR

I was recently trying to test micro ros client and agent using an app similar to rtt-test demo code, but written in rclc instead. (https://github.com/micro-ROS/micro-ROS-rtt)
I found that after publish and subscribe certain amount of data, the client would failed and showed segmentation fault. My test case was that the client continuously publish a roughly 2k Byte data and subscribe the same topic in each round, and it failed at about 16k th round.
Core was generated by `install/micro_ros_demos_rclc/lib/micro_ros_demos_rclc/rtt_test_host/rtt_test_ho'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f88af8d3187 in ucdr_deserialize_uint8_t (ub=ub@entry=0x7fff024f5380, value=value@entry=0x7fff024f52fa "\t") at /root/microros_ws/src/eProsima/Micro-CDR/src/c/types/basic.c:181
181	        *value = (uint8_t)*ub->iterator;
[Current thread is 1 (Thread 0x7f88af3a7000 (LWP 79974))]
(gdb) bt
#0  0x00007f88af8d3187 in ucdr_deserialize_uint8_t (ub=ub@entry=0x7fff024f5380, value=value@entry=0x7fff024f52fa "\t") at /root/microros_ws/src/eProsima/Micro-CDR/src/c/types/basic.c:181
#1  0x00007f88af8f6a88 in uxr_deserialize_submessage_header (ub=ub@entry=0x7fff024f5380, id=id@entry=0x7fff024f52fa "\t", flags=flags@entry=0x7fff024f52fb "", length=length@entry=0x7fff024f52fc)
    at /root/microros_ws/src/eProsima/Micro-XRCE-DDS-Client/src/c/core/serialization/xrce_subheader.c:15
#2  0x00007f88af8f0891 in uxr_read_submessage_header (ub=ub@entry=0x7fff024f5380, submessage_id=submessage_id@entry=0x7fff024f52fa "\t", length=length@entry=0x7fff024f52fc,
    flags=flags@entry=0x7fff024f52fb "") at /root/microros_ws/src/eProsima/Micro-XRCE-DDS-Client/src/c/core/session/submessage.c:23
#3  0x00007f88af8eea5b in read_submessage_list (session=session@entry=0x7f88afc0ca98 <custom_sessions+632>, submessages=submessages@entry=0x7fff024f5380, stream_id=<EMAIL_ADDRESS>    at /root/microros_ws/src/eProsima/Micro-XRCE-DDS-Client/src/c/core/session/session.c:594
#4  0x00007f88af8eefa2 in read_stream (session=session@entry=0x7f88afc0ca98 <custom_sessions+632>, ub=ub@entry=0x7fff024f5430, stream_id=..., seq_num=<optimized out>)
    at /root/microros_ws/src/eProsima/Micro-XRCE-DDS-Client/src/c/core/session/session.c:580
#5  0x00007f88af8ef1ce in read_message (ub=0x7fff024f5430, session=0x7f88afc0ca98 <custom_sessions+632>) at /root/microros_ws/src/eProsima/Micro-XRCE-DDS-Client/src/c/core/session/session.c:543
#6  listen_message (session=session@entry=0x7f88afc0ca98 <custom_sessions+632>, poll_ms=poll_ms@entry=100) at /root/microros_ws/src/eProsima/Micro-XRCE-DDS-Client/src/c/core/session/session.c:411
#7  0x00007f88af8ef501 in listen_message_reliably (session=session@entry=0x7f88afc0ca98 <custom_sessions+632>, poll_ms=<optimized out>)
    at /root/microros_ws/src/eProsima/Micro-XRCE-DDS-Client/src/c/core/session/session.c:448
#8  0x00007f88af8ef720 in uxr_run_session_until_data (session=0x7f88afc0ca98 <custom_sessions+632>, timeout_ms=100)
    at /root/microros_ws/src/eProsima/Micro-XRCE-DDS-Client/src/c/core/session/session.c:218
#9  0x00007f88afbff38a in rmw_wait (subscriptions=0x556fdbbbcb48, guard_conditions=0x556fdbbbcb60, services=0x556fdbbbcb90, clients=0x556fdbbbcb78, events=0x556fdbbbcba8, wait_set=0x556fdbbbed90,
    wait_timeout=0x7fff024f5670) at /root/microros_ws/src/uros/rmw_microxrcedds/rmw_microxrcedds_c/src/rmw_wait.c:82
#10 0x00007f88afdf4d89 in rcl_wait () from /root/ros2_foxy/install/rcl/lib/librcl.so
#11 0x00007f88afd9aa3f in rclc_executor_spin_some (executor=0x7fff02505d00, timeout_ns=100000000) at /root/microros_ws/src/uros/rclc/rclc/src/rclc/executor.c:974
#12 0x00007f88afd9ac9e in rclc_executor_spin (executor=0x7fff02505d00) at /root/microros_ws/src/uros/rclc/rclc/src/rclc/executor.c:1000
#13 0x0000556fda6d6fc3 in main ()

here's a print screen for easy reading

Please tell me what else I need to provide to solve this, thanks!

Could you provide instructions for replicating this?

Sorry I spend some time to make a simplified code to replicate the situation, here is the code:
`#include <rcl/rcl.h>
#include <rcl/error_handling.h>
#include <rclc/rclc.h>
#include <rclc/executor.h>
#include <std_msgs/msg/string.h>
#include <stdio.h>
#include <unistd.h>
#define ARRAY_LEN 1900
#define RCCHECK(fn) { rcl_ret_t temp_rc = fn; if((temp_rc != RCL_RET_OK)){printf("Failed status on line %d: %d. Aborting.\n",LINE,(int)temp_rc); return 1;}}
#define RCSOFTCHECK(fn) { rcl_ret_t temp_rc = fn; if((temp_rc != RCL_RET_OK)){printf("Failed status on line %d: %d. Continuing.\n",LINE,(int)temp_rc);}}
rcl_publisher_t publisher;
rcl_subscription_t sub;
std_msgs__msg__String pub_msg;
std_msgs__msg__String sub_msg;
int counter = 0;
void timer_callback(rcl_timer_t * timer, int64_t last_call_time)
{
(void) last_call_time;
if (timer != NULL) {
//sprintf(pub_msg.data.data, "Hello from micro-ROS #%d", counter++);
counter++;
//pub_msg.seqno = counter;
pub_msg.data.size = 1900;
RCSOFTCHECK(rcl_publish(&publisher, &pub_msg, NULL));
//printf("I have publish %dth data: "%s"\n", counter, pub_msg.data.data);
}
}
void subscription_callback(const void * sub_msgin)
{
const std_msgs__msg__String * sub_msg = (const std_msgs__msg__String *)sub_msgin;
printf("received %dth data \r\n", counter);
}
int main(int argc, const char * const * argv)
{
rcl_allocator_t allocator = rcl_get_default_allocator();
rclc_support_t support;
// create init_options
RCCHECK(rclc_support_init(&support, argc, argv, &allocator));

// create node
rcl_node_t node = rcl_get_zero_initialized_node();
RCCHECK(rclc_node_init_default(&node, "string_node", "", &support));

// create publisher
RCCHECK(rclc_publisher_init_default(
	&publisher,
	&node,
	ROSIDL_GET_MSG_TYPE_SUPPORT(std_msgs, msg, String),
	"/string_publisher"));

RCCHECK(rclc_subscription_init_default(&sub, &node,
            ROSIDL_GET_MSG_TYPE_SUPPORT(std_msgs, msg, String), "/string_publisher"));

// create timer,
rcl_timer_t timer = rcl_get_zero_initialized_timer();
const unsigned int timer_timeout = 1;
RCCHECK(rclc_timer_init_default(
	&timer,
	&support,
	RCL_MS_TO_NS(timer_timeout),
	timer_callback));

// create executor
rclc_executor_t executor = rclc_executor_get_zero_initialized_executor();
RCCHECK(rclc_executor_init(&executor, &support.context, 2, &allocator));
RCCHECK(rclc_executor_add_timer(&executor, &timer));
RCCHECK(rclc_executor_add_subscription(&executor, &sub, &sub_msg,
            &subscription_callback, ON_NEW_DATA));

// Fill the array with a known sequence
pub_msg.data.data = (char * ) malloc(ARRAY_LEN * sizeof(char));
for(int i=0;i<1899;i++){
	pub_msg.data.data[i] = 'A';
}
pub_msg.data.data[1899] = '\0';
pub_msg.data.size = 1900;
pub_msg.data.capacity = ARRAY_LEN;
//pub_msg.seqno = 0;
//pub_msg.deviceid = 0;
//pub_msg.arrivetime = 0;

sub_msg.data.data = (char * ) malloc(ARRAY_LEN * sizeof(char));
    sub_msg.data.size = 1900;
    sub_msg.data.capacity = ARRAY_LEN;

rclc_executor_spin(&executor);

RCCHECK(rcl_subscription_fini(&sub, &node))
RCCHECK(rcl_publisher_fini(&publisher, &node))
RCCHECK(rcl_node_fini(&node))

}
`

To run the code, please replace the "main.c" in microros_ws/src/uros/micro-ROS-demos/rclc/string_publisher(or subscription)/.
The code contains a timer that publish a 1.9kB string and a subscription that receive the data, both in a same node, and in my case, the fault always occurs when after 16383 th publish (Actually in executor wait as showed in the gdb result above).
Such fault won't happen when timer and subscription is in separate file (or application) and execute individually.
The environment is in a docker, ubuntu 20.04, and ros2 foxy( ros and micro ros should be the newest version because I just update it last week)

I'm going to test this, I'll be back in a while

I have it replicated and I have found a bug in the XRCE-DDS Client. Working on the solution I'll be back in a while.

I have prepared a patch here so this segfault should not appear anymore. But it is still pending to determine the XRCE-DDS Client fragmentation corner case when the sub-message sequence number rollover.
Closing since solved.
Please reopen if the problem still happens when the PR is merged.

Update, also we have found a problem with the middleware history, solved here: https://github.com/micro-ROS/micro_ros_setup/pull/292
Please notify when you can retry your code and everything works.

Hello, Thanks for the help and I have checked out those changes. I will retry codes ASAP. However, I am wondering why this situation only happens when stream->base.history is not power of two?

I had done the test with the new patch, the segmentation fault have disappeared. However, I tried to change the XRCE stream history size to 32 without add lines in input_reliable_stream.c, and the segmentation fault still happened.

As far as I know, the library is implemented in order to handle the reliability of the streams based on these assumptions. Because each XRCE message has a sequence number from 0 to 2^16-1, so having a power of two buffer historic size allow us to assign sequence numbers to historic slots in an optimal way using a modulo operation.
This segmentation fault is happening only when XRCE is fragmenting and the sequence number rolls over. So:

Having a 10 historic size configured was error prune
The changes on input_reliable_stream.c allows to detect when the fragmenting buffer callback fails, preventing the seg fault.
I need to investigate further the case because I guess that something else is happening.

Another solution would be to increase MTU in order to avoid XRCE fragmenting packages.

Thanks for the explanation!


Fourcastnetv2-small output.grib can not be used as the input file

I want to use the output grib file as the input file of the fourcastnet. What I have done is:
1.set --path 'out-{step}.grib' --lead-time = 6. Then there are two grib files, out-0 and out-6.grib. Both are grib2 verison.
2. use ai-models --file 'out-6.grib' --path 'outtest.grib' --lead-time 6 --debug 'fourcastnetv2-small'
ValueError: operands could not be broadcast together with shapes (66,721,1440) (1,73,1,1)
Here we missed some part of the data, should be 73 but not 66. I use grib_ls to check the out-6.grib it shows

but the single level data from cds shows

So, the out-6 is edition 2 while cds data is edition 1. the typeOfLevel and level keys have also changed. Recall ValueError: operands could not be broadcast together with shapes (66,721,1440) (1,73,1,1) the missed 7 dims are 10u 10v 100u 100v 2t msl and tcwv because the changes of the key.
What should be the best way to solve this?

Issue solved. Change output as grib1 file.

@Siming-Liang I don't fully understand how you solved the issue. Can you explain it in more detail?

Issue solved. Change output as grib1 file.

Hi. Could you please elaborate on what exactly you did to resolve the issue? I also do not understand it and would greatly appreciate it!

Issue solved. Change output as grib1 file.

Hi. Could you please elaborate on what exactly you did to resolve the issue? I also do not understand it and would greatly appreciate it!


Upgrade jetty version

Upgrade jetty version to 8.2.0.X

Pull Request Test Coverage Report for Build 5749

10 of 22 (45.45%)  changed or added relevant lines in 6 files are covered.
26 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.07%) to 43.393%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

azkaban-web-server/src/main/java/azkaban/webapp/TriggerPluginLoader.java
0
1
0.0%

azkaban-web-server/src/main/java/azkaban/webapp/AzkabanWebServer.java
0
3
0.0%

azkaban-web-server/src/main/java/azkaban/webapp/WebServerProvider.java
2
5
40.0%

azkaban-common/src/main/java/azkaban/jmx/JmxJettyServer.java
0
5
0.0%

Files with Coverage Reduction
New Missed Lines
%

azkaban-common/src/main/java/azkaban/jobExecutor/AbstractJob.java
2
37.93%

azkaban-common/src/main/java/azkaban/jobExecutor/AbstractProcessJob.java
3
54.64%

azkaban-exec-server/src/main/java/azkaban/execapp/event/JobCallbackRequestMaker.java
21
56.25%

Totals

Change from base Build 5739:
-0.07%

Covered Lines:
14992

Relevant Lines:
34549

💛  - Coveralls


Skeleton annotation task: can't export dataset

My actions before raising this issue

[X] Read/searched the docs
[X] Searched past issues

Without skeletal annotation (or only bbox), I can't succesfully export the data.
Once I annotated the task, upon export as 'COCO Keypoints 1.1' I have the following error:

Expected Behaviour

The annotation and images are exported.
 No
Operating System and version (e.g. Linux, Windows, MacOS): Ubuntu 22.04
Other diagnostic information / logs:
The way I defined the annotations in the project:

[
  {
    "name": "person",
    "id": 22,
    "color": "#c06060",
    "type": "any",
    "attributes": []
  },
  {
    "name": "pose",
    "id": 66,
    "color": "#1446c2",
    "type": "skeleton",
    "sublabels": [
      {
        "name": "head",
        "attributes": [],
        "type": "points",
        "color": "#d12345",
        "id": 67
      },
      {
        "name": "l_shoulder",
        "attributes": [],
        "type": "points",
        "color": "#350dea",
        "id": 68
      },
      {
        "name": "l_elbow",
        "attributes": [],
        "type": "points",
        "color": "#479ffe",
        "id": 69
      },
      {
        "name": "l_hand",
        "attributes": [],
        "type": "points",
        "color": "#4a649f",
        "id": 70
      },
      {
        "name": "r_shoulder",
        "attributes": [],
        "type": "points",
        "color": "#478144",
        "id": 71
      },
      {
        "name": "r_elbow",
        "attributes": [],
        "type": "points",
        "color": "#57236b",
        "id": 72
      },
      {
        "name": "r_hand",
        "attributes": [],
        "type": "points",
        "color": "#1cdda5",
        "id": 73
      }
    ],
    "svg": "<line x1="68.81401824951172" y1="26.254180908203125" x2="46.40599060058594" y2="12.374582290649414" stroke="black" data-type="edge" data-node-from="5" stroke-width="0.5" data-node-to="1"></line><line x1="77.8441162109375" y1="43.4782600402832" x2="66.138427734375" y2="65.0501708984375" stroke="black" data-type="edge" data-node-from="6" stroke-width="0.5" data-node-to="7"></line><line x1="68.81401824951172" y1="26.254180908203125" x2="77.8441162109375" y2="43.4782600402832" stroke="black" data-type="edge" data-node-from="5" stroke-width="0.5" data-node-to="6"></line><line x1="23.997961044311523" y1="28.595317840576172" x2="68.81401824951172" y2="26.254180908203125" stroke="black" data-type="edge" data-node-from="2" stroke-width="0.5" data-node-to="5"></line><line x1="23.997961044311523" y1="28.595317840576172" x2="46.40599060058594" y2="12.374582290649414" stroke="black" data-type="edge" data-node-from="2" stroke-width="0.5" data-node-to="1"></line><line x1="12.292276382446289" y1="47.15719223022461" x2="24.666858673095703" y2="65.88628387451172" stroke="black" data-type="edge" data-node-from="3" stroke-width="0.5" data-node-to="4"></line><line x1="23.997961044311523" y1="28.595317840576172" x2="12.292276382446289" y2="47.15719223022461" stroke="black" data-type="edge" data-node-from="2" stroke-width="0.5" data-node-to="3"></line><circle r="1.5" stroke="black" fill="#b3b3b3" cx="46.40599060058594" cy="12.374582290649414" stroke-width="0.1" data-type="element node" data-element-id="1" data-node-id="1" data-label-id="67"></circle><circle r="1.5" stroke="black" fill="#b3b3b3" cx="23.997961044311523" cy="28.595317840576172" stroke-width="0.1" data-type="element node" data-element-id="2" data-node-id="2" data-label-id="68"></circle><circle r="1.5" stroke="black" fill="#b3b3b3" cx="12.292276382446289" cy="47.15719223022461" stroke-width="0.1" data-type="element node" data-element-id="3" data-node-id="3" data-label-id="69"></circle><circle r="1.5" stroke="black" fill="#b3b3b3" cx="24.666858673095703" cy="65.88628387451172" stroke-width="0.1" data-type="element node" data-element-id="4" data-node-id="4" data-label-id="70"></circle><circle r="1.5" stroke="black" fill="#b3b3b3" cx="68.81401824951172" cy="26.254180908203125" stroke-width="0.1" data-type="element node" data-element-id="5" data-node-id="5" data-label-id="71"></circle><circle r="1.5" stroke="black" fill="#b3b3b3" cx="77.8441162109375" cy="43.4782600402832" stroke-width="0.1" data-type="element node" data-element-id="6" data-node-id="6" data-label-id="72"></circle><circle r="1.5" stroke="black" fill="#b3b3b3" cx="66.138427734375" cy="65.0501708984375" stroke-width="0.1" data-type="element node" data-element-id="7" data-node-id="7" data-label-id="73"></circle>",
    "attributes": [
      {
        "id": 3,
        "name": "track_id",
        "input_type": "select",
        "mutable": false,
        "values": [
          "1",
          "2",
          "3",
          "4",
          "5",
          "6",
          "7",
          "8",
          "9",
          "10"
        ],
        "default_value": "1"
      }
    ]
  }
]

@ConsciousML
Please, provide docker logs cvat_server

Hi @bsekachev,
Here it is:
2022-11-08 13:54:04,112 DEBG 'runserver' stderr output:
[Tue Nov 08 13:54:04.112792 2022] [wsgi:error] [pid 68:tid<PHONE_NUMBER>24864] [remote <IP_ADDRESS>:50008] [2022-11-08 13:54:04,112] INFO cvat.server.task_285: get repository request

2022-11-08 13:54:04,113 DEBG 'runserver' stderr output:
[Tue Nov 08 13:54:04.113083 2022] [wsgi:error] [pid 68:tid<PHONE_NUMBER>24864] [remote <IP_ADDRESS>:50008] INFO:cvat.server.task_285:get repository request

2022-11-08 13:54:04,145 DEBG 'runserver' stderr output:
[Tue Nov 08 13:54:04.145671 2022] [wsgi:error] [pid 68:tid<PHONE_NUMBER>39456] [remote <IP_ADDRESS>:50006] INFO:cvat.client.task_285:{"job_id":240,"task_id":285,"client_id":477228,"name":"Load job","time":"2022-11-08T13:53:59.327000Z","payload":{"duration":881,"frame count":71,"track count":8,"object count":479,"box count":1,"polygon count":0,"polyline count":0,"points count":7,"cuboids count":0,"tag count":0},"is_active":true,"username":"admin"}

2022-11-08 13:54:04,146 DEBG 'runserver' stderr output:
[Tue Nov 08 13:54:04.146084 2022] [wsgi:error] [pid 68:tid<PHONE_NUMBER>39456] [remote <IP_ADDRESS>:50006] INFO:cvat.client.task_285:{"job_id":240,"task_id":285,"client_id":477228,"name":"Fit image","time":"2022-11-08T13:54:00.241000Z","payload":{},"is_active":true,"username":"admin"}

2022-11-08 13:54:04,146 DEBG 'runserver' stderr output:
[Tue Nov 08 13:54:04.146600 2022] [wsgi:error] [pid 68:tid<PHONE_NUMBER>39456] [remote <IP_ADDRESS>:50006] INFO:cvat.client:{"client_id":477228,"name":"Send user activity","time":"2022-11-08T13:54:04.067000Z","payload":{"working_time":3507},"is_active":true,"username":"admin"}

2022-11-08 13:54:25,736 DEBG 'runserver' stderr output:
[Tue Nov 08 13:54:25.736578 2022] [wsgi:error] [pid 68:tid<PHONE_NUMBER>24864] [remote <IP_ADDRESS>:50102] [2022-11-08 13:54:25,736] ERROR django.request: Internal Server Error: /api/tasks/285/dataset

2022-11-08 13:54:25,737 DEBG 'runserver' stderr output:
[Tue Nov 08 13:54:25.736681 2022] [wsgi:error] [pid 68:tid<PHONE_NUMBER>24864] [remote <IP_ADDRESS>:50102] ERROR:django.request:Internal Server Error: /api/tasks/285/dataset

@ConsciousML
Hm, what about docker logs cvat_worker_default?

@bsekachev
2022-11-08 13:54:24,611 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Handling failed execution of job /api/task/285/annotations/COCO Keypoints 1.0

2022-11-08 13:54:24,612 DEBG 'rqworker_default_1' stderr output:
ERROR:rq.worker:Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/datumaro/plugins/coco_format/converter.py", line 765, in apply
    task_conv.save_annotations(item)
  File "/opt/venv/lib/python3.8/site-packages/datumaro/plugins/coco_format/converter.py", line 388, in save_annotations
    elem.update(self.convert_points_object(skeleton))
  File "/opt/venv/lib/python3.8/site-packages/datumaro/plugins/coco_format/converter.py", line 418, in convert_points_object
    elements[self._point_label_to_position[element.label]] = element
IndexError: list assignment index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/rq/worker.py", line 936, in perform_job
    rv = job.perform()
  File "/opt/venv/lib/python3.8/site-packages/rq/job.py", line 684, in perform
    self._result = self._execute()
  File "/opt/venv/lib/python3.8/site-packages/rq/job.py", line 690, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/home/django/cvat/apps/dataset_manager/views.py", line 114, in export_task_as_dataset
    return export(dst_format, task_id=task_id, server_url=server_url, save_images=True)
  File "/home/django/cvat/apps/dataset_manager/views.py", line 81, in export
    export_fn(db_instance.id, temp_file, dst_format,
  File "/home/django/cvat/apps/dataset_manager/task.py", line 766, in export_task
    task.export(f, exporter, host=server_url, save_images=save_images)
  File "/home/django/cvat/apps/dataset_manager/task.py", line 648, in export
    exporter(dst_file, task_data, **options)
  File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 37, in __call__
    f_or_cls(*args, **kwargs)
  File "/home/django/cvat/apps/dataset_manager/formats/coco.py", line 46, in _export
    dataset.export(temp_dir, 'coco_person_keypoints', save_images=save_images,
  File "/opt/venv/lib/python3.8/site-packages/datumaro/util/scope.py", line 158, in wrapped_func
    ret_val = func(*args, **kwargs)
  File "/opt/venv/lib/python3.8/site-packages/datumaro/components/dataset.py", line 1111, in export
    raise e.__cause__
datumaro.components.errors.AnnotationExportError: Failed to export item ('frame_000021', 'default')
Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/datumaro/plugins/coco_format/converter.py", line 765, in apply
    task_conv.save_annotations(item)
  File "/opt/venv/lib/python3.8/site-packages/datumaro/plugins/coco_format/converter.py", line 388, in save_annotations
    elem.update(self.convert_points_object(skeleton))
  File "/opt/venv/lib/python3.8/site-packages/datumaro/plugins/coco_format/converter.py", line 418, in convert_points_object
    elements[self._point_label_to_position[element.label]] = element
IndexError: list assignment index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/rq/worker.py", line 936, in perform_job
    rv = job.perform()
  File "/opt/venv/lib/python3.8/site-packages/rq/job.py", line 684, in perform
    self._result = self._execute()
  File "/opt/venv/lib/python3.8/site-packages/rq/job.py", line 690, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/home/django/cvat/apps/dataset_manager/views.py", line 114, in export_task_as_dataset
    return export(dst_format, task_id=task_id, server_url=server_url, save_images=True)
  File "/home/django/cvat/apps/dataset_manager/views.py", line 81, in export
    export_fn(db_instance.id, temp_file, dst_format,
  File "/home/django/cvat/apps/dataset_manager/task.py", line 766, in export_task
    task.export(f, exporter, host=server_url, save_images=save_images)
  File "/home/django/cvat/apps/dataset_manager/task.py", line 648, in export
    exporter(dst_file, task_data, **options)
  File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 37, in __call__
    f_or_cls(*args, **kwargs)
  File "/home/django/cvat/apps/dataset_manager/formats/coco.py", line 46, in _export
    dataset.export(temp_dir, 'coco_person_keypoints', save_images=save_images,
  File "/opt/venv/lib/python3.8/site-packages/datumaro/util/scope.py", line 158, in wrapped_func
    ret_val = func(*args, **kwargs)
  File "/opt/venv/lib/python3.8/site-packages/datumaro/components/dataset.py", line 1111, in export
    raise e.__cause__
datumaro.components.errors.AnnotationExportError: Failed to export item ('frame_000021', 'default')

@yasakova-anastasia , could you check this problem?

@zhiltsov-max I have no issue when I export with the format 'CVAT for videos 1.1' with the 'Save Images Option'.
Whereas the issue happens with the 'COCO Keypoints 1.1' format.

Hi @bsekachev @zhiltsov-max,
Sorry to disturb.
Would it be possible to have someone on this issue ?
It is a major problem for our team as we want to start delegating a skeleton annotation project with external annotators.
We are stuck since we can't export the annotations.
Any workaround ?
I could try to help, problem is I never contributed in the CVAT code base.

I couldn't reproduce this.
@ConsciousML Would it be possible to create a small task that exhibits this problem, back it up, and attach the backup here?

I am also not able to export the annotation file for keypoint annotation from CVAT,
The issue that I am getting is.

Please help me solve this issue.

Hi, @AnandGajaria
Could you please provide more information?

What format do you export annotations to?
Please, provide git hash commit (git log -1) and  docker logs cvat_worker_default

Thank you so much for the reply, @yasakova-anastasia
I am trying to export in COCO keypoint 1.0 Format.
The process that I am following is.

Create a Task.
Create a Skeleton using an image.
Then use that skeleton on an Image I want to annotate.
-Adjust the annotation.
-Save and Export it.
-So while Exporting I am getting the issue.

Thank you so much for the help.

@AnandGajaria, I think this problem has already been fixed. Could you tell me what version of CVAT you are using?

I am also not able to export the annotation file for keypoint annotation from CVAT,
The issue that I am getting is.

Please help me solve this issue.

I got same error.
I am using dev version.
Is there any updated..?

@DoHoNi , @AnandGajaria , could you please backup your task and share with us the archive?

I get the same error using the web application and exporting skeleton annotation to COCO keypoints. In addition, when exporting using YOLO 1.1 I get empty .txt files. Do you know if there is a work around to this problem?

I am having the same problem while exporting the data for Skeleton Keypoints


New Crowdin translations

New Crowdin pull request with translations

Codecov Report

Merging #356 (ca05cdd) into master (24a589b) will not change coverage.
The diff coverage is n/a.

 Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Create test

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

AWS CodeBuild CI Report

CodeBuild project: codebuildgithubautobuildPro-zk9yviAhnVl6
Commit ID: e1aa092ade05827c4ca277eaac919580f30476cc
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Bump Jekyll dependency

This PR is a little outdated since Jekyll 3.3.0 is released, bump dependency to support latest Jekyll.

cc/ @gjtorikian
Could you take a look at this PR. I'd love to submit another one on a different project, but it depends on this plugin updating its dependency.

Totally spaced. This is now out as 1.0.1!

Thank you very much!


Helm chart with minikube tests (#432)

A Helm Chart is provided to be used to deploy the EDC runtime in Kubernetes environments.
Chart design
The chart is highly configurable:

Configurable runtime based on Docker image of base EDC, Data Plane Framework Server, or custom runtime.
Configurable environment variables (as ConfigMap) for runtime settings, following 12-factor principles.
Configurable secrets (injected as environment variables into pods).
Configurable labels and annotations.
Configurable ingress.

The chart was generated using helm create in order to follow best practice for components, configurability and ingress definitions, and adapted to fit the EDC runtime.
A generic Dockerfile was created that builds a Docker image using any runtime JAR. It is designed to be used as-is and take configuration from environment variables. Alternatively, implementors can extend the Dockerfile to package a configuration file.
Testing
The chart must be tested in CI, like any other code, to ensure it remains valid as either the chart or EDC code evolves.
A GitHub worklow installs minikube (a local lightweight kubernetes installation) and deploys two releases of the chart, for the Consumer and Provider EDC connector of sample 04.0. The workflow then triggers file copy and verifies that the destination file was created on the Provider pod.

Moving the charts folder to resources as @ndr-brt suggested seems reasonable and will probably have little impact on the rest of the code. Other than that I'm OK with this.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@63ebf68). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #701   +/-   ##
=======================================
  Coverage        ?   56.45%
  Complexity      ?     2097
=======================================
  Files           ?      589
  Lines           ?    13027
  Branches        ?      925
=======================================
  Hits            ?     7355
  Misses          ?     5254
  Partials        ?      418

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 63ebf68...31f993c. Read the comment docs.

I would avoid the charts folder in an already crowded root folder, I would put it into resources

Done. I would still recommend to put it in the root though, so that “casual” users find it

I would avoid the charts folder in an already crowded root folder, I would put it into resources

Done. I would still recommend to put it in the root though, so that “casual” users find it

We can document that in the README, probably is the first place where a "casual" user take a look at.

I would avoid the charts folder in an already crowded root folder, I would put it into resources

Done. I would still recommend to put it in the root though, so that “casual” users find it

We can document that in the README, probably is the first place where a "casual" user take a look at.

I have already :) thanks


Subtitle Synchronizer

Is your feature request related to a problem? Please describe.
Subtitles we often download are out of sync because of different variation of a single movie of different lengths being out there. We often compensate for it by setting the offset via some video player like VLC. But every time we forward or try to rewind the movie it lags because it needs to again synchronize the subtitles.
Describe the solution you'd like
Using subtitle synchronizer, we can edit the time stamps of the srt file itself so the subtitles are perfectly aligned with the video.
Describe alternatives you've considered
I am using this program for the past 3 weeks and it is working perfectly so haven't really looked at any alternatives. Would love to have suggestions for other implementations though.

Great program to make , go for it .
If you need any help mention me , I will contribute to it .
Assigning you this issue 👍

@Mrinank-Bhowmick I have made the PR #51. Also currently we have to manually provide the shift. Any idea how we can completely automate it?

@Mrinank-Bhowmick I have made the PR #51. Also currently we have to manually provide the shift. Any idea how we can completely automate it?

I will review it and let you know
Thanks 👍

@Mrinank-Bhowmick I have made the PR #51. Also currently we have to manually provide the shift. Any idea how we can completely automate it?

I will review it and let you know Thanks 👍

Great👍


Avoid undefined variable

I believe this was the original intent of 3a9ce07be5dc73a6ae816e880a384186335f043a based on the indentation there and the fact that $tmpXML is of no use if not defined.

Thanks! That is a good catch!


Team Edition Chart version 3.6.1?

Hey folks!  I'm not sure this is the right place for this, so feel free to close this issue and redirect me if so.
I'm just getting started with Mattermost team edition within a hobby kubernetes cluster.
It looks like the team edition chart got bumped to 3.6.1 (and Mattermost 5.13.2) when the Enterprise Edition chart got updated (and dockerhub has had the latest image for a while) - but the https://helm.mattermost.com/index.yaml didn't get updated.
$ helm search -l mattermost/mattermost-team-edition
NAME                              	CHART VERSION	APP VERSION	DESCRIPTION
mattermost/mattermost-team-edition	3.6.0        	5.13.0     	Mattermost Team Edition server.
mattermost/mattermost-team-edition	3.5.1        	5.12.4     	Mattermost Team Edition server.
mattermost/mattermost-team-edition	3.4.1        	5.11.0     	Mattermost Team Edition server.
mattermost/mattermost-team-edition	3.4.0        	5.11.0     	Mattermost Team Edition server.
mattermost/mattermost-team-edition	3.3.0        	5.10.0     	Mattermost Team Edition server.
mattermost/mattermost-team-edition	3.2.0        	5.10.0     	Mattermost Team Edition server.

I'm guessing that was just a oversight in the publication process, but I wanted to raise it up in case something was missed somewhere.
Thanks!

Clearly this isn't an issue any longer.


DMP-172 - created event handler that only logs event information.

created event handler that only logs event information.
https://tools.hmcts.net/jira/browse/DMP-172

Is there a change we need to make somewhere to link the event type to the handler?

I don't think so the jira is only mentions the possibility of doing it in the future.

Looking at legacy there is 1 event for which the null handler is used. The event has already been added I will update the handler for it.


Contributing guide

@agilesteel I've added some details about sbt-errors-summary and how to contribute, but it's probably incomplete. Let me know what you think and don't hesitate to add more information!
And I'd be delighted to get some help from you if you're still interested in maintaining this plugin!

@agilesteel Hey, no worries. If he wants to contribute that's great too!


VM: Failing (flaky) service test service/smart_next_test

Is this a known issue @johnmccutchan @rmacnak-google ?
=== Failure summary:
FAILED: none-vm debug_x64 service/smart_next_test
Expected: Pass
Actual: RuntimeError
CommandOutput[vm]:
stdout:
** Launching out/DebugX64/dart --error_on_bad_type --error_on_bad_override --ignore-unrecognized-flags --package-root=out/DebugX64/packages/ --enable-vm-service:0 /mnt/data/b/build/slave/vm-linux-debug-x64-be/build/sdk/runtime/observatory/tests/service/smart_next_test.dart --testee-mode
Observatory listening on http://<IP_ADDRESS>:42330
** Signaled to run test queries on 42330
Running smart_next_test.dart [1/27]
Breakpoint reached
Running smart_next_test.dart [2/27]
Checking we are at line 16
We are at Frame(doAsync.<doAsync_async_body> smart_next_test.dart:token(57))
Running smart_next_test.dart [3/27]
next-sync
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Running smart_next_test.dart [4/27]
Breakpoint reached
Running smart_next_test.dart [5/27]
Checking we are at line 16
We are at Frame(doAsync.<doAsync_async_body> smart_next_test.dart:token(55))
Running smart_next_test.dart [6/27]
next-async
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
Running smart_next_test.dart [7/27]
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Event: ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Breakpoint reached
Breakpoint reached
Running smart_next_test.dart [8/27]
Checking we are at line 17
We are at Frame(doAsync.<doAsync_async_body> smart_next_test.dart:token(62))
Running smart_next_test.dart [9/27]
next-sync
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Running smart_next_test.dart [10/27]
Breakpoint reached
Running smart_next_test.dart [11/27]
Checking we are at line 18
We are at Frame(doAsync.<doAsync_async_body> smart_next_test.dart:token(67))
Running smart_next_test.dart [12/27]
next-sync
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Running smart_next_test.dart [13/27]
Breakpoint reached
Running smart_next_test.dart [14/27]
Checking we are at line 19
We are at Frame(doAsync.<doAsync_async_body> smart_next_test.dart:token(74))
Running smart_next_test.dart [15/27]
next-sync
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Running smart_next_test.dart [16/27]
Breakpoint reached
Running smart_next_test.dart [17/27]
Checking we are at line 19
We are at Frame(doAsync.<doAsync_async_body> smart_next_test.dart:token(72))
Running smart_next_test.dart [18/27]
next-async
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
Running smart_next_test.dart [19/27]
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Event: ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Breakpoint reached
Breakpoint reached
Running smart_next_test.dart [20/27]
Checking we are at line 20
We are at Frame(doAsync.<doAsync_async_body> smart_next_test.dart:token(81))
Running smart_next_test.dart [21/27]
next-sync
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
Running smart_next_test.dart [22/27]
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Event: ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Breakpoint reached
Running smart_next_test.dart [23/27]
Checking we are at line 20
We are at Frame(doAsync.<doAsync_async_body> smart_next_test.dart:token(79))
Running smart_next_test.dart [24/27]
next-sync
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='Resume')
Running smart_next_test.dart [25/27]
VM-VERBOSE: smart_next_test.dart$main pause event ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Event: ServiceEvent(owner='isolates/721152444', kind='PauseBreakpoint')
Breakpoint reached
Breakpoint reached
Running smart_next_test.dart [26/27]
Checking we are at line 21
We are at Frame(_RootZone.runUnary zone.dart:token(8932))
** Killing script
Unexpected exception in service tests: Expected: <21>
Actual: <1165>
#0      DefaultFailureHandler.fail (package:unittest/src/matcher/expect.dart:117:5)
#1      DefaultFailureHandler.failMatch (package:unittest/src/matcher/expect.dart:121:5)
#2      expect (package:unittest/src/matcher/expect.dart:95:20)
#3      stoppedAtLine..<_async_body> (file:///mnt/data/b/build/slave/vm-linux-debug-x64-be/build/sdk/runtime/observatory/tests/service/test_helper.dart:223:5)
#4      _asyncThenWrapperHelper. (dart:async-patch/async_patch.dart:27)
#5      _rootRunUnary (dart:async/zone.dart:910)
#6      _CustomZone.runUnary (dart:async/zone.dart:810)
#7      _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:502)
#8      _Future._propagateToListeners (dart:async/future_impl.dart:585)
#9      _Future._completeWithValue (dart:async/future_impl.dart:376)
#10     _Future._asyncComplete. (dart:async/future_impl.dart:430)
#11     _rootRun (dart:async/zone.dart:903)
#12     _CustomZone.run (dart:async/zone.dart:802)
#13     _CustomZone.runGuarded (dart:async/zone.dart:708)
#14     _CustomZone.bindCallback. (dart:async/zone.dart:733)
#15     _microtaskLoop (dart:async/schedule_microtask.dart:43)
#16     _microtaskLoopEntry (dart:async/schedule_microtask.dart:52)
#17     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:96)
#18     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:149)
stderr:
Unhandled exception:
Uncaught Error: Expected: <21>
Actual: <1165>
Stack Trace:
#0      DefaultFailureHandler.fail (package:unittest/src/matcher/expect.dart:117:5)
#1      DefaultFailureHandler.failMatch (package:unittest/src/matcher/expect.dart:121:5)
#2      expect (package:unittest/src/matcher/expect.dart:95:20)
#3      stoppedAtLine..<_async_body> (file:///mnt/data/b/build/slave/vm-linux-debug-x64-be/build/sdk/runtime/observatory/tests/service/test_helper.dart:223:5)
#4      _asyncThenWrapperHelper. (dart:async-patch/async_patch.dart:27)
#5      _rootRunUnary (dart:async/zone.dart:910)
#6      _CustomZone.runUnary (dart:async/zone.dart:810)
#7      _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:502)
#8      _Future._propagateToListeners (dart:async/future_impl.dart:585)
#9      _Future._completeWithValue (dart:async/future_impl.dart:376)
#10     _Future._asyncComplete. (dart:async/future_impl.dart:430)
#11     _rootRun (dart:async/zone.dart:903)
#12     _CustomZone.run (dart:async/zone.dart:802)
#13     _CustomZone.runGuarded (dart:async/zone.dart:708)
#14     _CustomZone.bindCallback. (dart:async/zone.dart:733)
#15     _microtaskLoop (dart:async/schedule_microtask.dart:43)
#16     _microtaskLoopEntry (dart:async/schedule_microtask.dart:52)
#17     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:96)
#18     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:149)
#0      _rootHandleUncaughtError. (dart:async/zone.dart:894)
#1      _microtaskLoop (dart:async/schedule_microtask.dart:43)
#2      _microtaskLoopEntry (dart:async/schedule_microtask.dart:52)
#3      _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:96)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:149)
Command[vm]: DART_CONFIGURATION=DebugX64 out/DebugX64/dart --error_on_bad_type --error_on_bad_override --ignore-unrecognized-flags --package-root=out/DebugX64/packages/ /mnt/data/b/build/slave/vm-linux-debug-x64-be/build/sdk/runtime/observatory/tests/service/smart_next_test.dart
Took 0:00:10.449000
Short reproduction command (experimental):
python tools/test.py -ax64 --write-debug-log --write-test-outcome-log --copy-coredumps --exclude-suite pkg -t120 service/smart_next_test

@fsc8000 Known issue. Thanks for filing.

Dup of #24176


support for webseeds in torrent creation

It would be helpful to be able to specify a web seed when creating a new torrent file with MetadataBuilder.
It would also be nice if I could hand in file info in the form of path, length and a list of sha1 hashes but that is kinda special to my needs.  I never actually have all the files on disk or in memory at once and reading from the streams is kinda expensive, so I like to do that only once.  Gathering the sha1 hashes every x bytes is easy though.  But anyways, not a big deal, I can work around that one.

can you link please specification about "web seed in torrent file"? I don't know what is it

Sorry, I should have included it.
http://bittorrent.org/beps/bep_0019.html

i added setFilesInfo method for creating metadata from hashes
https://github.com/mpetazzoni/ttorrent/commit/35b2303d038228c8cbb23336e0da608125ebaa62
try to use it


[Drawer] Button behind Drawer

If I had a Drawer inside Header container, the Header container is position fixed,then Drawer will be coverd by another Buttons.
import React, { Component } from 'react'
import Navbar from '../components/Navbar'

export default class Header extends Component {
  render() {
    return (

        <Navbar />

    )
  }
}

const style= {
  container: {
    width: '100%',
    height: '50px',
    background: 'yellow',
    position: 'fixed',
    top: '0px'
  }
}

Navbar
import React from 'react';
import Drawer from 'material-ui/Drawer';
import MenuItem from 'material-ui/MenuItem';
import IconButton from 'material-ui/IconButton';
import MainNavBtn from 'material-ui/svg-icons/action/toc';
export default class Navbar extends React.Component {

  constructor(props) {
    super(props);
    this.state = {open: false};
  }

  handleToggle = () => this.setState({open: !this.state.open});

  handleClose = () => this.setState({open: false});

  render() {
    return (

        <IconButton tooltip="主選單" onTouchTap={this.handleToggle}>
          <MainNavBtn />
        </IconButton>
        <Drawer
          docked={false}
          width={200}
          open={this.state.open}
          onRequestChange={(open) => this.setState({open})}
        >
          <MenuItem onTouchTap={this.handleClose}>Menu Item</MenuItem>
          <MenuItem onTouchTap={this.handleClose}>Menu Item 2</MenuItem>
        </Drawer>

    );
  }
}

The Button including Flat and Raised ,always behind the Drawer component
I have set the z-index but same problem,how to fix this?Thanks for reply.
If no position fixed on the Header container,it works good.

may this help:
const style = {
    top:50
  };
<Drawer
                  docked={false}
                  containerStyle={style}
                  overlayStyle={style}
                 ....
                >
.....
</Drawer>

After add zIndex to the Header container solved this,but not complete figure it out.

if i understand u well , you want to put your header container on top of the Drawer, so the top part of the drawer is gonna be hidden by HC anyway , so why not setting the top of the  make it the same height of
e.g : like i show u above top:50

when the drawer open, the background button will appear on the drawer list.

Please submit support requests and questions to StackOverflow using the tag material-ui. StackOverflow is better suited for this kind of support. The issue tracker is for bug reports and feature discussions. See also our CONTRIBUTING guidelines.


Update the pypi package

System OS
Linux
Python Version
3.8 (CPython)
Install Source
pip / PyPi
Install version / commit hash
v3.9.6
Expected Behavior vs Actual Behavior
I installed spotdl via pip but when I tried to launch the web UI by typing spotdl web as written in the README.md, it is searching for the song "web" instead of launching the UI.
Steps to reproduce - Ensure to include actual links!

pip install spotdl
spotdl web

Traceback
Searching Spotify for song named "web"...

Other details
No response

V4 is still not yet released as stable, so pip will default to install v3. The instructions for downloading v4 are available on the readme in the repo.


Change Menu Order

Move Create button from main menu to My Account Sub Menu and change to Create Collection place it between General and My Collections
Move Stake To Earn from My Account sub Menu to Main Menu Between LaunchPad and Stats. Change to Stake2Earn

Done at commit [66973a7](https://github.com/ArtZero-io/frontend-react/commit/66973a7d8121eef9007dbeaf469a7fe82f78a8ad)


[SimpleReduceHandler] handle more than two args

Fixes #163
https://travis-ci.org/marscher/jsonpickle/builds/215615881

So, I now remember why this was weird to me - jsonpickle's  default Pickler and Unpickler classes already handle the full reduce protocol correctly. This just introduces code duplication. I'm going to look at removing this class other than as a wrapper.
Either that, or we should return this as it was, to explicitly not handle the full protocol. While that might sound crazy, it makes sense if we also deprecate it and put it on a path for removal. @davvid What are your thoughts.

Specifically, it looks like the problem that this handler was solving is that certain classes are not getting serialised using the reduce method

Removing the duplication by making the class a wrapper would be a nice improvement, though I'll take any improvement to how it's currently implemented.  It's true that the handler is there to catch those other classes; if there's a more general approach that simplifies the code then I'm all for it.

The thing is, that the default pickler also invokes getstate even if the reduce protocol is present. So in a class hierarchy with getstate present, one can not give the reduce protocol the highest priority. Note that getstate is ignored by Python (2 and 3), if the reduce protocol is available.


Update git action go versions

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.06%. Comparing base (68ecfbf) to head (7586efc).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##           master      #17   +/-   ##
=======================================
  Coverage   99.06%   99.06%
=======================================
  Files           3        3
  Lines         213      213
=======================================
  Hits          211      211
  Misses          1        1
  Partials        1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Weblogic fmw_infrastructure 12c Docker Container- Domain Creation Failed.. Please check the Domain Logs

HI Mriccell/others,
I am getting below errors during oracle/fmw-infrastructure:<IP_ADDRESS> docker creation.
[oracle@localdev fmwInfra]$ docker run -d -p 9001:7001 --network=InfraNET -v /u01/oracle/fmwInfra:/u01/oracle/user_projects --name InfraAdminContainer --env-file ./infraDomain.env.list oracle/fmw-infrastructure:<IP_ADDRESS>.01
5fdfc4446b3f7c09461ab979e6d25118e52a76c92f02fd11132411c825bea55b
[oracle@localdev fmwInfra]$ docker logs --details InfraAdminContainer -f
 Configuring Domain for first time
 If Domain has already been Configured
 Start the Admin and Managed Servers
 =====================================
 CONNECTION_STRING=fc54f8444045:1521/ORCLPDB1
 RCUPREFIX=INFRA7
 DOMAIN_NAME=InfraDomain
 Loading RCU Phase
 =================
 CONNECTION_STRING=fc54f8444045:1521/ORCLPDB1
 RCUPREFIX=INFRA7
 DB_PASSWORD=password
 jdbc_url=jdbc:oracle:thin:@fc54f8444045:1521/ORCLPDB1
 ADMIN_PASSWORD=password
 DB_SCHEMA_PASSWORD=password
 DB_USERNAME=sys
 DOMAIN_HOME: /u01/oracle/user_projects/domains/InfraDomain
 MW_HOME: /u01/oracle
 WL_HOME: /u01/oracle/wlserver
 ADMIN_USER: weblogic
 DOMAIN_ROOT: /u01/oracle/user_projects/domains
 DOMAIN HOME: /u01/oracle/user_projects/domains/InfraDomain
 Loading SOA RCU into database
 Domain Configuration Phase
 ==========================

 Initializing WebLogic Scripting Tool (WLST) ...

 Welcome to WebLogic Server Administration Scripting Shell

 Type help() for help on available commands

 /u01/oracle/container-scripts/createInfraDomain.py called with the following sys.argv array:
 sys.argv[0] = /u01/oracle/container-scripts/createInfraDomain.py
 sys.argv[1] = -oh
 sys.argv[2] = /u01/oracle
 sys.argv[3] = -jh
 sys.argv[4] = /usr/java/default
 sys.argv[5] = -parent
 sys.argv[6] = /u01/oracle/user_projects/domains
 sys.argv[7] = -name
 sys.argv[8] = InfraDomain
 sys.argv[9] = -user
 sys.argv[10] = weblogic
 sys.argv[11] = -password
 sys.argv[12] = password
 sys.argv[13] = -rcuDb
 sys.argv[14] = fc54f8444045:1521/ORCLPDB1
 sys.argv[15] = -rcuPrefix
 sys.argv[16] = INFRA7
 sys.argv[17] = -rcuSchemaPwd
 sys.argv[18] = password
 Error: set() failed. Do dumpStack() to see details.
 Error: runCmd() failed. Do dumpStack() to see details.
 Problem invoking WLST - Traceback (innermost last):
   File "/u01/oracle/container-scripts/createInfraDomain.py", line 271, in ?
   File "/u01/oracle/container-scripts/createInfraDomain.py", line 59, in createInfraDomain
   File "/u01/oracle/container-scripts/createInfraDomain.py", line 72, in createBaseDomain
   File "/tmp/WLSTOfflineIni7109568471411681869.py", line 79, in set
   File "/tmp/WLSTOfflineIni7109568471411681869.py", line 19, in command
        at com.oracle.cie.domain.script.jython.CommandExceptionHandler.handleException(CommandExceptionHandler.java:69)
        at com.oracle.cie.domain.script.jython.WLScriptContext.handleException(WLScriptContext.java:2983)
        at com.oracle.cie.domain.script.jython.WLScriptContext.runCmd(WLScriptContext.java:735)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)

 com.oracle.cie.domain.script.jython.WLSTException: com.oracle.cie.domain.script.jython.WLSTException

 RetVal from Domain creation 1
 Domain Creation Failed.. Please check the Domain Logs

Please can you help with sorting out this issue.
Cheers,
Manish

Please note that RCU was successfully completed. See logs below.
Processing command line ....
Repository Creation Utility - Checking Prerequisites
Checking Global Prerequisites
Repository Creation Utility - Checking Prerequisites
Checking Component Prerequisites
Repository Creation Utility - Creating Tablespaces
Validating and Creating Tablespaces
Repository Creation Utility - Create
Repository Create in progress.
Percent Complete: 20
Percent Complete: 20
Percent Complete: 22
Percent Complete: 24
Percent Complete: 26
Percent Complete: 26
Percent Complete: 26
Percent Complete: 35
Percent Complete: 35
Percent Complete: 45
Percent Complete: 45
Percent Complete: 45
Percent Complete: 53
Percent Complete: 53
Percent Complete: 63
Percent Complete: 63
Percent Complete: 63
Percent Complete: 71
Percent Complete: 71
Percent Complete: 73
Percent Complete: 73
Percent Complete: 75
Percent Complete: 75
Percent Complete: 84
Percent Complete: 84
Percent Complete: 84
Percent Complete: 85
Percent Complete: 86
Percent Complete: 87
Percent Complete: 87
Percent Complete: 92
Percent Complete: 92
Percent Complete: 96
Percent Complete: 96
Percent Complete: 100
Repository Creation Utility: Create - Completion Summary
Database details:
Host Name                                    : fc54f8444045
Port                                         : 1521
Service Name                                 : ORCLPDB1
Connected As                                 : sys
Prefix for (prefixable) Schema Owners        : INFRA7
RCU Logfile                                  : /tmp/RCU2018-04-13_03-11_1065786166/logs/rcu.log
Component schemas created:
Component                                    Status         Logfile
Common Infrastructure Services               Success        /tmp/RCU2018-04-13_03-11_1065786166/logs/stb.log
Oracle Platform Security Services            Success        /tmp/RCU2018-04-13_03-11_1065786166/logs/opss.log
Audit Services                               Success        /tmp/RCU2018-04-13_03-11_1065786166/logs/iau.log
Audit Services Append                        Success        /tmp/RCU2018-04-13_03-11_1065786166/logs/iau_append.log
Audit Services Viewer                        Success        /tmp/RCU2018-04-13_03-11_1065786166/logs/iau_viewer.log
Metadata Services                            Success        /tmp/RCU2018-04-13_03-11_1065786166/logs/mds.log
WebLogic Services                            Success        /tmp/RCU2018-04-13_03-11_1065786166/logs/wls.log
Repository Creation Utility - Create : Operation Completed

The problem is after RCU creation when the actual domain is created you are running into a WLST error.  It is failing in set('Password', password) probably the password you used is not valid so it is not able to create the domain. Every time you are going to build a new image make sure to have a new RCU Prefix and that your domain has not been created in the PV.

Closing due to inactivity, please reopen if you are still seeing the problem.

Hi Guys,
I'm not sure if there was a fix for the above issue.  But I'm running into exactly the same error, where it is failing to create the domain at the spot where it needs to set/define the password for the domain creation(line 80 in createInfraDomain.py). I've tried putting in a complex password as suggested above with no luck.
running an image <IP_ADDRESS> pulled from docker hub
docker pull store/oracle/fmw-infrastructure:<IP_ADDRESS>-191221
Here are my logs
c:\soa-<IP_ADDRESS>>docker logs --details InfraSOA
Domain Home is:  /u01/oracle/user_projects/domains/myinfraDomain1
Configuring Domain for first time
Start the Admin and Managed Servers
CONNECTION_STRING=InfraDB:1521/InfraPDB1.us.oracle.com
RCUPREFIX=INFRA001
DOMAIN_NAME=myinfraDomain1
DOMAIN_HOME=/u01/oracle/user_projects/domains/myinfraDomain1
Loading RCU Phase
CONNECTION_STRING=InfraDB:1521/InfraPDB1.us.oracle.com
RCUPREFIX=INFRA001
jdbc_url=jdbc:oracle:thin:@InfraDB:1521/InfraPDB1.us.oracle.com
Creating Domain 1st execution
Loading SOA RCU into database
Domain Configuration Phase
Initializing WebLogic Scripting Tool (WLST) ...
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands
Creating Admin Server...
Error: set() failed. Do dumpStack() to see details.
Error: runCmd() failed. Do dumpStack() to see details.
Problem invoking WLST - Traceback (innermost last):
File "/u01/oracle/container-scripts/createInfraDomain.py", line 308, in ?
File "/u01/oracle/container-scripts/createInfraDomain.py", line 46, in createInfraDomain
File "/u01/oracle/container-scripts/createInfraDomain.py", line 80, in createBaseDomain
File "/tmp/WLSTOfflineIni1955096353876950427.py", line 79, in set
File "/tmp/WLSTOfflineIni1955096353876950427.py", line 19, in command
at com.oracle.cie.domain.script.jython.CommandExceptionHandler.handleException(CommandExceptionHandler.java:69)
at com.oracle.cie.domain.script.jython.WLScriptContext.handleException(WLScriptContext.java:3085)
at com.oracle.cie.domain.script.jython.WLScriptContext.runCmd(WLScriptContext.java:738)
at sun.reflect.GeneratedMethodAccessor125.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
com.oracle.cie.domain.script.jython.WLSTException: com.oracle.cie.domain.script.jython.WLSTException
RetVal from Domain creation 1
Domain Creation Failed.. Please check the Domain Logs

I have Same issue , any one knows how to slove  ?

Try using a password with both numbers and characters e. g. weblogic1234
On Mon, 12 Sept 2022, 13:35 maxdone1985, @.***> wrote:

I have Same issue , any one knows how to slove ?
—
Reply to this email directly, view it on GitHub
https://github.com/oracle/docker-images/issues/844#issuecomment-1243542438,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGJPBAYVVCAHTGWVWPA3LVLV54BOZANCNFSM4E2LFZUQ
.
You are receiving this because you commented.Message ID:
@.***>

this is what i am using
username=weblogic
password=weblogic1
DOMAIN_NAME=infraDom
ADMIN_LISTEN_PORT=7001
ADMIN_NAME=myadmin
ADMIN_HOST=InfraAdminContainer
ADMINISTRATION_PORT_ENABLED=false
ADMINISTRATION_PORT=9002
MANAGEDSERVER_PORT=8001
MANAGED_NAME=infraServer13
RCUPREFIX=INFRA019
PRODUCTION_MODE=dev
CONNECTION_STRING=InfraDB:1521/InfraPDB1.us.oracle.com
DOMAIN_HOST_VOLUME=C:\apps\docker\oracle-weblogic\properties

You have to use an encrypted password in domain_security.properties and not clear text
echo -n 'WebLogic_Password' | base64


Failed to create a keyvault

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az keyvault create
Errors:
400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token
Traceback (most recent call last):
python3.6/site-packages/knack/cli.py, ln 215, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 631, in execute
    raise ex
cli/core/commands/__init__.py, ln 695, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
...
python3.6/site-packages/msrestazure/azure_active_directory.py, ln 486, in get_msi_token
    result.raise_for_status()
python3.6/site-packages/requests/models.py, ln 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az keyvault create --resource-group {} --name {} --enabled-for-deployment

Expected Behavior
Environment Summary
Linux-4.15.0-1077-azure-x86_64-with-debian-stretch-sid
Python 3.6.5
Installer: DEB

azure-cli 2.4.0 *

Additional Context

@bim-msft please take a look

duplicate with https://github.com/Azure/azure-cli/issues/13426

#13426 https://github.com/Azure/azure-cli/issues/13426

Please close as this is a duplicate issue.


fix: bump edge-runtime to 1.20.1

What kind of change does this PR introduce?
Bumps edge-runtime to 1.20.1

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.03%) to 56.646%

Files with Coverage Reduction
New Missed Lines
%

internal/gen/keys/keys.go
5
12.31%

Totals

Change from base Build<PHONE_NUMBER>:
-0.03%

Covered Lines:
5012

Relevant Lines:
8848

💛  - Coveralls

Let's wait for prod deploy before merging


make install command uses the wrong directory name

Hi, I simply updated the directory name in the Makefile. Thought I'd let you know.

Thank you very much for fixing the typos :)


Update sketchfab viewer aspect ratio

Fixes

Fixes #888 by @zackkrida
Description

Constrains the sketchfab viewer to the 1000px/500px (2/1) aspect ratio indicated by @panchovm in the comments in the linked issue.
Testing Instructions

Check out this branch and run the app using pnpm dev. Then visit localhost:8443/image/00a6a7a4-6605-4cba-bacf-a08b0da546b9 and verify that the viewer is the correct aspect ratio at various screen widths.
I thought about adding a snapshot test for this but wasn't sure about it. Would we mock the request to the sketchfab-viewer script and the subsequent sketchfab API requests for the model information? Seems thorny!
Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[N/A] I added or updated tests for the changes I made (if applicable).
[N/A] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

It looks great to me. Nice work


🐛 Bug: Type snapshots can't infer ESLint fix mode without a process.argv --fix

Bug Report Checklist

[X] I have tried restarting my IDE and the issue persists.
[X] I have pulled the latest main branch of the repository.
[X] I have searched for related issues and found none that matched my issue.

Expected
As originally reported by #14: type snapshots should update if ESLint is run in --fix mode. For a while they were always being updated (unless disableExpectTypeSnapshotFix was enabled).
Actual
As of #113, type snapshots are only updated if process.argv.includes('--fix'). That doesn't account for programmatic usage of ESLint's Node.js API such as from an editor extension.
Additional Info
No response

Just trapped on issue https://github.com/eslint/eslint/issues/17881, and I can tell you that 🐊Putout already handle such cases, you can read more about it in:

v33;
v34;
redlint;

@coderaiser thanks for the suggestion. As much as I love that Antoine de Saint Exupéry quote, this isn't an appropriate place to advertise a neighboring ecosystem project.
As a general rule I don't recommend advertising "competing" (or, as I like to call them, "neighboring") projects for issues that are still new and in-discussion.


Add Linkage section

Create chapter file for Declarations and add section about linkage. Enumerate the language defaults and where it is implementation dependent. Add empty chapter file for Statements to preserve the chapter ordering.

In the cases of external and internal linkage there's also a property where a declaration inherits the linkage of its encompassing declaration. For example:
namespace PublicStuff {
  void Fn(int); // external linkage

  class Obj {
    void MemberFn(); // external linkage
  }; // external linkage
}

namespace {
  void Fn(int); // internal linkage

  class Obj {
    void MemberFn(); // internal linkage
  }; // internal linkage
}

DXC currently does not support unnamed namespaces, that's why I did not include it here.

This isn't true for all declarations. For example declarations in function-scope and scopes that derive from function-scope are always no-linkage unless they are static.

I believe this is described in the No Linkage section under the term block-scope. I can add more details to that section to make it clearer.