container_name returns

Hello,
I have a problem using your plugin with an EFK stack.
Every thing seems OK at startup, the conf is exactly the same as yours, I can see everything up, logs are forwarded to elasticsearch and Kibana retrieves them.
But my container_name remains at  value.
I checked your code and did not notice anything where you get container_name.
One thing i noticed is that i have to use sudo to open config.v2.json. Maybe your plugin does not have read access on it ? (see the CONTAINER FOLDER RIGHTS part).
Can you have a look please, and give me some tips to debug your pluign (i am a Java developer...)

CONF

  log_level debug

  type tail
  path /var/lib/docker/containers/_/_-json.log
  pos_file fluentd-docker.pos
  time_format %Y-%m-%dT%H:%M:%S
  tag docker.*
  format json

  type docker_format
  container_id ${tag_parts[5]}
  tag ${name}
  docker_containers_path /var/lib/docker/containers

#
# KIBANA
@log_name       e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b
@timestamp      October 18th 2016, 16:31:06.000
t_id        AVfYNAJ2sUxcrcA5hnKg
t_index     logstash-2016.10.18
_score
t_type      fluentd
tcontainer_id       e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b
tcontainer_name
tlog        2016-10-18 14:31:06.083  INFO 7 --- [           main] o.a.camel.spring.SpringCamelContext      : StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
tstream     stdout

FLUENTD
fluentd_2       | 2016-10-18T14:29:57.190742756Z Fluentd waithing TCP connection :...
fluentd_2       | 2016-10-18T14:29:58.194248978Z Start fluentd
fluentd_2       | 2016-10-18T14:29:58.461565888Z 2016-10-18 14:29:58 +0000 [info]: fluent/supervisor.rb:471:read_config: reading config file path="/fluentd/etc/fluent.conf"
fluentd_2       | 2016-10-18T14:29:58.464357222Z 2016-10-18 14:29:58 +0000 [info]: fluent/supervisor.rb:337:supervise: starting fluentd-0.12.29
fluentd_2       | 2016-10-18T14:29:58.543209510Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-mixin-config-placeholders' version '0.4.0'
fluentd_2       | 2016-10-18T14:29:58.543257242Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-docker-format' version '0.2.3'
fluentd_2       | 2016-10-18T14:29:58.543317650Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-elasticsearch' version '1.7.0'
fluentd_2       | 2016-10-18T14:29:58.543408305Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-s3' version '0.7.1'
fluentd_2       | 2016-10-18T14:29:58.543525763Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluentd' version '0.12.29'
fluentd_2       | 2016-10-18T14:29:58.543638143Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluentd' version '0.10.61'
fluentd_2       | 2016-10-18T14:29:58.543915201Z 2016-10-18 14:29:58 +0000 [info]: fluent/agent.rb:129:add_match: adding match pattern="docker.var.lib.docker.containers...log" type="docker_format"
fluentd_2       | 2016-10-18T14:29:58.546189569Z 2016-10-18 14:29:58 +0000 [info]: fluent/agent.rb:129:add_match: adding match pattern="" type="copy"
fluentd_2       | 2016-10-18T14:29:58.546478263Z 2016-10-18 14:29:58 +0000 [debug]: plugin/out_copy.rb:44:block in configure: adding store type="elasticsearch"
fluentd_2       | 2016-10-18T14:29:58.654633862Z 2016-10-18 14:29:58 +0000 [debug]: plugin/out_copy.rb:44:block in configure: adding store type="s3"
fluentd_2       | 2016-10-18T14:29:58.706809710Z 2016-10-18 14:29:58 +0000 [info]: fluent/root_agent.rb:147:add_source: adding source type="tail"
fluentd_2       | 2016-10-18T14:29:58.717655521Z 2016-10-18 14:29:58 +0000 [info]: fluent/engine.rb:133:configure: using configuration file:
fluentd_2       | 2016-10-18T14:29:58.717667174Z
fluentd_2       | 2016-10-18T14:29:58.717670825Z     log_level debug
fluentd_2       | 2016-10-18T14:29:58.717673963Z
fluentd_2       | 2016-10-18T14:29:58.717677094Z
fluentd_2       | 2016-10-18T14:29:58.717680053Z     type tail
fluentd_2       | 2016-10-18T14:29:58.717683043Z     path /var/lib/docker/containers//-json.log
fluentd_2       | 2016-10-18T14:29:58.717686178Z     pos_file fluentd-docker.pos
fluentd_2       | 2016-10-18T14:29:58.717689108Z     time_format %Y-%m-%dT%H:%M:%S
fluentd_2       | 2016-10-18T14:29:58.717692020Z     tag docker.*
fluentd_2       | 2016-10-18T14:29:58.717694865Z     format json
fluentd_2       | 2016-10-18T14:29:58.717697657Z
fluentd_2       | 2016-10-18T14:29:58.717700640Z   <match docker.var.lib.docker.containers...log>
fluentd_2       | 2016-10-18T14:29:58.717703595Z     type docker_format
fluentd_2       | 2016-10-18T14:29:58.717706470Z     container_id ${tag_parts[5]}
fluentd_2       | 2016-10-18T14:29:58.717718503Z     tag docker.${name}
fluentd_2       | 2016-10-18T14:29:58.717724682Z     docker_containers_path /var/lib/docker/containers
fluentd_2       | 2016-10-18T14:29:58.717727907Z
fluentd_2       | 2016-10-18T14:29:58.717730840Z   <match docker.>
fluentd_2       | 2016-10-18T14:29:58.717733834Z     @type copy
fluentd_2       | 2016-10-18T14:29:58.717736595Z
fluentd_2       | 2016-10-18T14:29:58.717739495Z       @type elasticsearch
fluentd_2       | 2016-10-18T14:29:58.717742347Z       host elasticsearch
fluentd_2       | 2016-10-18T14:29:58.717745172Z       port 9200
fluentd_2       | 2016-10-18T14:29:58.717747949Z       include_tag_key true
fluentd_2       | 2016-10-18T14:29:58.717750819Z       tag_key @log_name
fluentd_2       | 2016-10-18T14:29:58.717753656Z       logstash_format true
fluentd_2       | 2016-10-18T14:29:58.717756494Z       flush_interval 5s
fluentd_2       | 2016-10-18T14:29:58.717759227Z
fluentd_2       | 2016-10-18T14:29:58.717762106Z
fluentd_2       | 2016-10-18T14:29:58.717764943Z       @type s3
fluentd_2       | 2016-10-18T14:29:58.717767631Z       aws_key_id xxxxxx
fluentd_2       | 2016-10-18T14:29:58.717770376Z       aws_sec_key xxxxxx
fluentd_2       | 2016-10-18T14:29:58.717773109Z       s3_bucket fluentd
fluentd_2       | 2016-10-18T14:29:58.717775893Z       s3_region eu-central-1
fluentd_2       | 2016-10-18T14:29:58.717778631Z       path logs/
fluentd_2       | 2016-10-18T14:29:58.717781389Z       buffer_path /var/log/fluent/s3
fluentd_2       | 2016-10-18T14:29:58.717784193Z       time_slice_format %Y%m%d%H
fluentd_2       | 2016-10-18T14:29:58.717786958Z       time_slice_wait 5s
fluentd_2       | 2016-10-18T14:29:58.717789730Z       utc
fluentd_2       | 2016-10-18T14:29:58.717793708Z       buffer_chunk_limit 256m
fluentd_2       | 2016-10-18T14:29:58.717796698Z
fluentd_2       | 2016-10-18T14:29:58.717799677Z
fluentd_2       | 2016-10-18T14:29:58.717802648Z
fluentd_2       | 2016-10-18T14:29:59.612179183Z 2016-10-18 14:29:59 +0000 [info]: plugin/in_tail.rb:557:initialize: following tail of /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b-json.log

CONTAINER FOLDER RIGHTS
[...]$ sudo ls -al /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b
total 4644
drwx------    3 root     root          4096 Oct 18 14:30 .
drwx------   12 root     root         20480 Oct 18 14:31 ..
-rw-rw-rw-    1 root     root          9708 Oct 18 14:30 config.v2.json
-rw-r-----    1 root     root       4692269 Oct 18 14:31 e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b-json.log
-rw-rw-rw-    1 root     root          1282 Oct 18 14:30 hostconfig.json
-rw-r--r--    1 root     root            13 Oct 18 14:30 hostname
-rw-r--r--    1 root     root           174 Oct 18 14:30 hosts
-rw-r--r--    1 root     root           109 Oct 18 14:30 resolv.conf
-rw-r--r--    1 root     root            71 Oct 18 14:30 resolv.conf.hash
drwxrwxrwt    2 root     root            40 Oct 18 14:30 shm

CONFIG.V2.JSON
[...]$ sudo cat /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/config.v2.json
{"State":{"Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"RemovalInProgress":false,"Dead":false,"Pid":6820,"StartedAt":"2016-10-18T14:30:40.006984652Z","FinishedAt":"2016-10-18T14:30:35.585791768Z","Health":null},"ID":"e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b","Created":"2016-10-17T17:42:42.568174797Z","Managed":false,"Path":"/bin/sh", ........,"Config":{"Hostname":"e47bced8e497","Domainname":"","User":"",
[...]
,"LogPath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b-json.log","Name":"/r-stack_my-container","Driver":"overlay","MountLabel":"","ProcessLabel":"","RestartCount":0,"HasBeenStartedBefore":false,"HasBeenManuallyStopped":false,"MountPoints":{"/tmp":{"Source":"","Destination":"/tmp","RW":true,"Name":"3f92ecaad901e05cc4ce24f998a4605a2861590ff22f3924abe594e79c8b8262","Driver":"local","Relabel":"","Propagation":"","Named":false,"ID":"13a83ceabb05a0128342ba0b52db7443de61bd03ed95a36ee9356c66c658af72"}},"AppArmorProfile":"","HostnamePath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/hostname","HostsPath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/hosts","ShmPath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/shm","ResolvConfPath":"/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/resolv.conf","SeccompProfile":"","NoNewPrivileges":false}

CAT
[...]$ cat /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b
cat: can't open '/var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b': Permission denied
WHEREAS
sudo cat /var/lib/docker/containers/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b/e47bced8e497f96b6da082c40c530b184a4ff33cdd45664386aca078c8b75e0b-json.
shows file content

+1


Why does using scripts 'src/export_for_gaussian_splatting.py' cause image resolution to decrease?

When I used thie script to generate colmap data, I found that the output image resolution was reduced

Hi!
Script generates undistorted images, because of this - lots of black pixels can appear near edges similar to this example:

And black zones lead to bad gaussian splatting results. So script crops the output undistorted image to its inner part like this:


Fleet deploys repo to Harvester Clusters

See: https://zube.io/rancher/dashboard-ui/c/4266
When the user selects 'all clusters' in the UI for the target for a git repo, this deploys to Harvester clusters as well as regular k8s clusters.
The UI creates the resource with spec.targets = [{ clusterSelector: {} }]; - the backend should not deploy to Harvester clusters in this case.

@cloudnautique What do you think we should do for v2.6.1? Disable Fleet to Harvester entirely seems to make sense but I am not sure if it's a blocker for v.2.6.1

@yasker as discussed, I moved this to v2.6.2. I will have @SheilaghM sync up with @StrongMonkey and @nickgerace to see what the best course of action is here.

This is more like a UI change. By default UI can create gitrepos that exclude some labels(there should some labels that are unique to harvester cluster).

Fleet doesn't have knowledge around which cluster it should/should not deploy to, it onlys looks for the targets(labels) that user specifies. It wouldn't make sense for fleet by default to exclude some pre-populate sets of labels in the backend.

Harvester baremetal clusters have a special provider field shows it's Harvester baremetal cluster. See https://github.com/rancher/rancher/issues/34880#issuecomment-927218920 for details.

To make it easier to filter the Harvester cluster like what Daishan did in https://github.com/rancher/rancher/issues/34954#issuecomment-931653234 , we will need to automatically add a label to any cluster with provider=harvester, so the filter can be used.

As discussed with @StrongMonkey , we will add the following logic in the backend:

If a cluster with provider="", no change to the label cluster.harvesterhci.io regardless it exists or not.
If a cluster with provider="harvester", label cluster.harvesterhci.io will be added if it doesn't exist.
If a cluster with provider != "" && provider != "harvester", label cluster.harvesterhci.io will be removed if it exists.

This should also fix the potential inconsistency issue in the UI since filters are done via the label only at the moment.

https://github.com/rancher/rancher/pull/35007 proposes to add provider label to v3 cluster. We can change to only rely on provider.cattle.io to detect harvester cluster, and also set this label in UI to pin down the cluster in Virtualization Management at first. After cluster is being imported and provider is set to the right value, it will be moved to where it should be.


PowerRename option not appearing in the right-click menu

ℹ Computer information

PowerToys version: 0.24.0
PowerToy Utility: PowerRename
Running PowerToys as Admin: Yes
Windows build number: 1809

📝 Provide detailed reproduction steps (if any)

Check every option in the PowerRename Settings.
Select 3 files, shift + right-click.

✔ Expected result
Power Rename option available in the context menu.
❌ Actual result
Power Rename option not there.
📷 Screenshots
Are there any useful screenshots? WinKey+Shift+S and then just paste them directly into the form

i am sure you mean version 0.24.0.

@github-account1111
please add a screenshot of the context menu

@github-account1111
are the files on a local drive? (shouldn't make any difference, but just to rule out possible conditions)
If you turn off:

will it make any difference and show PowerRename in the context menu?

are the files on a local drive?

Yes. It's also the system drive.

will it make any difference and show PowerRename in the context menu?

Makes no difference (no matter whether I press shift or not).

@github-account1111
open the Registry Editor and check if you have these keys:
Computer\HKEY_CLASSES_ROOT\AllFilesystemObjects\shellex\ContextMenuHandlers\PowerRenameExt

Computer\HKEY_CLASSES_ROOT\CLSID\{0440049F-D1DC-4E46-B27B-98393D79486B}\InprocServer32

Computer\HKEY_CLASSES_ROOT\AllFilesystemObjects\shellex\ContextMenuHandlers\PowerRenameExt

Yep, same exact Data value.

Computer\HKEY_CLASSES_ROOT\CLSID\{0440049F-D1DC-4E46-B27B-98393D79486B}\InprocServer32

@github-account1111
I guess you already tried rebooting the machine.
Can you try uninstalling and reinstalling? https://github.com/microsoft/PowerToys/releases/download/v0.23.2/PowerToysSetup-0.23.2-x64.exe

Will I lose all of my settings and layouts?

@github-account1111

Will I lose all of my settings and layouts?

No, but if you want to be extra safe you can always back-up
%localappdata%\Microsoft\PowerToys\

This plus #7397 makes me feel like something bad happened during install

I've updated and restarted, but the problem is still there.

Also have this issue, not sure why.

Same issue

Got notification about the new comment, and this seems to have been fixed???

I still do not have the PowerRename right-click option after trying with and without Shift right-click, disabling and re-enabling PowerRename and context menu options, updating PowerToys to latest, confirming registry values are there, uninstalling and reinstalling twice, and restarting my computer.

@dlon Could you please post a bug report from your desktop by launching PowerToys\Tools\BugReportTool.exe?

PowerToysReport_2021-07-16-13-26-46.zip

@dlonraydoc Thanks! Indeed, logs indicate that the module should work as expected. Shell extensions are loaded by explorer.exe when you right-click on a corresponding file. To make sure PowerRename is loaded properly, could you please:

download process explorer
open explorer and right click on any file
launch process explorer, press CTRL+F and search for PowerRenameExt.dll
verify that you see something like this:

@dlonraydoc Thanks! Indeed, logs indicate that the module should work as expected. Shell extensions are loaded by explorer.exe when you right-click on a corresponding file. To make sure PowerRename is loaded properly, could you please:

download process explorer
open explorer and right click on any file
launch process explorer, press CTRL+F and search for PowerRenameExt.dll
verify that you see something like this:

its not present in my process explorer

open the Registry Editor and check if you have these keys:
Computer\HKEY_CLASSES_ROOT\CLSID{0440049F-D1DC-4E46-B27B-98393D79486B}\InprocServer32

i dont have this key

i removed the folders rlated to power rename that is th appdata and installation directory folders and also cleared out the registry. i then reinstalled it and it seems to be working now. does use boost library affect this?

@thaynos I don't think so, the module itself is a single dll. Looks like the issue was caused by the lack of registry entires.

@yuyoyuppe in my case yes but not in OP's case. also why was there a lack to begin with?

That could be caused by multiple factors - either installation failure or some 3rd party tool interference with the registry

Is this issue still occurring in v0.49.1. I can't repro on my end.

Is this issue still occurring in v0.49.1. I can't repro on my end.

Sorry not sure. I'm on 1809.

Ok, then well close this for now. Let us know if it does re-occur. and well continue investigating.

Ok, then we'll close this for now. Let us know if it does re-occur. and we`ll continue investigating.

It is occuring for me now, on 0.49.1
Specs:

Windows 11 latest stable, upgraded from Windows 10

What I have tried:

Turning on and off to enable/disable PowerRename
Restarting PowerToys app
Restarting File Explorer process from Task Manager app
Restarting Windows

Ok, then we'll close this for now. Let us know if it does re-occur. and we`ll continue investigating.

It is occuring for me now, on 0.49.1
Specs:
* Windows 11 latest stable, upgraded from Windows 10

What I have tried:
* Turning on and off to enable/disable PowerRename

* Restarting PowerToys app

* Restarting File Explorer process from Task Manager app

* Restarting Windows

UPD: The option shows up in Total Commander app

I can confirm this, same behaviour here:

Edition	Windows 11 Pro
Version	21H2
OS build	22000.318
PowerToys v0.49.1

@Kenya-West @mijenix  that seems to be another issue altogether.
Perhaps because for Windows 11 it appears only if you want it to show more options?
Please check if this is your issue: https://github.com/microsoft/PowerToys/issues/13168

@Kenya-West @mijenix that seems to be another issue altogether. Perhaps because for Windows 11 it appears only if you want it to show more options? Please check if this is your issue: #13168

No, I mark the files with the mouse, right click, "Show more options" and choose "PowerRename".
PowerRename is not started.

@Kenya-West @mijenix that seems to be another issue altogether. Perhaps because for Windows 11 it appears only if you want it to show more options? Please check if this is your issue: #13168

No, I mark the files with the mouse, right click, "Show more options" and choose "PowerRename". PowerRename is not started.

I just saw it generates an Application event entry:

Faulting application name: PowerRename.exe, version: <IP_ADDRESS>, time stamp: 0x617c36c0
Faulting module name: PowerRename.exe, version: <IP_ADDRESS>, time stamp: 0x617c36c0
Exception code: 0xc0000409
Fault offset: 0x000000000008e73d
Faulting process ID: 0x1bd0
Faulting application start time: 0x01d7d71aa834627f
Faulting application path: C:\Program Files\PowerToys\modules\PowerRename\PowerRename.exe
Faulting module path: C:\Program Files\PowerToys\modules\PowerRename\PowerRename.exe
Report ID: 5f340ed6-6414-41d0-b440-b29c37192478
Faulting package full name:
Faulting package-relative application ID:

So, it's not the same issue. This one is closed and it's about another topic.
Please look for or open a new issue with your details.

why is PowerRename version <IP_ADDRESS>?


OOM in rnn-t training

Background information: I use recipes in wenetspeech for training.  There's no problem with oom  in   scan_pessimistic_batches_for_oom, but it's always oom in the later training. it will not work even max_duration is setted 60.
Error information:
RuntimeError: CUDA out of memory. Tried to allocate 50.73 GiB (GPU 0; 31.75 GiB total capacity; 3.75 GiB already allocated; 26.30 GiB free; 4.27 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Environment: pytorch-1.12, k2-1.17, python-3.7
Gpu:v100, 32g memory, 8
Data: more than 10000 hours of data, 8000 sampling rate, 40 dimensional features, audio length between 1-15s

Some people had CUDA OOM errors with CUDA 11.1 in https://github.com/k2-fsa/icefall/issues/247
I am not sure which CUDA version you are using. Can you try torch 1.10 + CUDA 10.2 ?

Thank you very much for your reply. I'll try it

Why does the GPU which tot_loss is Nan use so much memory

After change the environment to pytorch1.11+python3.8.5+k2-1.17+cuda10.2, and the max_duration is 120, it is still oom，the gpu=4 is oom

Are you using your own dataset? Does reducing max duration help?

Yes, I use my own dataset.
I try to reduce max_duration to 60. When the batch is 78476, Oom still appears.       50G memory is required for error reporting

When the batch is 78476

What does this mean ? Does it mean the batch size is 78476 or does it throw the OOM at the 78476th batch?

it throw the OOM at the 78476th batch

it throw the OOM at the 78476th batch

Could you post more log messages?
There must be a .pt file saved to your exp_dir (check the log message in the terminal).
You can load it with torch.load() and check what it contains and see if there are any peculiar things in the batch,
e.g., number of tokens, utterance duration, etc.

number of tokens：8007
Parameters：93M
Load 72000.pt，see the following figure for the sampler information：

The detailed error information is shown in the figure below：

I see. pruned_transducer_stateless2/train.py in wenetspeech does not have the following function, which will save the problematic batch on exception.
https://github.com/k2-fsa/icefall/blob/7157f62af3b7712eda2186f7e3d253df7cde65b5/egs/librispeech/ASR/pruned_transducer_stateless2/train.py#L1007
Could you  copy the code from librispeech to wenetspeech and post the log after the change?
If possible, would you mind making a PR with your changes about adding display_and_save_batch to wenentspeech?

@luomingshuang

Are you sure that you have used trim_to_supervisions in compute_fbank_feature_wenetspeech.py to clip long utterance to short utterances?

I notice that your number of tokens is 8007. In my training for wenetspeech, the number of tokens is 5537. Comparing the two numbers, the 8007 is too large. I think it may causes more computing memory. If possible, I suggest that you can have a try with 5537 tokens. You can get the tokens.txt with 5537 tokens in https://huggingface.co/luomingshuang/icefall_asr_wenetspeech_pruned_transducer_stateless2/blob/main/data/lang_char/tokens.txt .

I notice that your number of tokens is 8007. In my training for wenetspeech, the number of tokens is 5537. Comparing the two numbers, the 8007 is too large. I think it may causes more computing memory. If possible, I suggest that you can have a try with 5537 tokens. You can get the tokens.txt with 5537 tokens in https://huggingface.co/luomingshuang/icefall_asr_wenetspeech_pruned_transducer_stateless2/blob/main/data/lang_char/tokens.txt .

I think he uses his own datasets, your tokens might not be suitable for him. I'd rather he can dump the problematic batches, so we can debug the code to see if there is any bugs in our loss.

It is not a regular OOM issue, it's a bug because it's trying to allocate nearly 60G.  He needs to run it in gdb with e.g.
gdb --args python3 script.py args...
(gdb) r

so we can get a stack trace.  It may be the same issue reported in #396

Collaborator
This may be the problem. I used the features I extracted earlier. Can the pre extracted features be segmented？

The training data information I use is as follows

I see. pruned_transducer_stateless2/train.py in wenetspeech does not have the following function, which will save the problematic batch on exception.
https://github.com/k2-fsa/icefall/blob/7157f62af3b7712eda2186f7e3d253df7cde65b5/egs/librispeech/ASR/pruned_transducer_stateless2/train.py#L1007
Could you copy the code from librispeech to wenetspeech and post the log after the change?
If possible, would you mind making a PR with your changes about adding display_and_save_batch to wenentspeech?

OK, I'll try
I have saved some. Pt, but there is a problem in training when using -- start batch. I don't know if it is because of this

Are you sure that you have used trim_to_supervisions in compute_fbank_feature_wenetspeech.py to clip long utterance to short utterances?

This may be the problem. I used the features I extracted earlier. Can the pre extracted features be segmented？
The training data information I use is as follows:

Are you sure that you have used trim_to_supervisions in compute_fbank_feature_wenetspeech.py to clip long utterance to short utterances?

This may be the problem. I used the features I extracted earlier. Can the pre extracted features be segmented？
The training data information I use is as follows:

Are you sure that you have used trim_to_supervisions in compute_fbank_feature_wenetspeech.py to clip long utterance to short utterances?

This may be the problem. I used the features I extracted earlier. Can the pre extracted features be segmented？
The training data information I use is as follows:

I just looked at the function trim_to_supervision.The audio in wenetspeech is very long, so it is necessary to trim according to supervision. However, my data is distributed between 1-15s,  it should not be trim, so I still can't determine what the problem is.

Are you sure that you have used trim_to_supervisions in compute_fbank_feature_wenetspeech.py to clip long utterance to short utterances?

I just looked at the function trim_to_supervision.The audio in wenetspeech is very long, so it is necessary to trim according to supervision. However, my data is distributed between 1-15s, it should not be trim, so I still can't determine what the problem is.

so I still can't determine what the problem is.

Could you follow https://github.com/k2-fsa/icefall/issues/521#issuecomment-1206208083
to get the batch that causes the OOM?

so I still can't determine what the problem is.

Could you follow #521 (comment) to get the batch that causes the OOM?

OK, I will try it when my GPU is free. According to my GPU, what is the appropriate duration to set? 180?

OK, I will try it when my GPU is free. According to my GPU, what is the appropriate duration to set? 180?

You can try 200 first and use nvidia-smi to view the GPU RAM usage. If it is low, you can increase it. If it results in OOM, you can reduce it.

The problem has been solved. Thank you for all replies.
It is the problem of my data. The length of some text in the data is too long, resulting in a sudden increase in memory.


FileNotFoundError when starting the kernel in 4.1.0

Problem Description
Spyder4.1.0 IPython kernel startup error
Close spyder and restart, the problem persists.
What steps reproduce the problem?
After upgrading spyder4.0.1 to 4.1.0, IPython kernel startup error occurs when starting spyder.
The following command is used to upgrade in a virtual environment:
conda install spyder=4.1.0
启动 IPython 内核时发生错误（An error occurred while starting the IPython kernel）
错误：（Error）
Traceback (most recent call last):
File "C:\Users\Admin\Anaconda3\envs\python37\lib\site‑packages\spyder\plugins\ipythonconsole\plugin.py", line 1209, in create_kernel_manager_and_kernel_client
kernel_manager.start_kernel(stderr=stderr_handle, **kwargs)
File "C:\Users\Admin\Anaconda3\envs\python37\lib\site‑packages\jupyter_client\manager.py", line 259, in start_kernel
**kw)
File "C:\Users\Admin\Anaconda3\envs\python37\lib\site‑packages\jupyter_client\manager.py", line 204, in _launch_kernel
return launch_kernel(kernel_cmd, **kw)
File "C:\Users\Admin\Anaconda3\envs\python37\lib\site‑packages\jupyter_client\launcher.py", line 138, in launch_kernel
proc = Popen(cmd, **kwargs)
File "C:\Users\Admin\Anaconda3\envs\python37\lib\subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "C:\Users\Admin\Anaconda3\envs\python37\lib\subprocess.py", line 1207, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] 系统找不到指定的文件（The system can not find the file specified）。

Versions

Spyder version: 4.1.0
Python version: Python3.7.5
Qt version: 5.9.6
PyQt version: 5.9.2
Operating System name/version: win10

Roll back to the 4.0.1 version, there is no error when starting the spyder.

@dalthviz, please try to reproduce this one by installing Spyder in a conda env and trying to start from it.

Create the new Python virtual environment with the following command, and install spyder4.1.0 at the same time, the same error occurs
conda create -n python37_new python=3.7 spyder=4.1.0

I too was getting this same error after updating to 4.1.0.  A rollback to 4.0.1 fixed it for me.

Some questions:

Did you install Anaconda for all users or just for your user?
Did you install Anaconda in another partition (like D:\)?

I created issue #11829 from the Spyder issue reporting interface. My issue appears to be similar to this issue. I should have checked for this issue already existing first!

I don't remember if I installed Anaconda for all users or just for my user on this machine, and I am aware this is [I think] a well known annoying Anaconda install issue, and I never remember which is the preferred install option.
Anaconda was installed to D:\ mechanical hard drive because I prefer to install mostly only OS on the smaller capacity C:\ SSD for OS performance and reliability reasons. I imagine many others also do this.
Spyder 4.0.1 was very frustrating to use because of the constant freezing problem. I am motivated to continue finding a solution to getting 4.1.0 working on my system.

Same issue.
Notes: default partition information, single user installation, basic environment, fresh anaconda install with conda-forge packages updated.
At one point I got the kernel to load using python start.py in the
spyder-#.#.#-py##_#/Lib/site-packages/spyder/app of a separate environment, but it couldn't locate libraries.

Same thing here, had to rollback

Notes: default partition information,

@b1139atpmme, so you don't have Anaconda installed in another partition?

I too was getting this same error after updating to 4.1.0. A rollback to 4.0.1 fixed it for me.

Can confirm rolling back has no issues for me (apart from being back to 4.01!)

Same issue here.

i had the same issue as all the others with spyder 4.1 in both of my environments (python 3.8.2 and 3.7.6).
will now go back to 4.0.1

Same issue for me too. Removed and recreated env and the issue persists

just went back to spyder 4.0.1 in my python 3.8.2 env...
now it didn't even start anymore! not so great.

just went back to spyder 4.0.1 in my python 3.8.2 env...
now it didn't even start anymore! not so great.

Rolled back to 4.0.1 and worked as b4 for me.

just went back to spyder 4.0.1 in my python 3.8.2 env...
now it didn't even start anymore! not so great.

Rollback worked fine for me as well

@gabrielclow @b1139atpmme @leavor @bmarr @cccsmoreno @manfred2020 @yomoko I created envs in C and D (with conda-forge and the default channel) but I'm unable to reproduce this :/ could you guys check if you have anything different in your envs than the following?:
Env in C with default channel (Note: creating it in D gives the same list of packages)

What I run was something like: conda create -n python37_new python=3.7 spyder=4.1.0
# packages in environment at C:\Users\Daniel\Anaconda3\envs\python37_new:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                   py37_0
argh                      0.26.2                   py37_0
asn1crypto                1.3.0                    py37_0
astroid                   2.3.3                    py37_0
atomicwrites              1.3.0                    py37_1
attrs                     19.3.0                     py_0
autopep8                  1.4.4                      py_0
babel                     2.8.0                      py_0
backcall                  0.1.0                    py37_0
bcrypt                    3.1.7            py37he774522_0
bleach                    3.1.0                    py37_0
ca-certificates           2020.1.1                      0
certifi                   2019.11.28               py37_0
cffi                      1.14.0           py37h7a1dbc1_0
chardet                   3.0.4                 py37_1003
cloudpickle               1.3.0                      py_0
colorama                  0.4.3                      py_0
cryptography              2.8              py37h7a1dbc1_0
decorator                 4.4.2                      py_0
defusedxml                0.6.0                      py_0
diff-match-patch          20181111                   py_0
docutils                  0.16                     py37_0
entrypoints               0.3                      py37_0
flake8                    3.7.9                    py37_0
future                    0.18.2                   py37_0
icu                       58.2                 ha66f8fd_1
idna                      2.9                        py_1
imagesize                 1.2.0                      py_0
importlib_metadata        1.5.0                    py37_0
intervaltree              3.0.2                      py_0
ipykernel                 5.1.4            py37h39e3cac_0
ipython                   7.13.0           py37h5ca1d4c_0
ipython_genutils          0.2.0                    py37_0
isort                     4.3.21                   py37_0
jedi                      0.14.1                   py37_0
jinja2                    2.11.1                     py_0
jpeg                      9b                   hb83a4c4_2
jsonschema                3.2.0                    py37_0
jupyter_client            6.0.0                      py_0
jupyter_core              4.6.1                    py37_0
keyring                   21.1.0                   py37_0
lazy-object-proxy         1.4.3            py37he774522_0
libpng                    1.6.37               h2a8f88b_0
libsodium                 1.0.16               h9d3ae62_0
libspatialindex           1.9.3                h33f27b4_0
markupsafe                1.1.1            py37he774522_0
mccabe                    0.6.1                    py37_1
mistune                   0.8.4            py37he774522_0
nbconvert                 5.6.1                    py37_0
nbformat                  5.0.4                      py_0
numpydoc                  0.9.2                      py_0
openssl                   1.1.1e               he774522_0
packaging                 20.3                       py_0
pandoc                    <IP_ADDRESS>                       0
pandocfilters             1.4.2                    py37_1
paramiko                  2.7.1                      py_0
parso                     0.5.2                      py_0
pathtools                 0.1.2                      py_1
pexpect                   4.8.0                    py37_0
pickleshare               0.7.5                    py37_0
pip                       20.0.2                   py37_1
pluggy                    0.13.1                   py37_0
prompt_toolkit            3.0.3                      py_0
psutil                    5.7.0            py37he774522_0
pycodestyle               2.5.0                    py37_0
pycparser                 2.20                       py_0
pydocstyle                4.0.1                      py_0
pyflakes                  2.1.1                    py37_0
pygments                  2.6.1                      py_0
pylint                    2.4.4                    py37_0
pynacl                    1.3.0            py37h62dcd97_0
pyopenssl                 19.1.0                   py37_0
pyparsing                 2.4.6                      py_0
pyqt                      5.9.2            py37h6538335_2
pyrsistent                0.15.7           py37he774522_0
pysocks                   1.7.1                    py37_0
python                    3.7.6                h60c2a47_2
python-dateutil           2.8.1                      py_0
python-jsonrpc-server     0.3.4                      py_0
python-language-server    0.31.8                   py37_0
pytz                      2019.3                     py_0
pywin32                   227              py37he774522_1
pywin32-ctypes            0.2.0                 py37_1000
pyyaml                    5.3              py37he774522_0
pyzmq                     18.1.1           py37ha925a31_0
qdarkstyle                2.8                        py_0
qt                        5.9.7            vc14h73c81de_0
qtawesome                 0.7.0                      py_0
qtconsole                 4.7.1                      py_0
qtpy                      1.9.0                      py_0
requests                  2.23.0                   py37_0
rope                      0.16.0                     py_0
rtree                     0.9.3            py37h21ff451_0
setuptools                46.0.0                   py37_0
sip                       4.19.8           py37h6538335_0
six                       1.14.0                   py37_0
snowballstemmer           2.0.0                      py_0
sortedcontainers          2.1.0                    py37_0
sphinx                    2.4.0                      py_0
sphinxcontrib-applehelp   1.0.2                      py_0
sphinxcontrib-devhelp     1.0.2                      py_0
sphinxcontrib-htmlhelp    1.0.3                      py_0
sphinxcontrib-jsmath      1.0.1                      py_0
sphinxcontrib-qthelp      1.0.3                      py_0
sphinxcontrib-serializinghtml 1.1.4                      py_0
spyder                    4.1.0                    py37_0
spyder-kernels            1.9.0                    py37_0
sqlite                    3.31.1               he774522_0
testpath                  0.4.4                      py_0
tornado                   6.0.4            py37he774522_1
traitlets                 4.3.3                    py37_0
ujson                     1.35             py37hfa6e2cd_0
urllib3                   1.25.8                   py37_0
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.16.27012          hf0eaf9b_1
watchdog                  0.10.2                   py37_0
wcwidth                   0.1.8                      py_0
webencodings              0.5.1                    py37_1
wheel                     0.34.2                   py37_0
win_inet_pton             1.1.0                    py37_0
wincertstore              0.2                      py37_0
wrapt                     1.12.1           py37he774522_1
yaml                      0.1.7                hc54c509_2
yapf                      0.28.0                     py_0
zeromq                    4.3.1                h33f27b4_3
zipp                      2.2.0                      py_0
zlib                      1.2.11               h62dcd97_3

Env in D with conda-forge

What I run was something like: conda create --prefix D:/kernel_test_forge -c conda-forge python=3.7 spyder=4.1.0

# packages in environment at D:\kernel_test_forge:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                     py_0    conda-forge
argh                      0.26.2                py37_1001    conda-forge
astroid                   2.3.3                    py37_1    conda-forge
atomicwrites              1.3.0                      py_0    conda-forge
attrs                     19.3.0                     py_0    conda-forge
autopep8                  1.5                        py_0    conda-forge
babel                     2.8.0                      py_0    conda-forge
backcall                  0.1.0                      py_0    conda-forge
bcrypt                    3.1.7            py37hfa6e2cd_0    conda-forge
bleach                    3.1.3              pyh8c360ce_0    conda-forge
ca-certificates           2019.11.28           hecc5488_0    conda-forge
certifi                   2019.11.28       py37hc8dfbb8_1    conda-forge
cffi                      1.14.0           py37ha419a9e_0    conda-forge
chardet                   3.0.4           py37hc8dfbb8_1006    conda-forge
cloudpickle               1.3.0                      py_0    conda-forge
colorama                  0.4.3                      py_0    conda-forge
cryptography              2.8              py37hb32ad35_1    conda-forge
decorator                 4.4.2                      py_0    conda-forge
defusedxml                0.6.0                      py_0    conda-forge
diff-match-patch          20181111                   py_0    conda-forge
docutils                  0.16             py37hc8dfbb8_1    conda-forge
entrypoints               0.3             py37hc8dfbb8_1001    conda-forge
flake8                    3.7.9            py37hc8dfbb8_1    conda-forge
future                    0.18.2           py37hc8dfbb8_1    conda-forge
icu                       64.2                 he025d50_1    conda-forge
idna                      2.9                        py_1    conda-forge
imagesize                 1.2.0                      py_0    conda-forge
importlib-metadata        1.5.0                    py37_0    conda-forge
importlib_metadata        1.5.0                         1    conda-forge
intervaltree              3.0.2                      py_0    conda-forge
ipykernel                 5.1.4            py37h5ca1d4c_0    conda-forge
ipython                   7.13.0           py37h5ca1d4c_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
isort                     4.3.21           py37hc8dfbb8_1    conda-forge
jedi                      0.14.1                   py37_0    conda-forge
jinja2                    2.11.1                     py_0    conda-forge
jpeg                      9c                hfa6e2cd_1001    conda-forge
jsonschema                3.2.0            py37hc8dfbb8_1    conda-forge
jupyter_client            6.0.0                      py_0    conda-forge
jupyter_core              4.6.3            py37hc8dfbb8_1    conda-forge
keyring                   21.1.1                   py37_0    conda-forge
lazy-object-proxy         1.4.3            py37hfa6e2cd_1    conda-forge
libclang                  9.0.1           default_hf44288c_0    conda-forge
libpng                    1.6.37               h7602738_0    conda-forge
libsodium                 1.0.17               h2fa13f4_0    conda-forge
libspatialindex           1.9.3                he025d50_3    conda-forge
markupsafe                1.1.1            py37h8055547_1    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4           py37hfa6e2cd_1000    conda-forge
nbconvert                 5.6.1                    py37_0    conda-forge
nbformat                  5.0.4                      py_0    conda-forge
numpydoc                  0.9.2                      py_0    conda-forge
openssl                   1.1.1d               hfa6e2cd_0    conda-forge
packaging                 20.1                       py_0    conda-forge
pandoc                    2.9.2                         0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
paramiko                  2.7.1                    py37_0    conda-forge
parso                     0.5.2                      py_0
pathtools                 0.1.2                      py_1    conda-forge
pexpect                   4.8.0            py37hc8dfbb8_1    conda-forge
pickleshare               0.7.5           py37hc8dfbb8_1001    conda-forge
pip                       20.0.2                     py_2    conda-forge
pluggy                    0.12.0                     py_0    conda-forge
prompt-toolkit            3.0.4                      py_0    conda-forge
prompt_toolkit            3.0.4                         0    conda-forge
psutil                    5.7.0            py37h8055547_1    conda-forge
pycodestyle               2.5.0                      py_0    conda-forge
pycparser                 2.20                       py_0    conda-forge
pydocstyle                5.0.2                      py_0    conda-forge
pyflakes                  2.1.1                      py_0    conda-forge
pygments                  2.6.1                      py_0    conda-forge
pylint                    2.4.4                    py37_0    conda-forge
pynacl                    1.3.0           py37h2fa13f4_1001    conda-forge
pyopenssl                 19.1.0                     py_1    conda-forge
pyparsing                 2.4.6                      py_0    conda-forge
pyqt                      5.12.3           py37h6538335_1    conda-forge
pyqt5-sip                 4.19.18                  pypi_0    pypi
pyqtwebengine             5.12.1                   pypi_0    pypi
pyrsistent                0.15.7           py37h8055547_1    conda-forge
pysocks                   1.7.1            py37hc8dfbb8_1    conda-forge
python                    3.7.6           h5b45d93_4_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python-jsonrpc-server     0.3.4                      py_0    conda-forge
python-language-server    0.31.8           py37hc8dfbb8_0    conda-forge
python_abi                3.7                     1_cp37m    conda-forge
pytz                      2019.3                     py_0    conda-forge
pywin32                   227              py37hfa6e2cd_0    conda-forge
pywin32-ctypes            0.2.0           py37hc8dfbb8_1001    conda-forge
pyyaml                    5.3              py37h8055547_1    conda-forge
pyzmq                     19.0.0           py37h8c16cda_1    conda-forge
qdarkstyle                2.8                        py_0    conda-forge
qt                        5.12.5               h7ef1ec2_0    conda-forge
qtawesome                 0.7.0                      py_0    conda-forge
qtconsole                 4.7.1                      py_0    conda-forge
qtpy                      1.9.0                      py_0    conda-forge
requests                  2.23.0             pyh8c360ce_2    conda-forge
rope                      0.16.0                     py_0    conda-forge
rtree                     0.9.4            py37h804a536_1    conda-forge
setuptools                46.0.0           py37hc8dfbb8_2    conda-forge
six                       1.14.0                     py_1    conda-forge
snowballstemmer           2.0.0                      py_0    conda-forge
sortedcontainers          2.1.0                      py_0    conda-forge
sphinx                    2.4.4                      py_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
spyder                    4.1.0            py37hc8dfbb8_0    conda-forge
spyder-kernels            1.9.0            py37hc8dfbb8_0    conda-forge
sqlite                    3.30.1               hfa6e2cd_0    conda-forge
testpath                  0.4.4                      py_0    conda-forge
tornado                   6.0.4            py37hfa6e2cd_0    conda-forge
traitlets                 4.3.3            py37hc8dfbb8_1    conda-forge
typed-ast                 1.4.1            py37hfa6e2cd_0    conda-forge
ujson                     1.35            py37h63f7a3c_1002    conda-forge
urllib3                   1.25.7           py37hc8dfbb8_1    conda-forge
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.16.27012          hf0eaf9b_1
watchdog                  0.10.2                   py37_0    conda-forge
wcwidth                   0.1.8                      py_0    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.34.2                     py_1    conda-forge
win_inet_pton             1.1.0                    py37_0    conda-forge
wincertstore              0.2                   py37_1003    conda-forge
wrapt                     1.12.1           py37h8055547_1    conda-forge
yaml                      0.2.2                hfa6e2cd_1    conda-forge
yapf                      0.29.0                     py_0    conda-forge
zeromq                    4.3.2                h6538335_2    conda-forge
zipp                      3.1.0                      py_0    conda-forge
zlib                      1.2.11            h2fa13f4_1006    conda-forge

you are listening plain python installations, right? of course there are a lot of differences. i have pandas, scipy, seaborn, numpy, matplotlib, etc. installed.
so both of my environments are broken right now (i.e. no sypder, could only work with jupyter notebook from now on). i'm off to delete my environments and recreate them. some work ahead...

@manfred2020 yep a basic installation, what I was thinking is that maybe you guys can create an env with the commands I supply just to check if the issue is related with some sort of package version or is something else. Also, could you provide your envs package list to check them then?

@dalthviz Sorry, I do not have a D drive. Only a C Drive
My list of packages running the first command is this one:
Package Plan
environment location: C:\Users\SMORENO\AppData\Local\Continuum\miniconda3\envs\python37_new
added / updated specs:
- python=3.7
- spyder=4.1.0
The following NEW packages will be INSTALLED:
alabaster          conda-forge/noarch::alabaster-0.7.12-py_0
argh               conda-forge/win-64::argh-0.26.2-py37_1001
astroid            conda-forge/win-64::astroid-2.3.3-py37_1
atomicwrites       conda-forge/noarch::atomicwrites-1.3.0-py_0
attrs              conda-forge/noarch::attrs-19.3.0-py_0
autopep8           conda-forge/noarch::autopep8-1.5-py_0
babel              conda-forge/noarch::babel-2.8.0-py_0
backcall           conda-forge/noarch::backcall-0.1.0-py_0
bcrypt             conda-forge/win-64::bcrypt-3.1.7-py37hfa6e2cd_0
bleach             conda-forge/noarch::bleach-3.1.3-pyh8c360ce_0
ca-certificates    conda-forge/win-64::ca-certificates-2019.11.28-hecc5488_0
certifi            conda-forge/win-64::certifi-2019.11.28-py37hc8dfbb8_1
cffi               conda-forge/win-64::cffi-1.14.0-py37ha419a9e_0
chardet            conda-forge/win-64::chardet-3.0.4-py37hc8dfbb8_1006
cloudpickle        conda-forge/noarch::cloudpickle-1.3.0-py_0
colorama           conda-forge/noarch::colorama-0.4.3-py_0
cryptography       conda-forge/win-64::cryptography-2.8-py37hb32ad35_1
decorator          conda-forge/noarch::decorator-4.4.2-py_0
defusedxml         conda-forge/noarch::defusedxml-0.6.0-py_0
diff-match-patch   conda-forge/noarch::diff-match-patch-20181111-py_0
docutils           conda-forge/win-64::docutils-0.16-py37hc8dfbb8_1
entrypoints        conda-forge/win-64::entrypoints-0.3-py37hc8dfbb8_1001
flake8             conda-forge/win-64::flake8-3.7.9-py37hc8dfbb8_1
future             conda-forge/win-64::future-0.18.2-py37hc8dfbb8_1
icu                conda-forge/win-64::icu-64.2-he025d50_1
idna               conda-forge/noarch::idna-2.9-py_1
imagesize          conda-forge/noarch::imagesize-1.2.0-py_0
importlib-metadata conda-forge/win-64::importlib-metadata-1.5.0-py37_0
importlib_metadata conda-forge/noarch::importlib_metadata-1.5.0-1
intervaltree       conda-forge/noarch::intervaltree-3.0.2-py_0
ipykernel          conda-forge/win-64::ipykernel-5.1.4-py37h5ca1d4c_0
ipython            conda-forge/win-64::ipython-7.13.0-py37h5ca1d4c_0
ipython_genutils   conda-forge/noarch::ipython_genutils-0.2.0-py_1
isort              conda-forge/win-64::isort-4.3.21-py37hc8dfbb8_1
jedi               conda-forge/win-64::jedi-0.14.1-py37_0
jinja2             conda-forge/noarch::jinja2-2.11.1-py_0
jpeg               conda-forge/win-64::jpeg-9c-hfa6e2cd_1001
jsonschema         conda-forge/win-64::jsonschema-3.2.0-py37hc8dfbb8_1
jupyter_client     conda-forge/noarch::jupyter_client-6.0.0-py_0
jupyter_core       conda-forge/win-64::jupyter_core-4.6.3-py37hc8dfbb8_1
keyring            conda-forge/win-64::keyring-21.1.1-py37_0
lazy-object-proxy  conda-forge/win-64::lazy-object-proxy-1.4.3-py37hfa6e2cd_1
libclang           conda-forge/win-64::libclang-9.0.1-default_hf44288c_0
libpng             conda-forge/win-64::libpng-1.6.37-h7602738_0
libsodium          conda-forge/win-64::libsodium-1.0.17-h2fa13f4_0
libspatialindex    conda-forge/win-64::libspatialindex-1.9.3-he025d50_3
markupsafe         conda-forge/win-64::markupsafe-1.1.1-py37h8055547_1
mccabe             conda-forge/noarch::mccabe-0.6.1-py_1
mistune            conda-forge/win-64::mistune-0.8.4-py37hfa6e2cd_1000
nbconvert          conda-forge/win-64::nbconvert-5.6.1-py37_0
nbformat           conda-forge/noarch::nbformat-5.0.4-py_0
numpydoc           conda-forge/noarch::numpydoc-0.9.2-py_0
openssl            conda-forge/win-64::openssl-1.1.1d-hfa6e2cd_0
packaging          conda-forge/noarch::packaging-20.1-py_0
pandoc             conda-forge/win-64::pandoc-2.9.2-0
pandocfilters      conda-forge/noarch::pandocfilters-1.4.2-py_1
paramiko           conda-forge/win-64::paramiko-2.7.1-py37_0
parso              pkgs/main/noarch::parso-0.5.2-py_0
pathtools          conda-forge/noarch::pathtools-0.1.2-py_1
pexpect            conda-forge/win-64::pexpect-4.8.0-py37hc8dfbb8_1
pickleshare        conda-forge/win-64::pickleshare-0.7.5-py37hc8dfbb8_1001
pip                conda-forge/noarch::pip-20.0.2-py_2
pluggy             conda-forge/noarch::pluggy-0.12.0-py_0
prompt-toolkit     conda-forge/noarch::prompt-toolkit-3.0.4-py_0
prompt_toolkit     conda-forge/noarch::prompt_toolkit-3.0.4-0
psutil             conda-forge/win-64::psutil-5.7.0-py37h8055547_1
pycodestyle        conda-forge/noarch::pycodestyle-2.5.0-py_0
pycparser          conda-forge/noarch::pycparser-2.20-py_0
pydocstyle         conda-forge/noarch::pydocstyle-5.0.2-py_0
pyflakes           conda-forge/noarch::pyflakes-2.1.1-py_0
pygments           conda-forge/noarch::pygments-2.6.1-py_0
pylint             conda-forge/win-64::pylint-2.4.4-py37_0
pynacl             conda-forge/win-64::pynacl-1.3.0-py37h2fa13f4_1001
pyopenssl          conda-forge/noarch::pyopenssl-19.1.0-py_1
pyparsing          conda-forge/noarch::pyparsing-2.4.6-py_0
pyqt               conda-forge/win-64::pyqt-5.12.3-py37h6538335_1
pyrsistent         conda-forge/win-64::pyrsistent-0.15.7-py37h8055547_1
pysocks            conda-forge/win-64::pysocks-1.7.1-py37hc8dfbb8_1
python             conda-forge/win-64::python-3.7.6-h5b45d93_4_cpython
python-dateutil    conda-forge/noarch::python-dateutil-2.8.1-py_0
python-jsonrpc-se~ conda-forge/noarch::python-jsonrpc-server-0.3.4-py_0
python-language-s~ conda-forge/win-64::python-language-server-0.31.8-py37hc8dfbb8_0
python_abi         conda-forge/win-64::python_abi-3.7-1_cp37m
pytz               conda-forge/noarch::pytz-2019.3-py_0
pywin32            conda-forge/win-64::pywin32-227-py37hfa6e2cd_0
pywin32-ctypes     conda-forge/win-64::pywin32-ctypes-0.2.0-py37hc8dfbb8_1001
pyyaml             conda-forge/win-64::pyyaml-5.3-py37h8055547_1
pyzmq              conda-forge/win-64::pyzmq-19.0.0-py37h8c16cda_1
qdarkstyle         conda-forge/noarch::qdarkstyle-2.8-py_0
qt                 conda-forge/win-64::qt-5.12.5-h7ef1ec2_0
qtawesome          conda-forge/noarch::qtawesome-0.7.0-py_0
qtconsole          conda-forge/noarch::qtconsole-4.7.1-py_0
qtpy               conda-forge/noarch::qtpy-1.9.0-py_0
requests           conda-forge/noarch::requests-2.23.0-pyh8c360ce_2
rope               conda-forge/noarch::rope-0.16.0-py_0
rtree              conda-forge/win-64::rtree-0.9.4-py37h804a536_1
setuptools         conda-forge/win-64::setuptools-46.0.0-py37hc8dfbb8_2
six                conda-forge/noarch::six-1.14.0-py_1
snowballstemmer    conda-forge/noarch::snowballstemmer-2.0.0-py_0
sortedcontainers   conda-forge/noarch::sortedcontainers-2.1.0-py_0
sphinx             conda-forge/noarch::sphinx-2.4.4-py_0
sphinxcontrib-app~ conda-forge/noarch::sphinxcontrib-applehelp-1.0.2-py_0
sphinxcontrib-dev~ conda-forge/noarch::sphinxcontrib-devhelp-1.0.2-py_0
sphinxcontrib-htm~ conda-forge/noarch::sphinxcontrib-htmlhelp-1.0.3-py_0
sphinxcontrib-jsm~ conda-forge/noarch::sphinxcontrib-jsmath-1.0.1-py_0
sphinxcontrib-qth~ conda-forge/noarch::sphinxcontrib-qthelp-1.0.3-py_0
sphinxcontrib-ser~ conda-forge/noarch::sphinxcontrib-serializinghtml-1.1.4-py_0
spyder             conda-forge/win-64::spyder-4.1.0-py37hc8dfbb8_0
spyder-kernels     conda-forge/win-64::spyder-kernels-1.9.0-py37hc8dfbb8_0
sqlite             conda-forge/win-64::sqlite-3.30.1-hfa6e2cd_0
testpath           conda-forge/noarch::testpath-0.4.4-py_0
tornado            conda-forge/win-64::tornado-6.0.4-py37hfa6e2cd_0
traitlets          conda-forge/win-64::traitlets-4.3.3-py37hc8dfbb8_1
typed-ast          conda-forge/win-64::typed-ast-1.4.1-py37hfa6e2cd_0
ujson              conda-forge/win-64::ujson-1.35-py37h63f7a3c_1002
urllib3            conda-forge/win-64::urllib3-1.25.7-py37hc8dfbb8_1
vc                 pkgs/main/win-64::vc-14.1-h0510ff6_4
vs2015_runtime     pkgs/main/win-64::vs2015_runtime-14.16.27012-hf0eaf9b_1
watchdog           conda-forge/win-64::watchdog-0.10.2-py37_0
wcwidth            conda-forge/noarch::wcwidth-0.1.8-py_0
webencodings       conda-forge/noarch::webencodings-0.5.1-py_1
wheel              conda-forge/noarch::wheel-0.34.2-py_1
win_inet_pton      conda-forge/win-64::win_inet_pton-1.1.0-py37_0
wincertstore       conda-forge/win-64::wincertstore-0.2-py37_1003
wrapt              conda-forge/win-64::wrapt-1.12.1-py37h8055547_1
yaml               conda-forge/win-64::yaml-0.2.2-hfa6e2cd_1
yapf               conda-forge/noarch::yapf-0.29.0-py_0
zeromq             conda-forge/win-64::zeromq-4.3.2-h6538335_2
zipp               conda-forge/noarch::zipp-3.1.0-py_0
zlib               conda-forge/win-64::zlib-1.2.11-h2fa13f4_1006
I tried comparing them with text compare but it says "Sorry, but finding all the differences for these two texts takes too long", I noticed asn1crypto does not show for me though

ok, got it.
i did you D installation and compared it with yours.
they are identical.

Some more questions:

What happens if you open the Anaconda Prompt and type spyder?
And what's the result when you start Spyder graphically from its entry on the Anaconda menu?

in my 3.7.6 env. (where i still have spyder 4.1):
graphically: works now
prompt: works now
didn't work before (see above). only thing i changed since then: removed spyder from my 3.8.2 env. and created your conda forge test env (where spyder 4.1 works too).
very strange....

in fact, I am getting the same error if I start spyder as a shortcut (or from console)
But if I start the same spyder 4.1 from Anaconda Navigator, it works OK.

yes, via desktop shortcut sypder 4.1 still doesn't work (works from prompt and anaconda)

This is very strange.. For me it works the same either from console or shortcut.. The GUI opens for me in both cases but its the kernel that shows the error. Maybe these are seperate issues?

Testing with the desktop shortcut I can reproduce this! Thank you all for the feedback! for now please launch Spyder from the Anaconda prompt/Anaconda Navigator

Testing with the desktop shortcut I can reproduce this! Thank you all for the feedback! for now please launch Spyder from the Anaconda prompt/Anaconda Navigator

Working perfect here through with Anaconda Prompt

I get the exact same problem when starting spyder 4.1.0 from Windows 10 Start Menu shortcut.  Running spyder from anaconda prompt (type "spyder") or from Anaconda Navigator menu both work perfectly.  Spyder 4.1.0 is installed into a conda 4.8.3 env (fully updated) running Python 3.7.6.  Spyder isn't installed in simple Miniconda base environment where Navigator is installed.

Thanks everyone for your feedback. We were analyzing the problem with @dalthviz for the last hour and finally found out the root cause for it.
We'll release 4.1.1 with a fix for this as soon as we can.

Our 4.1.1 version, with a fix for this problem is part of Anaconda now. You can update with
conda install spyder=4.1.1


Text within list item breaks numbering

Before starting I wanted to thank you for your work, inlyne is a really amazing tool I love it!
Description
Inlyne version 0.3.2 doesn't render numbered lists correctly in some scenarios, see attached screenshot.

It seems like numbered lists with blanklines remove numbering completely.
Reproduce
Just open the Ratatui release file.
Correct behavior
I won't get into details here are inlyne main branch seems to have a functioning behavior.
Additional info
The bug also occurs in 0.3.1.
It works perfectly with the main branch of inlyne though, so maybe there isn't even something to fix ...
PS: feel free to rename the issue to something more expressive

Thanks for the very detailed bug report!

It works perfectly with the main branch of inlyne though, so maybe there isn't even something to fix ...

Well that should make the fix easy at the very least. The main branch isn't 100% right since it doesn't indent the nested element, but it does correctly keep the lists number/bullet which is good
I'll work on backporting the fix and releasing v0.3.3 over the next few days (intentionally going slow in case any other issues pop up with v0.3.2)

This should be fixed with the v0.3.3 release
https://github.com/trimental/inlyne/releases/tag/v0.3.3


[5.3] nested many to many relationship loading

Laravel Version: 5.3.*
PHP Version: 7.0
Database Driver & Version: mysql

Description:
The query builder returns a inner join with a IN statement with all the ids from many to many relationship.
In my case this results with a IN statement that has 11 thousand ids inside of it.
As you can image that is so slow that crashes.
Is that the way it is suppose to work or am i misusing the ORM?
To me the proper way to do that is a join on the pivot table which the relationship instead of doing a IN statement.

Please as on the forums, we try to keep the repo for bug reporting only :)


first commit for hacktoberfest

:warning: This is a Pull Request Template :warning:

Checkout all the things you've done by placing an x within the square braces below.

Proposed Changes

[x] I've read the CODE OF CONDUCT and abide to it.
[x] I've read the CONTRIBUTING.md
[x] I've forked the repository.
[x] I've created a branch and made my changes in it.
[x] I understand opening a PULL REQUEST doesn't mean it will be merged for sure.

This is a first try to commit to open source for hacktoberfest.


Bug fix hang on insert into select when enable pipeline engine

Proposed changes
Fix hang on insert into select when enable pipeline engine.
This problem only occurs under multi-be, so there is no need to add test case.
Problem summary
Describe your changes.
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.64 seconds
stream load tsv:          430 seconds loaded<PHONE_NUMBER>9 Bytes, about 165 MB/s
stream load json:         21 seconds loaded<PHONE_NUMBER> Bytes, about 107 MB/s
stream load orc:          60 seconds loaded<PHONE_NUMBER> Bytes, about 17 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230425153314_clickbench_pr_135092.html

run buildall

run buildall


Queue timeout error when using typescript_go_to_definition

The function typescript_go_to_definition used to work quite well for me and I have it mapped to my middle-click so that when I middle-click a name, it will jump to the code definition. Recently, this hasn't been working reliably and I don't know what changed. It started requiring multiple clicks, first click would pause then nothing, second click the same, third click would jump to the definition with no pause.
Now it hangs almost all the time and doesn't jump to the definition and opening the console log, it says queue timeout after each click.
I excluded a lot of folders and files from indexing like node_modules and build files and that sped up indexing but hasn't fixed this issue.
Is there a way I can diagnose this like a verbose mode to see where it is getting stuck?

Error reporting also stopped working around the same time this started but code auto-completion is ok. Maybe Sublime handles the auto-completion.


Would be nice to use a IconCollectionName object to reference icons in Compose

Currently, icons are kinda "standalone" when using "Copy compose" button and not linked to the collection they come from.
Defining an object LucideIcons and then, each icon being an extension of it would group icons from the same collection and disambiguate at call site.
object LucideIcons

public val LucideIcons.Calendar: ImageVector
    get() {
...

Then, at call site
Before
import Calendar as LucideIconCalendar

@Composable
fun MyScreen() {
  Column {
    Icon(LucideIconCalendar, null)
    Icon(Calendar, null) // barely manageable due to Jvm auto imports
  }
}

After
import LucideIcons
import LucideIcons.Calendar

@Composable
fun MyScreen() {
  Column {
    Icon(LucideIcons.Calendar, null)
  }
}

Would it help if I added a button to copy the reference of the Lucide gradle dependency on the site instead?

I currently need to download icons to tweak them somehow, like forcing the stroke line width to a value I want (related to #2).


use yf.download for 2017 data, will have 9/4/2017 as a index, while no data

2017-09-04 is Labor day. If we download data use start and end dates such that this day is in between, then the returned data frame contains this day in the index, while data is empty.

You could drop NaN values and it would be gone


Running Docker NAV container with external SQL server

Is scenario of using SQL server outside docker image supported?
I suppose it is impossible to run NAV service on docker container, without joining it to domain, where SQL server is located.
Other option is to specify database credentials, but I cannot find the way to submit those, when running docker image.

There are a lot of options supported - some are documented in the how-to.docx in this repo, some will be documented later - but you can definitely use SQL outside the image. The SQL inside the image is only there for demo etc.

Thank you! I have managed to get working with NavUserPasswordauthentication using SetupDatabase override script example provided in how-to. My mistake was useSSL=N which is not valid for NavUserPassword authentication. I have also changed TrustSQLServerCertificate to true.
Do you have any sources about how i could make docker to access other company servers using windows authentication?

@acyvas I can recommend using gMSA.
We use it and works really well.

I have tried running container with credentialspec on Windows 10, but it times out.
@Koubek, does docker support gMSA on Windows 10?

@acyvas I am not using gMSA on my own notebook but I can try it and confirm the results.

@Koubek I have tried running container with credentialspec on Windows Server 2016 and at least packages runs fine and I can see right domain when I run nltest /parentdomain from docker container.
Looks that there are problems with gMSA support on Windows 10.
It has been reported on this post as well:
https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/manage-serviceaccounts

I have tested it on my Win10 and no success. Anyway, I am personally OK with the situation. We have automated the process of the creation of the containers via gMSA thanks to using security groups and multiple scripts (within a specific PS module deployed on each Docker host). It is very easy for us to spin any container with gMSA from WinServer (docker hosts - we have 3 hosts right now).
But I can`t imagine doing this from each user machine. Of course, you could create the groups as well but from my point of view, you would need to have a really good security strategy for these groups etc.

I did set gMSA, but running a docker image with external SQL server returns this error in docker log right after Creating Windows User step...:
Microsoft Dynamics NAV Server is currently not ready to serve requests. Try again later, or contact your system administ
rator.

I will check this probably the next week... I still don`t have the fall Creator Update installed on my notebook and will install the next week.


Deleting a feature, disease, interpretation on a phenopacket doesn't remove it

After deleting a feature, disease interpretation from a phenopacket, closing the phenopacket tab, reopening it will show these feature, disease, interpretation previously deleted. The deletion is not working properly.

this bug seems to be the same for all elements of a phenopacket. (medical actions, measurements, biosamples etc)

@belkassaby The deletion is now working. However, when I delete a disease it is asking me twice to confirm the deletion. It asks me once as expected. Then, when I leave that phenopacket to look at another one I get a second deletion confirmation request. It does not matter if I choose the "No" option since it is already deleted. I believe it is only disease that is doing this.

@BethSundberg https://github.com/TheJacksonLaboratory/PhenopacketLab/pull/254 should fix this

This is now working properly on the dev site.


Fitbit Sensor Take Deux

Description:
This adds a sensor for Fitbit
Related issue (if applicable): #1028, #1662
Example entry for configuration.yaml (if applicable):
sensor:
  platform: fitbit
  client_id: 'CLIENT_ID'
  client_secret: 'CLIENT_SECRET'
  monitored_resources:
    - "activities/steps"

Checklist:
If code communicates with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

Looks good! :dolphin:


fix typo in basic_mapping.xml

We don't use the old User Guide anymore. Now, we use the asciidoc based Usre Guide and the old one is only available in 4.x branches which are no longer maintained.


The "path" argument must be of type string. Received undefined

Hello guys !
Im trying to install your component but I have some issues with. When I try to compile, I have this error :
internal/validators.js:124
    throw new ERR_INVALID_ARG_TYPE(name, 'string', value);
    ^

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at validateString (internal/validators.js:124:11)
    at Object.resolve (path.js:980:7)
    at afterResolveFn (/var/www/project/node_modules/offline-plugin/lib/index.js:213:42)
    at Hook.eval [as callAsync] (eval at create (/var/www/project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:7:1)
    at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (/var/www/project/node_modules/tapable/lib/Hook.js:18:14)
    at /var/www/project/node_modules/webpack/lib/NormalModuleFactory.js:287:30
    at eval (eval at create (/var/www/project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at /var/www/project/node_modules/webpack/lib/NormalModuleFactory.js:550:7
    at /var/www/project/node_modules/webpack/lib/NormalModuleFactory.js:117:11
    at /var/www/project/node_modules/webpack/lib/NormalModuleFactory.js:582:8 {
  code: 'ERR_INVALID_ARG_TYPE'
}

Any idea where is the problem ?  I just added the plugin and used it without options.

This plugin is not compatible with webpack 5.  I believe the error your encountering is related to that : https://github.com/NekR/offline-plugin/issues/491

Yeah, didn't see this issue, seems a duplicate, thank you for your answer @oller.

NP, in fact commenting on that I noticed https://github.com/NekR/offline-plugin/issues/497 which seems like a fork with webpack 5 compatibility.  Props to @LeComptoirDesPharmacies


Removed Travis CI & added GitHub actions

Travis CI is not working anymore
To run the tests, added GitHub actions

@tarcieri can you help me remove travis-ci from the builds? only removing .travis.yml is not enough


SD Card Driver (API branch)

Created an initial skeleton of the second revision of the SD card driver.

Codecov Report

Merging #116 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #116   +/-   ##
=======================================
  Coverage   73.23%   73.23%
=======================================
  Files          18       18
  Lines         710      710
=======================================
  Hits          520      520
  Misses        190      190

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 641e976...f1596c4. Read the comment docs.

Closing stale PR.


fix(auto-fill): change apply type will undo last commands

close https://github.com/dream-num/univer-pro/issues/1390

Pull Request Checklist

[ ] Related tickets or issues have been linked in the PR description (or missing issue).
[ ] Naming convention is followed (do please check it especially when you created new plugins, commands and resources).
[ ] Unit tests have been added for the changes (if applicable).
[ ] Breaking changes have been documented (or no breaking changes introduced in this PR).

来回勾选复制单元格&填充序列后再勾选不带格式填充，还是会携带条件格式
操作步骤
1、选中内容设置条件格式-数据条
2、选中步骤一的内容进行下拉填充
3、在复制单元格/仅填充样式来回切换后，勾选不带格式填充
预期结果：勾选不带格式填充，填充的内容不再展示条件格式
2024_9_4 16_07_44 video.webm
实际结果：勾选不带格式填充，填充的内容展示条件格式

2024_9_4 16_04_20 video.webm


tests/kernel/timer/timer_api failed on reel_board and mec15xxevb_assy6853.

Describe the bug
for mec15xxevb_assy6853, it was the same error log as reel_board.
To Reproduce
Steps to reproduce the behavior:

west build -b reel_board -p auto tests/kernel/timer/timer_api/
west flash
See error

Screenshots or console output
*** Booting Zephyr OS build zephyr-v2.2.0-1006-g41712204059e  ***
Running test suite timer_api
starting test - test_time_conversions
PASS - test_time_conversions
starting test - test_timer_duration_period
PASS - test_timer_duration_period
starting test - test_timer_period_0
PASS - test_timer_period_0
starting test - test_timer_expirefn_null
PASS - test_timer_expirefn_null
starting test - test_timer_periodicity
PASS - test_timer_periodicity
starting test - test_timer_status_get
PASS - test_timer_status_get
starting test - test_timer_status_get_anytime
PASS - test_timer_status_get_anytime
starting test - test_timer_status_sync
PASS - test_timer_status_sync
starting test - test_timer_k_define
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:64: duration_expire: interval >= 100 is false
FAIL - test_timer_k_define
starting test - test_timer_user_data
PASS - test_timer_user_data
starting test - test_timer_remaining
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:548: test_timer_remaining: (exp_ticks - now) <= dur_ticks / 2 is false
FAIL - test_timer_remaining
starting test - test_timeout_abs
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:561: test_timeout_abs: t2.ticks == t.ticks is false
FAIL - test_timeout_abs
Test suite timer_api failed.
PROJECT EXECUTION FAILED
Environment (please complete the following information):

OS: fedora28
Toolchain: zephyr-sdk-0.11.1
Commit ID:<PHONE_NUMBER>

for mec board, this test passed, but for reel board, it still failed.
starting test - test_timer_remaining
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:547: test_timer_remaining: rem_ticks <= dur_ticks / 2 is false
FAIL - test_timer_remaining
starting test - test_timeout_abs
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:591: test_timeout_abs: k_timer_remaining_ticks(&remain_timer) + k_uptime_ticks() + 1 == exp_ticks is false
FAIL - test_timeout_abs
Test suite timer_api failed.

Can someone recheck the reel_board?  There was an unrelated fix on nRF that went in a week or three ago that I'm guessing is what was reported here.

tested it with the latest commit ed2d263e on reel_board, it still was the same error.

  This failure on my nrf5 board was the one tracked in #24784, and it's not fixed yet.  Can you check to see that one of the workarounds listed there resolves this?
I think we can close this as a dup, if so.

tested the patch #24817 and #24972 in #24784, the test unit 'test_timer_k_define' and 'test_timer_remaining' passed, but the 'test_timeout_abs' still failed like below.
starting test - test_timeout_abs
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:591: test_timeout_abs: k_timer_remaining_ticks(&remain_timer) + k_uptime_ticks() + 1 == exp_ticks is false
FAIL - test_timeout_abs

I can confirm that the above and other checks in this test fail on reel_board with varying frequency.
As @andyross suggests this is likely due to #24784.  All the failing tests are comparing durations calculated from HFCLK (k_busy_wait()) against durations calculated from LFCLK (sys_clock i.e. timeout/k_timer).  In all cases the actual error is one tick the wrong side of the check.
Using a 1pps signal from a Maxim DS3231 TCXO the reel board 32 KiHz LFXT runs about 60 ppm fast on both the reel boards I have, compared to between 25-30 ppm for most Nordic dev boards I've tested.  (@jfischer-phytec-iot fyi, not sure this is known.)  This is probably why the out-of-spec accuracy checks fail more frequently on this platform.
I still feel that Zephyr cannot assume that the clock used for k_busy_wait() maintains a stable and precise relationship with the clock used for sys_clock, and that as a result these tests are inherently not portable.
If the tests must pass, there are several paths:

The expedient and misleading one is to test Nordic hardware with a non-standard Nordic clock configuration that hides the error by locking LFCLK to HFCLK.  Thid can be done quickly and should only affect Nordic hardware, but it misrepresents the behavior of that hardware in normal applications.
The justifiable but complex and fragile one is to refactor the tests to compare the measured error against a target-specific tolerance, which for Nordic could be up to 13%.  This is very complex and might break on other platforms that use significantly higher or lower ticks-per-second values that complicate accurate calculation of the clock error.
The expedient and justifiable one is to manually tweak the thresholds again.

If the goal is to get medium bugs below a threshold a fourth path is to close this as a duplicate of #24784, and leave that unresolved for now.
Unless I hear otherwise I'll tweak the thresholds again until the test passes reliably on the two reel boards I have.  @carlescufi

Using a 1pps signal from a Maxim DS3231 TCXO the reel board 32 KiHz LFXT runs about 60 ppm fast on both the reel boards I have, compared to between 25-30 ppm for most Nordic dev boards I've tested. (@jfischer-phytec-iot fyi, not sure this is known.) This is probably why the out-of-spec accuracy checks fail more frequently on this platform.

Try to find out what is wrong. It should not be far from 20ppm. I can confirm that the test is failing on the reel board, even on the first version, where the layout is similar to nRF52804 DK.


Re-enable vectored writes

Tokio 0.3 removed an API to do vectored writes. During the upgrade, hyper disabled its support, but we should try to re-enable before releasing 0.14.
This requires a proposal for a new API in Tokio proper.

Proposal in Tokio: https://github.com/tokio-rs/tokio/issues/3135

tokio-rs/tokio#3149 has been merged and tokio 0.3.4 was released 2 days ago which includes this feature. :tada:

It's being worked on in https://github.com/hyperium/hyper/pull/2338 :D


⚠ Test Homepage has degraded performance

In ba96964, Test Homepage (https://test.h-edu.cz) experienced degraded performance:

HTTP code: 200
Response time: 756 ms

Resolved: Test Homepage performance has improved in a09d9ff.


Proof Of Stake

Description
Golden Gate should be POS for the mainnet release. Implement a POS mechanism with a staking and reward system for validators and delegated users. Validators will be controlled by ValidatorManager but will be required to stake.
Definition of "Done"
The validators are required to stake to become a validator. Validators and staked users receive rewards.
Default inflation is from 16% to 4% over 20 years but can be configured by democracy.
Suggested Solution
Add pallets:

staking
democracy
council
collective
membership
other POS mechanisms

Resolved by: #47


Possible improvements?

Hello,
Thank you for the (super fast and) very useful utility. I can think of a couple of optional improvements:

There could be a default pager when the output is more than one screen (similar to bat)
Colors could be also a good nice option

But again, good job for the useful util :+1:

I would prefer that you divide this issue into 2, one for pager, and one for color.

I wasn't sure if you'll take them into consideration, but sure, I'll create two issues then :slightly_smiling_face: (and close this one)


Filenotfounderror: WML Credentials not found

I installed the plugin and when I tried to call it, I am getting this error, im trying this in IBM cloud

@dhruv5995 Do you think the new release has addressed this issue?

Hi @Aruna-Saraswathy you need to provide the WML credentials and a deployment space name in order to be able to connect to the WML API Client. Please refer to the following file and based on the case that applies to you, create a .env file in the same directory as .env.template. This will enable the authentication and connection to the WML API Client.

Closing this now. Feel free to reopen if you have any questions or problem using the plugin.


Fix/rm systemd

Currently includes https://github.com/advancedtelematic/meta-updater/pull/616.
Note that this does not change the fact that aktualizr is built in meta-updater as a systemd service.

Oops, this was misguided. Please ignore, we do still use systemd for some secondary-related things. However, we probably don't need to build that by default.


Fix typo in comment

Wow, those are some pretty astounding typos.  Thanks for this!

I continue to believe that autocorrect is responsible for this.
😈


Align the count parameter of splitn with other languages

Rendered

See also https://github.com/rust-lang/rust/issues/14899

/cc @aturon or @alexcrichton . This proposes to change currently-stable methods in a backwards-incompatible manner.

I'm somewhat ambivalent on this one way or the other. I find the splitn(0, ",") case a little surprising where 0 is the number of returned matches (e.g. when I splitn I expect to get the remainder back at some point) but otherwise I agree that split(2, ",") looks nicer if it's actually yielding two elements.
I would personally vote for just breaking code without jumping through hoops to nudge crates to update.

FWIW, the regex crate is using the behavior being recommended here (n specifies the number of items returned). I wrote it this way because it seems more intuitive to me. However, I would be in favor of changing it to whatever is decided by this RFC.

I thought I had commented on this RFC a week ago, but my comment seems to be gone!
I definitely agree that the proposed semantics is the more intuitive one -- I've mistakenly assumed these semantics in the past, and seen other people do the same.
@alexcrichton makes a good point about the 0 case; we would probably want to panic (i.e. treat that as a contract violation). But I don't think that's a deal breaker or anything.
My bigger concern is: how can we roll this out in the short time before beta without silently breaking a lot of code? @alexcrichton, I lack your confidence that we can just change the semantics, but maybe you can elaborate on how you see that working out?
Otherwise, we might need to consider some multi-step deprecation (assuming we want to stick with the current names), which would be a bit painful and time consuming.

I'm in favor of the change. The current functionality loses information: there is no way to preserve the tail without doing a full split+rejoin of the tail. I also find the proposed functionality more natural and have explicitly relied on it a number of times in C# whereas I've had to work around JavaScript's functionality (see below).
In mostly valid Rust (I can't find a join method in the std library), the current functionality makes maintaining the tail difficult:
let a: Vec<_> = "a,b,c"
    .split(',')
    .iter()
    .take(1)
    .chain([a.iter().skip(1).join(',')].iter())
    .collect();
assert_eq!(a, ["a", "b,c"]);

With the proposed functionality the tail is maintained and can easily be trimmed with splitn(n+1).take(n).
let a: Vec<_> = "a,b,c".splitn(2, ',').collect();
assert_eq!(a, ["a", "b,c"]);

let n = 2
let a: Vec<_> = "a,b,c".splitn(n - 1, ',').take(n).collect();
assert_eq!(a, ["a", "b"]);

As a side note JavaScript provides another variant where N is the number of items returned and any additional items are lost.
'a,b,c'.split(',', 2)
["a", "b"]

@alexcrichton makes a good point about the 0 case; we would probably want to panic (i.e. treat that as a contract violation). But I don't think that's a deal breaker or anything.

If we accept this RFC I'd personally vote to accept it as-is where the integer parameter is the precise number of times the iterator will return Some, and the last time it returns Some will include the entire remainder of the string (which covers the 0 case). I'm a little surprised at the semantics, but they make the most sense to me!

My bigger concern is: how can we roll this out in the short time before beta without silently breaking a lot of code? @alexcrichton, I lack your confidence that we can just change the semantics, but maybe you can elaborate on how you see that working out?

I wouldn't necessarily say I had a grand vision here, mostly it was just that tests would catch this bug and general usage would weed out cases pretty quickly. This is a pretty minor method and I feel it's not worth an elaborate migration plan.

@alexcrichton Ah, I missed that point about 0. That seems OK.
I'm potentially OK with a direct change, but we would need to advertise it widely.

I'm vaguly :+1: about this RFC. Consistency with more languages is good. I agree with @alexcrichton that it probably doesn't need an elaborate upgrade path.

I'm tentatively in favor of this as well. I'm mildly concerned about silent breakage of existing, but I don't have any good ideas for how to fix that without causing more trouble than it's worth, unless we wanted to do something trivial like swap the order of the parameters (thus forcing all clients to update their code, except the clients may not realize the semantics changed).

So, it seems there are roughly three way to possibly handle the migration:

Do nothing, just change behavior. This will silently break code, and only be discovered if there are tests that can uncover the difference.
Temporary change signature or method name. This would alert users to the new semantic by causing compilation errors, and the temporary rename could indicated the changed semantic, eg splitn -> splitn_minus_one.
The issue here is that after some time passed, we would likely want to change the name back, requiring up-to-date users to fix compilation errors again to keep up.
A variant would be to also keep the old methods around temporary as deprecated, and finally switch them over after some transition time, but that would also incur two breakages for users that want to track master.
Remove the methods from the str, and add a temporary trait, say NewSplitNSemantic that is not in the prelude. User code would break, and can be fixed by importing that trait manually, hopefully alerting the user that the semantic changed in the meantime. After the migration time passed, the methods can be added back to str itself and the trait become deprecated. Done correctly, this only requires users that follow master to fix their code once, instead of twice.

@Kimundi
Thanks for laying those options out! I think we probably lack the time to do (2) before the beta.
It seems like (3) could be a pretty reasonable option, though. In particular, we could remove the trait altogether during the beta cycle, meaning that by 1.0 we'd have a clean API.
@alexcrichton Thoughts on that migration strategy?

We've already got quite the hefty amount of deprecated code in the standard library, so I'd still be voting for the cold-turkey approach, but option (3) does indeed sound plausible to me!

While this is a late-breaking change, there's broad agreement that it removes a potential footgun. We've decided to move forward, and to change the semantics in place (together with a very loud PSA that this is happening as part of the beta release).
Thanks for the RFC!

Tracking issue


Sync check incomplete notifications

Issue summary

On a certain failure in the sync check, we don't get any logs on the Slack channel, just the failure notification.

This is not optimal as it forces SSHing into the sync machine (which may be gone, along with the logs).
The forest-tester container has some errors that may indicate the culprit
root@forest-sync-check:~# docker logs forest-tester
/volumes/sync_check/sync_check.rb:18: warning: already initialized constant SLACK_TOKEN
/volumes/sync_check/sync_check_process.rb:12: warning: previous definition of SLACK_TOKEN was here
/volumes/sync_check/sync_check.rb:19: warning: already initialized constant CHANNEL
/volumes/sync_check/sync_check_process.rb:13: warning: previous definition of CHANNEL was here
/volumes/sync_check/sync_check.rb:18: warning: already initialized constant SLACK_TOKEN
/volumes/sync_check/sync_check_process.rb:12: warning: previous definition of SLACK_TOKEN was here
/volumes/sync_check/sync_check.rb:19: warning: already initialized constant CHANNEL
/volumes/sync_check/sync_check_process.rb:13: warning: previous definition of CHANNEL was here
D, [2023-11-14T13:18:30.435875 #7] DEBUG -- : Running `forest-tool` command with db destroy --chain mainnet --force
/usr/lib/ruby/3.0.0/open3.rb:221:in `spawn': No such file or directory - docker (Errno::ENOENT)
        from /usr/lib/ruby/3.0.0/open3.rb:221:in `popen_run'
        from /usr/lib/ruby/3.0.0/open3.rb:102:in `popen3'
        from /usr/lib/ruby/3.0.0/open3.rb:289:in `capture3'
        from /volumes/sync_check/sync_check_process.rb:28:in `run_forest_container'
        from /volumes/sync_check/sync_check_process.rb:41:in `run_forest_tool'
        from /volumes/sync_check/sync_check.rb:54:in `<main>'

I believe this is incorrect:
https://github.com/ChainSafe/forest-iac/blob/1c38f2d4f31d9a6dc68b79a88c816628eace6015/terraform/modules/sync_check/service/sync_check.rb#L54
The way I see it:

There should be a rescue block to send Slack messages with logs even in case of an exception.
The SyncCheck.new should get the client injected as dependency, not re-created.
Not sure how docker is not present, but this should be investigated and resolved as well.

Other information and links

Needless to say, let's also fix the underlying issue so that it doesn't happen or is adequately handled.


jwt_token_cache crashes if server contains "/"

If the host contains /, for exampe
trompa-mtg.upf.edu/ce
then
https://github.com/trompamusic/trompa-ce-client/blob/7162a2fb622d2fd364b2aa2a99fc3077e1d52ec7/trompace/config.py#L91
it tries to store the cache in a file in a folder that does not exist

Thanks for reporting


Add the ability to specify tags for new written data

Closes #1460
Currently, there is no way to specify tags for a new written asset. This issue deals with adding a way to do so.
As suggested by @shlomitk1 fybrikapplication can be extended to include resourceMetadata in the flow requirements that the user will fill, for example:
 - dataSetID: 'new-data-parquet'
      flow: write
      requirements:
        flowParams:
          isNewDataSet: true
          catalog: fybrik-notebook-sample
          metadata:
            tags:
              finance: true
            columns:
            - name: nameOrig
              tags:
                PII: true
            - name: oldbalanceOrg
              tags:
                sensitive: true

Signed-off-by: Revital Sur<EMAIL_ADDRESS>Co-authored-by: mohammad-nassar10<EMAIL_ADDRESS>

I have a problem with using metadata that is not in catalog yet for governance decisions.
Suppose I want to update someone else's asset which I am not allowed to do since I am not the owner. But I could provide my name as an owner in a new metadata, and suddenly it's ok, I have overwritten the asset marking myself as an owner.

I have a problem with using metadata that is not in catalog yet for governance decisions. Suppose I want to update someone else's asset which I am not allowed to do since I am not the owner. But I could provide my name as an owner in a new metadata, and suddenly it's ok, I have overwritten the asset marking myself as an owner.

Can we check that an asset doesn't exist before creating a new catalog entry for it?

@roytman the resource metadata from the application flow requirements can be used only when 'isNewDataSetis true so we will not update an existing asset. In the current implementation ifisNewDataSet` is true then a new asset is registered in the catalog - it is not updating an existing asset

@shlomitk1 please take a look at the updated version in particular the code related to the flow which calls policy compiler in GetPoliciesDecisions. please note that the policy compiler is been called also when isNewDataSet is true as we need to check for permission to write new data. it is still under testing but I appreciate your review. Thanks

Looking now at the information sent to the policy manager, it seems that no changes have been made since updating the API. The purpose of Resource field in the request is to avoid an extra call to catalog connector, and separate the connectors completely. Why does opa-connector need to invoke GetAssetInfo if it receives the information about the resource in the request?
Besides, it is not the responsibility of policy manager to decide how to merge two inputs of metadata.
Manager should provide to the policy manager connector the complete information about the asset in Resource field.

@shlomitk1 the call to GetAssetInfo from opa was before the resource metadata change. This pr contains the code to avoid it if resource metadata exists.

@shlomitk1 the call to GetAssetInfo from opa was before the resource metadata change. This pr contains the code to avoid it if resource metadata exists.

And if resource metadata does not exist? How would the call change anything? I'm saying that it is unnecessary, the calls to the policy manager should be updated accordingly.


Homepage Changes

Magic Airlines

https://user-images.githubusercontent.com/13886925/102948454-faabc180-44bd-11eb-9bc5-a87fa5363db0.mp4

Needs some changes to spacing in later iterations


fix: correctly handle changed files to generate a commit message #338

Change the method we use to retrieve files that were changed and included in the commit. The old method did not work as mentioned in issue #338. I followed the advice provided here.

Remove the "--cached" option when retrieving the git diff because the Idea does not perform a git add when a file is selected. As a result, with the cached option, we encounter an empty diff.

For unversioned files, add the file content to the prompt as well since there is no way to force git diff to show these files.

Unfortunately, when removing the --cached flag, the git diff will not be in sync anymore. For example, I do a lot of staging before the actual commit, and now when I try to get the diff, the cached/staged files aren't included in it.
I think we should probably fetch both diffs, with --cached and without, and include both of them in the final prompt.


add: unofficial rootless Helm 3 Chart

This PR aims to add the Unofficial rootless Helm 3 Chart, which I maintain, to the unofficial install section.
Hope the bit of extra text is okay, but mentioning the rootless/non-root bit is crucial (context: https://github.com/louislam/uptime-kuma/pull/1769).

Thanks! Just checked this repo.

No problem, maintaining and keeping track of all the changes is no easy task.
Thank you for the merger :+1:


#124 Populate dosdp tsv with xrefs from taxonomy

Doi information added. Couldn't find the pattern yaml we updated yesterday, so gave alias_citations name to the new column.
Run the pipeline and observed related annotations generated correctly.
AnnotationAssertion(Annotation(<http://www.geneontology.org/formats/oboInOwl#hasDbXref> "DOI:10.1101/2<IP_ADDRESS>0558"^^xsd:string) Annotation(<http://www.geneontology.org/formats/oboInOwl#hasDbXref> "DOI:10.1101/2<IP_ADDRESS>972"^^xsd:string) <http://purl.obolibrary.org/obo/IAO_0000115> <http://www.semanticweb.org/brain_data_standards/AllenDendClass_CS202002013_1> "A GABAergic interneuron of the Mus musculus primary motor cortex. These cells can be distinguished from other cells in the primary motor cortex by their expression of Pip5k1b, Pax6. These cells also express Pax6."^^xsd:string)

Looks good to me. Why no tests?

Github actions were triggered when a pull request made to 'main' branch. But this PR is made to a feature branch. Updated actions config to trigger for all PRs.

Since multi_clause is not officially released, build was failing. Temporarily disabled dosdp pattern building.