BBC Radio stops intermittently

What version of Music Assistant has the issue?
2.2.0b2
What version of the Home Assistant Integration have you got installed?
2024.6.2
Have you tried everything in the Troubleshooting FAQ and reviewed the Open and Closed Issues and Discussions to resolve this yourself?

[X] Yes

The problem
Listening to BBC Radio 6 from Radiobrowser. It starts up and works just fine for a while (anywhere from 5min to >1hr, but commonly less than 2hr), then it stops without warning. Radio X seems fine.
How to reproduce
Listen to the same radio station - see log for url
Music Providers
Problem occurs with radiobrowser
Player Providers
I'm running a bluetooth speaker through squeezelite
Full log output
Default log here (debug and verbose lines removed, as requested)
music-assistant.log
The same log file with debug and verbose lines included, which may give more insight into the problem.
music-assistant.log
This is the most relevant bit, full log attached:
2024-08-02 12:53:22.124 VERBOSE (MainThread) [music_assistant.streams.media_stream] [http @ 0x559cab908800] Opening 'http://as-hls-ww-live.akamaized.net/pool_904/live/ww/bbc_6music/bbc_6music.isml/bbc_6music-audio%3D320000.norewind.m3u8' for reading
2024-08-02 12:53:22.148 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('#EXT-X-VERSION:3')
2024-08-02 12:53:22.148 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('## Created with Unified Streaming Platform(version=1.8.4)')
2024-08-02 12:53:22.149 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('#EXT-X-INDEPENDENT-SEGMENTS')
2024-08-02 12:53:22.149 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('#USP-X-TIMESTAMP-MAP:MPEGTS=3148807584,LOCAL=2024-08-02T12:53:13.600000Z')
2024-08-02 12:53:22.149 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('#EXT-X-PROGRAM-DATE-TIME:2024-08-02T12:53:13.600000Z')
2024-08-02 12:53:28.559 VERBOSE (MainThread) [music_assistant.streams.media_stream] [http @ 0x559cab908800] Opening 'http://as-hls-ww-live.akamaized.net/pool_904/live/ww/bbc_6music/bbc_6music.isml/bbc_6music-audio%3D320000.norewind.m3u8' for reading
2024-08-02 12:53:28.580 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('#EXT-X-VERSION:3')
2024-08-02 12:53:28.581 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('## Created with Unified Streaming Platform(version=1.8.4)')
2024-08-02 12:53:28.581 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('#EXT-X-INDEPENDENT-SEGMENTS')
2024-08-02 12:53:28.581 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('#USP-X-TIMESTAMP-MAP:MPEGTS=3149959584,LOCAL=2024-08-02T12:53:26.400000Z')
2024-08-02 12:53:28.581 VERBOSE (MainThread) [music_assistant.streams.media_stream] [hls @ 0x559cab900880] Skip ('#EXT-X-PROGRAM-DATE-TIME:2024-08-02T12:53:26.400000Z')
2024-08-02 12:53:28.581 VERBOSE (MainThread) [music_assistant.streams.media_stream] [http @ 0x559cab92e300] Opening 'http://as-hls-ww-live.akamaized.net/pool_904/live/ww/bbc_6music/bbc_6music.isml/bbc_6music-audio=320000-269156755.ts' for reading
2024-08-02 12:53:28.605 VERBOSE (MainThread) [music_assistant.streams.media_stream] [http @ 0x559cab9392c0] Opening 'http://as-hls-ww-live.akamaized.net/pool_904/live/ww/bbc_6music/bbc_6music.isml/bbc_6music-audio=320000-269156756.ts' for reading
2024-08-02 12:53:28.655 VERBOSE (MainThread) [music_assistant.streams.media_stream] size=   71662kB time=00:06:55.99 bitrate=1411.2kbits/s speed=1.02x
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] video:0kB audio:71662kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] [Parsed_loudnorm_0 @ 0x559caba08bc0]
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] {
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] "input_i" : "-14.60",
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] "input_tp" : "-4.54",
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] "input_lra" : "3.70",
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] "input_thresh" : "-24.72",
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] "output_i" : "-17.15",
2024-08-02 12:53:28.656 VERBOSE (MainThread) [music_assistant.streams.media_stream] "output_tp" : "-7.09",
2024-08-02 12:53:28.657 VERBOSE (MainThread) [music_assistant.streams.media_stream] "output_lra" : "3.70",
2024-08-02 12:53:28.657 VERBOSE (MainThread) [music_assistant.streams.media_stream] "output_thresh" : "-27.27",
2024-08-02 12:53:28.657 VERBOSE (MainThread) [music_assistant.streams.media_stream] "normalization_type" : "linear",
2024-08-02 12:53:28.657 VERBOSE (MainThread) [music_assistant.streams.media_stream] "target_offset" : "0.15"
2024-08-02 12:53:28.657 VERBOSE (MainThread) [music_assistant.streams.media_stream] }
2024-08-02 12:53:28.663 DEBUG (MainThread) [music_assistant.streams.media_stream] Process ffmpeg with PID 262 stopped with returncode 0
2024-08-02 12:53:28.663 DEBUG (MainThread) [music_assistant.streams.media_stream] stream finished (with code 0) for radiobrowser://radio/5cb4a4cb-c811-43e2-8531-36e13bf48828 - seconds streamed: 416.0
2024-08-02 12:53:28.663 DEBUG (MainThread) [music_assistant.streams.media_stream] Loudness measurement for radiobrowser://radio/5cb4a4cb-c811-43e2-8531-36e13bf48828: LoudnessMeasurement(integrated=-14.6, true_peak=-4.54, lra=3.7, threshold=-24.72, target_offset=0.15)
2024-08-02 12:53:28.664 VERBOSE (MainThread) [music_assistant.audio.ffmpeg] [flac @ 0x55a3c4e82680] unable to rewrite FLAC header.
2024-08-02 12:53:28.664 VERBOSE (MainThread) [music_assistant.audio.ffmpeg] size=   45445kB time=00:06:56.00 bitrate= 894.9kbits/s speed=1.02x
2024-08-02 12:53:28.664 VERBOSE (MainThread) [music_assistant.audio.ffmpeg] video:0kB audio:45437kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.017809%
2024-08-02 12:53:28.668 DEBUG (MainThread) [music_assistant.audio.ffmpeg] Process ffmpeg with PID 261 stopped with returncode 0
2024-08-02 12:53:50.045 INFO (MainThread) [music_assistant.providers.spotify] Attempt 1/5 failed: Spotify Rate Limiter
2024-08-02 12:53:50.045 INFO (MainThread) [music_assistant.providers.spotify] Retrying in 9 seconds...
2024-08-02 12:54:01.347 INFO (MainThread) [music_assistant.providers.spotify] Attempt 1/5 failed: Spotify Rate Limiter
2024-08-02 12:54:01.347 INFO (MainThread) [music_assistant.providers.spotify] Retrying in 28 seconds...
Later in the log file I play RadioX for a while, stop it manually and go back to radio6. I then left the system unattended and did other stuff for a while, and radio6 stopped by itself. Strangely that incident has different (less) log output than above.
I'm using a beelink eq13 running docker on debian to host Home Assistant, Music Assistant, Squeezelite, and other related containers like portainer and nginx.
Additional information
I'm starting the stream from the main MA UI.
Local playback and spotify are working just fine with the setup.
What version of Home Assistant Core are your running
2024.7.4
What type of installation are you running?
Home Assistant Container
On what type of hardware are you running?
Linux

The instructions ask that you not NOT add debug level logs. Please edit your post and include only the default logs.

Sorry - I had missed that instruction and from my own investigation to get this far I thought the additional content in the debug logs would be useful for understanding the cause. I've attached an edited log with the DEBUG and VERBOSE lines removed.

So is it only BBC stations that have a problem? I’m actually surprised they work at all, most people can’t find a working steam.

Yes - That is my experience so far. If I can't get Radio 2 working reliably then my plan to rip out my google hardware and replace entirely with HASS kit will fail. I love the way MA+HA can integrate virtually any other source of music and play it out to multiple different speakers and then play announcements over the top.
To be honest an ability to automatically try restarting the stream a few times would be enough - a momentary glitch would be tolerable, needing to manually restart is where the user pain is.

I thought it was odd how the verbose logging showed repeated entries for:
Opening 'http://as-hls-ww-live.akamaized.net/pool_904/live/ww/bbc_6music/bbc_6music.isml/bbc_6music-audio%3d320000.norewind.m3u8' for reading
But then somehow the stream url changes to
Opening 'http://as-hls-ww-live.akamaized.net/pool_904/live/ww/bbc_6music/bbc_6music.isml/bbc_6music-audio=320000-269156755.ts' for reading

I’m no expert but maybe it’s getting redirected. Google suggests that working BBC streams are somewhat elusive. I know Marcel has patched some issues in the past. https://github.com/music-assistant/hass-music-assistant/issues?q=is%3Aissue+bbc+is%3Aclosed
If it is only BBC streams then please edit the title to reflect that. This will get looked at when resources allow.

Title updated. It seems there is an open source BBC Sounds client for LMS here. https://github.com/expectingtofly/LMS_BBC_Sounds_Plugin
I've been looking through that code, and it might not be too tricky to implement something similar in MA.
It requires the user to have a BBC account and holds the username and password in local config.
To sign in it scrapes the BBC login page to determine the signin url, then performs a second http request to sign in with the user's credentials to get a cookie.
After that it seems like a reasonable json api to get the list of stations and podcasts...

That would be a feature request that you can raise separately or better still raise a PR for.

It's on my list (UK based) to have a look at a BBC Sounds provider at some point, but a bit further down my priorities!

Yeah, the sequence should be fairly straightforward I think. I just need to have enough time to go work on implementing something. Happy to have others involved and happy to help someone learn the python necessary - it's not actually that complicated dealing with all the streaming etc - mostly one of the other providers can be cribbed.
Hit me up on the Discord channel if you like and we can discuss further. I suggest closing this bug in favour of a feature request to track implementation.
And we may even be able to pull up a couple of laptops over a beer sometime to do the implementation - I think from your website you live not a million miles from me (also just outside Cambridge).

Mel - that sounds excellent. Coding and beer go always go together really well!
I guess to make this work with MA we'd need to create a BBC Sounds Player Provider, that would take the user's BBC credentials as settings, then expose menus for the BBC radio stations and podcasts much like the LMS stuff. I think I'd need to find the time to spend a few days chugging through codeacademy so that I could at least read the existing MA code, then I might just take you up on the offer of a meetup.


Digilent Arty xc7a100t failure

Hello @tcal-x
I am using the Digilent Arty board containing the 100T device with the CFU Playground on the default proj_template example.
With the following command line, I see the following failure in the symbiflow_write_fasm stage.
Can you please provide pointers to get past this and {program, load subsequently}
Command line:
%make prog TARGET=digilent_arty USE_SYMBIFLOW=1 EXTRA_LITEX_ARGS="--variant=a7-100"
Terminal messages:
python3 -m litex.soc.software.mkmscimg bios.bin --little
python3 -m litex.soc.software.memusage bios.elf /home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/build/digilent_arty.proj_template/software/bios/../include/generated/regions.ld riscv64-unknown-elf
ROM usage: 24.86KiB 	(19.42%)
RAM usage: 1.60KiB 	(20.02%)
rm crt0.o
make[4]: Leaving directory '/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/build/digilent_arty.proj_template/software/bios'
INFO:SoC:Initializing ROM rom with contents (Size: 0x6380).
INFO:SoC:Auto-Resizing ROM rom from 0x20000 to 0x6380.
make[4]: Entering directory '/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/build/digilent_arty.proj_template/gateware'
symbiflow_synth -t digilent_arty -v /home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/proj/proj_template/cfu.v /home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/third_party/python/pythondata_cpu_vexriscv/pythondata_cpu_vexriscv/verilog/VexRiscv_FullCfu.v /home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/build/digilent_arty.proj_template/gateware/digilent_arty.v -d artix7 -p xc7a100tcsg324-1 -x digilent_arty.xdc > /dev/null
symbiflow_pack -e digilent_arty.eblif -d xc7a100t_test -s digilent_arty.sdc > /dev/null
symbiflow_place -e digilent_arty.eblif -d xc7a100t_test -n digilent_arty.net -P xc7a100tcsg324-1 -s digilent_arty.sdc > /dev/null
Warning: IDELAY_GROUPS parameters are currently being ignored!
symbiflow_route -e digilent_arty.eblif -d xc7a100t_test -s digilent_arty.sdc > /dev/null
symbiflow_write_fasm -e digilent_arty.eblif -d xc7a100t_test > /dev/null
/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/env/symbiflow/bin/vpr_common: line 116: 115099 Killed                  genfasm ${ARCH_DEF} ${EBLIF} --device ${DEVICE_NAME} ${VPR_OPTIONS} --read_rr_graph ${RR_GRAPH} $@
make[4]: *** [Makefile:46: digilent_arty.fasm] Error 137
make[4]: Leaving directory '/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/build/digilent_arty.proj_template/gateware'
Traceback (most recent call last):
File "./common_soc.py", line 57, in
main()
File "./common_soc.py", line 53, in main
workflow.run()
File "/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/board_specific_workflows/general.py", line 125, in run
soc_builder = self.build_soc(soc)
File "/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/board_specific_workflows/digilent_arty.py", line 73, in build_soc
return super().build_soc(soc, **kwargs)
File "/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/board_specific_workflows/general.py", line 102, in build_soc
soc_builder.build(run=self.args.build, **kwargs)
File "/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/third_party/python/litex/litex/soc/integration/builder.py", line 350, in build
vns = self.soc.build(build_dir=self.gateware_dir, **kwargs)
File "/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/third_party/python/litex/litex/soc/integration/soc.py", line 1147, in build
return self.platform.build(self, *args, **kwargs)
File "/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/third_party/python/litex/litex/build/xilinx/platform.py", line 73, in build
return self.toolchain.build(self, *args, **kwargs)
File "/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/third_party/python/litex/litex/build/xilinx/symbiflow.py", line 241, in build
_run_make()
File "/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/third_party/python/litex/litex/build/xilinx/symbiflow.py", line 84, in _run_make
raise OSError("Error occured during Symbiflow's script execution.")
OSError: Error occured during Symbiflow's script execution.
make[3]: *** [/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc/common_soc.mk:115: build/digilent_arty.proj_template/gateware/digilent_arty.bit] Error 1
make[3]: Leaving directory '/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/soc'
make[2]: *** [../proj.mk:310: prog] Error 2
make[2]: Leaving directory '/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/proj/proj_template'
Thanks
Raj

/home/rajsaktish/repos/enhance_CFU/as_is/CFU-Playground/env/symbiflow/bin/vpr_common: line 116: 115099 Killed genfasm

This means the genfasm process was killed by SIGKILL. Often it's because the OOM killer has triggered because the system has run out of memory. There's usually a corresponding kernel message (try journalctl -b -k) in that case.
Are you running this inside a VM? How much memory does it have available?
I'm not sure how much memory the genfasm step is expected to consume, but it might be relatively large.

Thanks @danc86 .
My desktop has 8GB memory. I am running the commands natively on linux, without virtual machines.
Tail lines of journalctl -k -b below:
$ grep -i memtotal /proc/meminfo
MemTotal:        8045528 kB
$ tail jctl_log.txt
Jun 23 18:27:29 desktop kernel: [ 115649]  1000 115649      656        0    40960       24             0 sh
Jun 23 18:27:29 desktop kernel: [ 115650]  1000 115650     2411        4    53248      133             0 pyrun
Jun 23 18:27:29 desktop kernel: [ 115656]  1000 115656    19392      130   204800    14983             0 python3
Jun 23 18:27:29 desktop kernel: [ 119744]  1000 119744     2148       46    53248       56             0 make
Jun 23 18:27:29 desktop kernel: [ 119851]  1000 119851      657        0    45056       24             0 sh
Jun 23 18:27:29 desktop kernel: [ 119852]  1000 119852     2411        4    57344      106             0 symbiflow_write
Jun 23 18:27:29 desktop kernel: [ 119862]  1000 119862  2259549  1678924 13971456    28008             0 genfasm
Jun 23 18:27:29 desktop kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service,task=genfasm,pid=119862,uid=1000
Jun 23 18:27:29 desktop kernel: Out of memory: Killed process 119862 (genfasm) total-vm:9038196kB, anon-rss:6715692kB, file-rss:4kB, shmem-rss:0kB, UID:1000 pgtables:13644kB oom_score_adj:0
Jun 23 18:27:29 desktop kernel: oom_reaper: reaped process 119862 (genfasm), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Any suggestions to make it work ?
Thanks
Raj

The output you pasted shows genfasm is consuming ~6.7GB of memory when the OOM killer is triggered.
I also found this old issue about the same problem (genfasm consuming 7GB when targetting Arty 100T):
https://github.com/chipsalliance/f4pga-examples/issues/37
So unfortunately it seems to be a known issue and probably the only workaround is to run on a system with more memory available (or swap I suppose).
FYI I tried it on my workstation (proj_template for Arty 100T) and genfasm goes up to 7618316KiB max resident size according to /usr/bin/time. So you may be able to build it with only slightly more memory or swap (a few more GB).

Trying it on a machine with larger memory seemed to work. Thanks @danc86


Installation fails with docker and debian jessie

Hi. I tried to install the sentry ldap plugin on a sentry docker installation (by adding sentry-ldap-auth to requirements.txt). Docker version is 1.12.1.
When I try to build everything, it fails on the python-ldap pip installation, with this error code:
    Modules/errors.h:8:18: fatal error: lber.h: No such file or directory
     #include "lber.h"
                      ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

This is weird, because the package libldap2-dev is installed, the file lber.h is installed on my system. Also, if I try to install sentry-ldap-auth in a virtualenv in my host system, everything works well.
What do you think?

Here's what our docker file has:
FROM sentry:8.5

# Install dependencies for LDAP authentication
RUN apt-get update && apt-get install -y libsasl2-dev python-dev libldap2-dev libssl-dev

@barronhagerman Should we add this to the README?

It works. Thanks!
That would be great to put this in the README.md.

Actually, @barronhagerman do you think we can switch ldap packages? See the instructions on https://bitbucket.org/psagers/django-auth-ldap/pull-requests/20/switched-to-python3-ldap/diff
I have no idea if it will actually work, but it would be worth testing.

Ok. Finally I think it does not work.
I am sorry, actually I am a bit new to docker.
I understand that sentry:onbuild and sentry are two different docker images. sentry:onbuild installs pip dependencies but not the apt ones. And sentry installs the apt dependencies, but not the pip ones.
How can I solve this?

I have not used ONBUILD, but I think that is what you would use if you were creating an image that would be the base for other images. Try removing ONBUILD from all of those lines, rebuild your image, and try again.
Also, before you do pip install, you'll probably need to ADD requirements.txt

Also, @azmeuk, you'll need to create a custom configuration file. Something like this:
from sentry_docker_conf import *

AUTH_LDAP_SERVER_URI = 'ldap://my.ldapserver.com'

AUTH_LDAP_USER_SEARCH = LDAPSearch(
    'dc=domain,dc=com',
    ldap.SCOPE_SUBTREE,
    '(mail=%(user)s)',
)

...

Then, set ENV SENTRY_CONF_FILE custom_settings.py in your Dockerfile

@barronhagerman Thanks, I will try that.
Do you a have a working Dockerfile sample?

FROM sentry:8.8

# Install dependencies for LDAP authentication
RUN apt-get update && apt-get install -y libsasl2-dev python-dev libldap2-dev libssl-dev

# Install dependencies for symsynd (for dSYM support)
RUN apt-get install -y clang g++

ADD requirements.txt /conf/
RUN pip install -r /conf/requirements.txt

ADD dsym_s3_import.sh /conf/

ENV SENTRY_CONF /conf
ADD config.yml $SENTRY_CONF/
ADD sentry.conf.py $SENTRY_CONF/
ENV PYTHONPATH /conf

I'm closing this issue because I don't think it is a problem with the sentry-ldap-auth package. If you continue to have troubles, I'll try to help.


Extension host messages flood and stall renderer

VSCode Version: 1.28.2
OS Version: Windows 10 Pro 64 bit

Steps to Reproduce:

Whenever I edit any file on VS code it stops working. Before last update it was working perfect with the extensions I have installed on my setup. Below are the extensions list that I am using currently.

Extensions Installed:

VS Live Share Extension Pack (https://marketplace.visualstudio.com/items?itemName=MS-vsliveshare.vsliveshare-pack)
Vetur (https://marketplace.visualstudio.com/items?itemName=octref.vetur)
TSLint (https://marketplace.visualstudio.com/items?itemName=eg2.tslint)
PHP Extension Pack (https://marketplace.visualstudio.com/items?itemName=felixfbecker.php-pack)
jshint (https://marketplace.visualstudio.com/items?itemName=dbaeumer.jshint)
Git extension pack (https://marketplace.visualstudio.com/items?itemName=donjayamanne.git-extension-pack)
Azure Repos (https://marketplace.visualstudio.com/items?itemName=ms-vsts.team)
Eslint (https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint)
GitHub Pull Requests (https://marketplace.visualstudio.com/items?itemName=GitHub.vscode-pull-request-github)

Does this issue occur when all extensions are disabled?: Yes/No
Yes

@iyashpal We are really keen on nailing down this issue but we have been able to reproduce yet. Can you please create CPU profile when things get slow. With that we can get a fix.
To create profile, do this: Hit F1 > Toggle Developer Tools. In the overflow menu select 'More Tools > JavaScript Profiler'. In there select start, let it profile for 30 to 60 seconds, stop it, save it to a file and attach the file here. Thanks.

@jrieken "able" should read "unable", I guess?

I have same problem.
Here CPU profile.

@rtemred Thanks. This is really help. From the profile it seems that VS Code is flooded with IPC messages and has trouble to keep up.

This might be related to https://github.com/Microsoft/vscode/issues/62421 and the question is where all these messages are being send from. Potential senders are file watching, search, and extensions. I don't think it search, which leaves extension and file watching.
@rtemred What extension do you have installed (code --list-extension)? Also, do you have some kind of builder running, e.g. webpack-dev-server or something else that would generate loads of file events?

@jrieken here:

alefragnani.Bookmarks
alefragnani.project-manager
alyaptsev.vscode-bemy
annsk.alignment
bradgashler.htmltagwrap
christian-kohler.path-intellisense
cpylua.language-postcss
donjayamanne.githistory
dracula-theme.theme-dracula
JerryHong.autofilename
lukasz-wronski.ftp-sync
mechatroner.rainbow-csv
mhmadhamster.postcss-language
michelemelluso.code-beautifier
midnightsyntax.vscode-wrap-console-log
mrmlnc.vscode-apache
ms-mssql.mssql
ms-vscode.Theme-MaterialKit
octref.vetur
PKief.material-icon-theme
robinbentley.sass-indented
RolandGreim.sharecode
Shan.code-settings-sync
sleistner.vscode-fileutils
sysoev.language-bemhtml
Tyriar.sort-lines
vincaslt.highlight-matching-tag
volkov97.vscode-bemmet
wayou.vscode-todo-highlight
wmaurer.change-case
zhuangtongfa.Material-theme

@rtemred Try disabling vincaslt.highlight-matching-tag and see if that helps. If it doesn't, try disabling all extensions (start with --disable-extensions or Reload Window With Extensions Disabled).

We have seen the same profile from @vitorsemeano in #62588 and these are extension both have installed:

annsk.alignment
donjayamanne.githistory
dracula-theme.theme-dracula
mechatroner.rainbow-csv
michelemelluso.code-beautifier
mrmlnc.vscode-apache
vincaslt.highlight-matching-tag
wayou.vscode-todo-highlight

@rtemred, @vitorsemeano what kind of files are you guys editing when this happens? JSX, HTML, CSV? This is while typing and/or moving the cursor, right?

This happens when i try to select something or write to the file, mostly selecting. I have seen this occurring in php, less, js, etc. When I select some text in a non stop matter on some random file it actually blocks the entire vscode for some seconds.

@jrieken yes it's while typing.  File types: js, less - for sure. Maybe postcss - not sure.
I disabled vincaslt.highlight-matching-tag.
I have disabled extension vincaslt.highlight-matching-tag. I will observe how VSC will behave. I will inform you tomorrow. This issue happens after a few hours of work. Then I reboot app. It's happens a couple of times a day.

Thank guys. Please keep us posted, we are currently looking at and testing those extensions.

I'm also having slimier issue and I had to restart vsc to have it fixed
I usually have 2-3 vsc running at same time and when one of them starts freezing, the rest of vsc still run ok.
here's the list of my installed extensions

Beautify 1.4.7
Chinese Language packet
Debugger for Java 0.15.0
ESLint 1.7.0
Java Extension Pack 0.4.0
Java Test Runner 0.10.0
Lnaguage Support for Java
Laravel Blade Snippets 1.18.0
Markdown All in One 1.7.0
Maven for Java 0.11.1
minify 0.3.0
Vetur 0.13.0

and the CPU profile:
CPU-20181108T003312.zip

@yyss8 What version of VS Code are using here?

@yyss8 What version of VS Code are using here?

v1.28.2 for x64, the latest version I think?

Thanks. What appears to be happening is that for a very long time (18secs) events are fired, from this one trace it looks like it's context keys...

This is quite interesting but might unrelated to the behaviour others have been mentioning here.

@iyashpal We are really keen on nailing down this issue but we have been unable to reproduce yet. Can you please create CPU profile when things get slow. With that we can get a fix.
To create profile, do this: Hit F1 > Toggle Developer Tools. In the overflow menu select 'More Tools > JavaScript Profiler'. In there select start, let it profile for 30 to 60 seconds, stop it, save it to a file and attach the file here. Thanks.

Hi @jrieken sorry for late reply here is the CPU profile.
CPU-20181109T161911.zip
While recording the CPU Profile VS Code handed two times and the file I was editing it .vue.
Thanks!
Yash

Today has passed without vincaslt.highlight-matching-tag and this issue has not be.

@artdude543 thanks for the profile. what you are seeing is https://github.com/Microsoft/vscode/issues/62829#issuecomment-437314957

Today has passed without vincaslt.highlight-matching-tag and this issue has not be.

I updated this extension (0.8.3), now VSC work fine.

That's good to hear. Yesterday we have also fixed a pretty bad listener leak which can show this behaviour (see https://github.com/Microsoft/vscode/issues/62970#issuecomment-438683094). Latest insider and the upcoming 1.29.1 release will contain a fix for this.

closing this has half duplicate and half caused-by-extension


connect to databases with dashes in names

You code not work with postgres. I fixed in MySQLType
https://github.com/Bajdzis/vscode-database/blob/master/extension/engine/mysql.js#L102
Next time write in commit message number issue like "fixed #17"
if commit message have issue number you pull and commit is show in issue.
Thanks for pull :)

Thanks. I was going to close the pull request anyway as you fixed it with your own commit.


OsvFunTest: Update expected test results

Add more references and reorder results as expected.
Signed-off-by: Sebastian Schuberth<EMAIL_ADDRESS>

Can it be that this test requires updates very frequently?

At least recently, yes. Maybe it's not a good idea to use NPM packages for vulnerability tests, as these have new findings all the way long 😉

Yes, different package could help. Any recommendations for the ecosystem / package to use?

Or refactoring the test so that it checks that the expected vulnerabilities are contained in the result instead of an equality check.  I'll take care of making it less flaky soon. IIRC I've chosen older versions of the package with the hope that vulnerabilities don't change too often.

Any recommendations for the ecosystem / package to use?

Anything Rust / Cargo maybe?


FirebaseAdMobPlugin iPhone ok iPad NG

FirebaseAdMobPlugin will display banner without problem on iPhone, but on iPad it will display "FirebaseAdMobPlugin  adView: didFailToReceiveAdWithError: Request Error: No ad to show. (MobileAd <FLTBannerAd: 0x1c000e070> PENDING mobileAdId: 915310470 for: <GADBannerView : 0x13bd31cc0; frame = (0 0; 320 50); clipsToBounds = YES; layer = <CALayer: 0x1c4228ea0 >>) "becomes warning and the banner is not displayed.

@Ken1101
The issue at https://github.com/flutter/flutter/issues/21204 has been closed and moved here. Future collaboration on this issue will be done here.

Closing based on that the original issue is open and seems obsolete


Outsourcing validators to separate class-lib is not possible

I have the following project setup:

My requests, responses and validators are living outside of the API project that references FastEndpoints in a project called Contract. This is done to allow for easy integration of requests, responses and validators into the API and Blazor frontends.
Due to changes in #141 AbstractValidator<T> instances are no longer registered by FastEndpoints, so  this doesn't work any longer.
Installing FastEndpoints into the Contract class lib is not an option as this will break Blazor WASM with the following error:
Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(427, 5): [NETSDK1082] There was no runtime pack for Microsoft.AspNetCore.App available for the specified RuntimeIdentifier 'browser-wasm'.

I know that @wjax will provide a proper solution for this, but for learning purposes I dabbled a little bit with the source code and got a solution working. My current solution can be found in this branch.
For a quick glance here are my changes (I've added some comments to see my changes better):
EndpointDiscoveryOptions.cs
    /// <summary>
    /// an optional list to allow the definition of what types should be handled as a validator.
    /// if no types are defined here only validators that inherit from <see cref="Validator{TRequest}"/> are registered.
    /// </summary>
    public Type[]? ValidationTypes { get; set; }

EndpointData.cs
private static EndpointDefinition[] BuildEndpointDefinitions(IServiceCollection services, EndpointDiscoveryOptions? options,
        ConfigurationManager? config)
    {
        if (options?.DisableAutoDiscovery is true && options.Assemblies?.Any() is false)
            throw new InvalidOperationException("If 'DisableAutoDiscovery' is true, a collection of `Assemblies` must be provided!");

        Stopwatch.Start();

        //also update FastEndpoints.Generator if updating these
        IEnumerable<string> excludes = new[]
        {
            "Microsoft",
            "System",
            "FastEndpoints",
            "testhost",
            "netstandard",
            "Newtonsoft",
            "mscorlib",
            "NuGet",
            "NSwag",
            "FluentValidation",
            "YamlDotNet",
            "Accessibility",
            "NJsonSchema",
            "Namotion"
        };

        // added this here to get the configured validation types if any
        var validationTypes = Enumerable.Empty<Type>();
        if (options?.ValidationTypes?.Any() is true)
        {
            validationTypes = options.ValidationTypes;
        }

        var discoveredTypes = options?.SourceGeneratorDiscoveredTypes ?? Enumerable.Empty<Type>();

        if (!discoveredTypes.Any())
        {
            var assemblies = Enumerable.Empty<Assembly>();

            if (options?.Assemblies?.Any() is true)
            {
                assemblies = options.Assemblies;
                excludes = excludes.Except(options.Assemblies.Select(a => a.FullName?.Split('.')[0]!));
            }

            if (options?.DisableAutoDiscovery != true)
                assemblies = assemblies.Union(AppDomain.CurrentDomain.GetAssemblies());

            if (options?.AssemblyFilter is not null)
                assemblies = assemblies.Where(options.AssemblyFilter);

            // refactored this to be it's own list so I can add any configured validation types
            var typesToRegister = new List<Type>() {
                Types.IEndpoint,
                Types.IEndpointValidator,
                Types.IEventHandler,
                Types.ISummary
            };

            // add configured validation types
            typesToRegister.AddRange(validationTypes);

            discoveredTypes = assemblies
                .Where(a =>
                    !a.IsDynamic &&
                    !excludes.Any(n => a.FullName!.StartsWith(n)))
                .SelectMany(a => a.GetTypes())
                .Where(t =>
                    !t.IsAbstract &&
                    !t.IsInterface &&
                    !t.IsGenericType &&
                    t.GetInterfaces().Intersect(typesToRegister).Any());
        }

        //Endpoint<TRequest>
        //Validator<TRequest>

        var epList = new List<(Type tEndpoint, Type tRequest)>();

        //key: TRequest //val: TValidator
        var valDict = new Dictionary<Type, Type>();

        //key: TEndpoint //val: TSummary
        var summaryDict = new Dictionary<Type, Type>();

        foreach (var t in discoveredTypes)
        {
            foreach (var tInterface in t.GetInterfaces())
            {
                if (tInterface == Types.IEndpoint)
                {
                    var tRequest = t.GetGenericArgumentsOfType(Types.EndpointOf2)?[0] ?? Types.EmptyRequest;

                    services.AddTransient(t);
                    epList.Add((t, tRequest));
                    continue;
                }

                // changed this check here to see if the tInterface is part of any configured validation types
                if (validationTypes.Contains(tInterface))
                {
                    var tRequest = t.GetGenericArgumentsOfType(Types.ValidatorOf1)?[0]!;
                    if (!valDict.ContainsKey(tRequest))
                    {
                        valDict.Add(tRequest, t);
                    }
                    continue;
                }

                if (tInterface == Types.ISummary)
                {
                    var tEndpoint =
                        t.GetGenericArgumentsOfType(Types.SummaryOf1)?[0]! ??
                        t.GetGenericArgumentsOfType(Types.SummaryOf2)?[0]!;
                    summaryDict.Add(tEndpoint, t);
                    continue;
                }

                if (tInterface == Types.IEventHandler)
                {
                    ((IEventHandler?)Activator.CreateInstance(t))?.Subscribe();
                    continue;
                }
            }
        }

This will result in a setup like this in Program.cs:
builder.Services.AddFastEndpoints(o =>
{
    o.SourceGeneratorDiscoveredTypes = DiscoveredTypes.All;
    o.ValidationTypes = new[] { typeof(IValidator) };
});

this is what I had in mind:
builder.Services.AddFastEndpoints(o =>
{
    o.SourceGeneratorDiscoveredTypes = DiscoveredTypes.All;
    o.IncludeAbstractValidators = true;
});

default value would be false. if user wants abstract validators discovered, just need to set a boolean.
also need to make the source generator library consistent with this behavior.

I think there may be a cleaner solution.
The issue was a duplicated entry into a dictionary that holds the validators. If we change and filter to only add validators of relevant types (requests) we could safely scan everything without issue and extra config options...

Ah, just thought that with the solution that I went with a user could limit the scan to exactly those validators that need to be handled by FE. But honestly if there is a solution that doesn't require any setup I'm now than happy 😁

I have been thinking and I do like the idea from @MichaelHochriegl to include and specify what types the dev wants to be scanned / act as validators. That way devs have full control.
I would have preferred to specify the validator to use in the Configure method of every endpoint so it is absolutely clear what validator is used by the endpoint. However I know it breaks compatibility and @dj-nitehawk will not like it :)
If we go Michael's road, I would name the options field: CustomValidatorTypes so it is optional. If null (default), internal IEndpointValidator is used, otherwise the list is honored.
I would remove the duplication check and let an exception raise in case of duplicate validators for the same type so the dev is properly warned about the situation. In Michaels code, it would pass without being noticed.

I would have preferred to specify the validator to use in the Configure method of every endpoint

i absolutely don't mind if this can be made optional. i.e. if someone wants to be explicit about which validator they want with an endpoint they will have to first set a startup flag c => c.Endpoints.SkipValidatorDiscovery = true and then call a Validator<TValidator>() method inside Configure(). if they haven't disabled validator discovery, then calling Validator<TValidator>() should throw to inform user of the correct usage. what do you think?
btw, the startup config has been restructured.
and yeah we can do the CustomValidatorTypes config too. if not null, then SkipValidatorDiscovery should be false, or an error is thrown.
thoughts?

Thanks that my thought-process and first implementation attempt isn't wasted. I really like the approach to provide both configurations (at the endpoint Configure and global type registration).
I would try again to look at it, but again I'm totally aware that my code will not necessarily land in the lib.
I really like the global type registration because it will allow me to create my own interface (e.g. IFastEndpointValidator) in my project and these will get autowired as request validators.

Go for it Michael. Your code had nothing bad.
We can help if needed but I encourage you to send a PR as you had some progress already in place

I'm currently implementing the changes from your comments and I'm slightly struggling with the FastEndpoints.Generator changes that I need to make.
Currently the EndpointsDiscoveryGenerator still uses the FluentValidation.IValidator and registered this. I tried to retrieve the FastEndpoints.EndpointDiscoveryOptions from the GeneratorExecutionContext but that's not referenced by the Generator.
So how do I need to change the Generator project to allow the configuration of the registered validator types?

you can ignore the generator for now. I'll look in to it later. for now it can be a limitation of the generator package. i.e. it will discover all validators.

Basic functionality is there, but I struggle with where to throw the exception when the user has misconfigured something (e.g. tries to use Validator<TValidator> in Configure() when .Endpoints.SkipValidatorDiscovery = false or not set). Same goes for the exception when the CustomValidatorTypes are set but SkipValidatorDiscovery = true.
Any tips where to throw them? And the type of exception, I would have chosen InvalidOperationException but you might prefer something different.

can you start a draft PR?
I'll have to see what you have done to comment.

guys give v5.0.0-beta7 a whirl...
so... by default only validators inheriting from FastEndpoints.Validator<TRequest> are auto discovered and registered.
if someone wants to include FluentValidation.AbstractValidator<TRequest> validators as well, all they gotta do is:
builder.Services.AddFastEndpoints(o => o.IncludeAbstractValidators = true);

an exception will be thrown during startup if more than one validator is found for a single request dto. in which case the user needs to be explicit about which exact validator must be used in the endpoint configuration like so:
    public override void Configure()
    {
        Get("test");
        Validator<MyValidator>();
    }

so... what do you think?
have i missed anything?


[DESIGN] Integrate Map with Search

A few screenshots of a map overlay + filters from toyota.com (the map slides out from the right in this example). This isn't necessarily ideal, but will keep an eye out for other precedents.

the obvious...

https://cities-innovation-oecd.com/explore?filter=all&filterType=population

mobile version

Another Map example: https://direct-democracy-navigator.org/

same organization does a datavis page using filters @paninee check this out:  https://direct-democracy-navigator.org/legal-design-filter

Design Decisions with @jesicarson

Hide the Methods and Collections
Add a map view choice in the top right corner along with other view choices
white frame around the map
allow the user to go full-screen with a full-screen icon button on the top right of the map
the pins should show according to the search/filter. We should already have this code somewhere. Our current map used to work like that.


Unable to make iOS Archive with Bitcode

I'm using the latest version of Kissme and tried to create an archive of the app with the multiplatform library.
Step to replicate:

XCode > Product > Archive
Error: bitcode bundle could not be generated

@samiuelson I think that this could be solved by recompiling static library with embed bitcode enabled


I would like the .exe

Hello, i'm not able to compile go to .exe. Is it possible to generate an .exe file?
thanks

sorry, Now i saw the release.


add support for Go 1.17 (tip)

@mvdan Ready for Go 1.17 beta1! All we have to do is rebase the qtls repository onto the beta1 Go commit. Assuming that there won't be any significant changes to crypto/tls, that should be trivial.

As soon as 1.17 is released you'll be able to remove 1.15, too.

That's the plan. And as soon as the beta is out, we can also test this in CI.


python3.pkgs.crate: 0.29.0 -> 0.30.0

Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 219189 run on x86_64-linux 1

5 packages built:

csvkit
python310Packages.agate-sql
python310Packages.crate
python311Packages.agate-sql
python311Packages.crate

Thanks!


Document how to generate dependency graph.

This is just so we do not forget. In #1692 I changed the scripts to support generating dependency graphs. In general, one can generate these pretty easily:
$ git clone ...
$ cmake -H. -Bbuild-output
$ cmake --graphviz=graphviz/google-cloud-cpp.dot --build build-output

But the graphs are more accurate if the dependencies are pre-installed, instead of using ExternalProject_Add, for that use:
env NCPU=$(nproc) \
    CXX=g++ CC=gcc \
    TEST_INSTALL=yes \
    CREATE_GRAPHVIZ=yes \
    TRAVIS_OS_NAME=linux DISTRO=ubuntu DISTRO_VERSION=bionic \
    ./ci/travis/build-linux.sh

These should be moved to a nice markdown file in doc/, maybe under "how to build with strange settings."

Note, this is also easy to do with bazel:
$ bazel query 'deps(//...) intersect //google/...' --output graph > graph.dot
$ dot -Tpng < graph.dot > graph.png

I've been meaning to comment on this for a while just wanted to point out two things:
cmake --graphviz=graphviz/google-cloud-cpp.dot --build build-output

I don't think you can combine graphviz and build. You have to do something like cmake --H. -Bout --graphviz. I'm not sure if CMake changed something. Also, because this project has a lot of files I ran into the "too many open files" error.
This also can be done with Ninja:
ninja -t graph all | dot -Tpng -ograph.png

People don't seem to care too much, and the documentation in this issue is pretty good too.


⚠ ClamAV has degraded performance

In fe16abb, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 70 ms

Resolved: ClamAV performance has improved in d4a37c8 after 11 minutes.


Fix error in ergas calculation

What does this PR do?
Fixes https://github.com/Lightning-AI/torchmetrics/issues/2497
Fixes https://github.com/Lightning-AI/torchmetrics/issues/2416

Before submitting

[ ] Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
[ ] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?

PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?
Make sure you had fun coding 🙃

📚 Documentation preview 📚: https://torchmetrics--2498.org.readthedocs.build/en/2498/

Wondering how the tests are passing before and now with two different computation...

Because the reference implementation was also written by us (because no reference seems to exist) which had the same computational bug


Log failed request messages for client disconnects

Likely to be punted

Also improve exception message.

Does this have any to do with this issue I'm having?

@pakrym will #1388 handle this?

No, I'll do another pass to add logging into appropriate places. That PR is getting too large.


Standalone Section/Field resources

Summary
It would be great to be able to combine multiple sources of secrets under same item.
I.e. manually created ones + automated ones from different components/modules.
Since there is no access control granularity past vault-level - there is no security downside to this.
However from UX standpoint - this has a benefit of keeping everything in one place vs needing to spam an item per individual piece of automation code + anything manual you'd have to add on top.
Use cases
PostgreSQL users/urls, multiple levels of grafana tokens (cloud/stack), secrets for a single app that comes from multiple sources (db, cdn, deploy keys, buckets, metrics/logs/traces publisher keys).
If you use something like terragrunt or magic python wrapper - you get separately scoped component for each service. (to reduce graph size, apply different concurrency or run less in CI/CD.. etc)
Proposed solution
Have dedicated section/item resource. Possibly needing a "custom composite" item type that supports this.
Is there a workaround to accomplish this today?
Hint of a workaround is to use ignore_changes+protect_from_destroy for manual secrets, but still does not handle multiple sources. Also, other code having to ignore things it should not know about is a scope leak.

This would be so helpful!


Create recurring tasks after finishing the original one

When I create recurring tasks, the application creates it's next recurrence immediately after saving the first task. Then when I edit the original task (for example change it's priority, etc.), it creates new duplicate recurrences of the original task on each edit.
When I miss completion of the task on time, it also creates duplicates for each day of delay.
I would suggest to change the behavior to create a next recurrence only after confirmation of completion of the original task ("Mark as done"). This would prevent creation of duplicates.

@rzelnik, I assume you set the due date to today to provoke the automated creation of the recurring task, right? Because just creating a task with recurrence and no due date should not create any new task by automatically.
The big issue here is that sleek (being sleek) doesn't use a database, it rereads the todo.txt file all the time, which so far works out surprisingly well. The issue you're having with changing the first todo, which then doesn't change the already existent recurring todo, is directly connected to this. Due to the lack of a database, both todos are not connected to each other (by an ID for instance) so sleek is having difficulties to change the recurring todo, as it doesn't know that it exists. We could only solve this by either introducing a database or more logic which adds too much complexity to such a simple application.
To be perfectly honest I never used this feature myself, as I don't have recurring tasks and so I'm having a hard time to understand what users do expect from a recurring todo feature. In my opinion your suggestion of cutting the automated creation would remove a lot of unwanted (at on my side) complexity from the code.
But I am not sure if other users would agree with your suggestion.
@zerodat, @velardevines, @jason-mehmel, @wsdfhjxc, as far as I remeber you guys are actively working with recurrences, correct? Would you mind sharing your opinion on this?
The idea is to trigger the creation of a new todo (based on an old todo with recurrence) only once the old todo is being set to complete by hand. This would remove the automated creation of new todos. Currently sleek is automatically creating a new todo once the due date has been reached.

Yes, I confirm that the automated creation of the recurring task is related to tasks with due date. And yes, I understand that processing the data stored in todo.txt file is somehow difficult compared with working with the database. On the other side, todo.txt-compatibility is a big advantage of this application, despite of some difficulties with processing the data. I would prefer to keep the app simple and todo.txt-based rather than adding a new layer of complexity by synchronizing between the database and the todo.txt file. Maybe the best solution would be to let the user choose in the preferences between automated creation each day/week/etc. and creation after finishing the previous recurrence.

My goal though would be to find a solution that offers a good user experience that comes without a new setting. Especially for a function that needs a lot of explaining. Plus from the telemetry data some users are kindly providing I can see that less than 1% of sleeks users are working with recurrences. Which makes it even harder to justify a dedicated setting for this.
The more I think about it the more I want to deactivate the automated creation. But let's see what the others are saying about it, before we just remove it ;)

I admit to not understanding how Sleek implements the recurrence feature. Nor do I really need to understand how it works.  :-)
Maintaining compatibility of todo.txt format and keeping Sleek code uncomplicated and easy to maintain are important.
For the recurrence feature, what is important to me is that a new task is created when I "mark as done" a task with a recurrence value.
So, I agree with the @rzelnik comment in his first post: "I would suggest to change the behavior to create a next recurrence only after confirmation of completion of the original task ("Mark as done"). This would prevent creation of duplicates."
Hope this helps.

@ransome1 @rzelnik @velardevines ... yes, recurrences are one of the trickiest parts of the todo logic, but essential for folks like myself who depend on them to keep from forgetting all of the bills, tax returns, and other periodic obligations that we don't want cluttering up our brains.  I have implemented some new features in my own build of sleek, because my workflow depends on having recurrences work like they do in other todo.txt tools I use (simpletask for android, topydo cli).    So here is the explanation of what I have working, and I'm curious to hear whether you want to get these features to be widely available in sleek.

I have added a new user preference called generatePastRecurrences which is on by default to keep sleek working as it does right now: that is, if you read a task with due date say 6 months in the past and the recurrence says rec:1m, sleek will immediately generate all of the 6 instances for the past 6 months.  For me, this isn't the desired behavior, because it fills up the file with many instances of the same task.  Instead, I set my user preferences to turn generatePastRecurrences to false and then sleek will only create one new recurrence item each time you mark the task completed.  In the example I mentioned, when you mark the 6 months past due task as completed, it will make a new one that is 5 months past due.  So you can step forward one recurrence at a time.  For instance if the task was pay this month's rent due:2021-01-01 rec:1m then when you pay the January rent, it will make one only for the February rent, and not generate all of the intermediate ones immediately.  I hope that is a clear explanation!
Now you might be saying, what if I want to always start the recurrence from today's date rather than from the previous due date?  This would make sense if your task is, for example, wash the windows due:2021-01-01 rec:1m.  In this case if you finally got around to washing the windows right now, 6 months late, it would still make sense to just do it again in a month, not do each of the five washings that you missed!  This is sometimes called non-strict recurrence, whereas the one where you have to do all five (like paying the rent) is called strict recurrence.  As far as I can tell, sleek always uses strict recurrence right now, but I have added a new user preference alwaysStrictRecurrences which defaults to true for the existing behavior, but I set it to false.  Then the syntax rec:1m will give the non-strict recurrence which is relative to today's date, for the window washing case.
And then I have implemented the syntax rec:+1m with the optional plus sign to indicate that I want strict recurrence.  So this gives you the option of recurring a month after the previous due date (for the rent case) and also you can skip the plus sign to recur a month after today's (completion) date, for the window-washing case.  This syntax is compatible with simpletask and topydo recurrence behavior.  Unfortunately it can't be used yet because it depends on a bug in jsTodoTxt getting fixed.  I have fixed that bug in my fork of jsTodoTxt and made a pull request several days ago, but haven't heard from the maintainer yet.

Sorry for the long explanation.  I hope that is clear.  For another explanation of recurrences, see also topydo — recurrence.
Let me know if you have questions.  I could push my changes up to my fork of sleek on github in case you want to take a look, but we really can't fully use them until the upstream bug Pull Request #30 · jmhobbs/jsTodoTxt with plus signs is fixed.
-- zerodat

@zerodat thank you for your explanation. Yes, "washing the windows" vs "paying the rent" are good examples of non-strict vs strict recurrences. Good to see that there are other users with the same need to differentiate between these two types of recurrence. Now I see that both types make sense in different use cases.

@zerodat, if it is not too much effort for you, please feel free to send me a PR and I'll check it. But I need to be honest here, it won't have the highest priority as it is not a critical fix or feature right now.
I think for now I will remove the automated creation as it seems to be connected to more confusion then actual benefits.

@ransome1, yes you could remove the old behavior or change the default to the new behavior.
My PR is a two-line change (one line in each of two files).  To make it default to the new behavior, just change the default value in main.js.
If you would rather remove it completely, then you could just delete the block in todos.mjs instead of checking for the user preference.  No change needed to main.js.  That was what I did originally last week, and it worked well for me, but I wasn't sure whether other users preferred the existing behavior, so I added the user preference.

@rzelnik, auto creation has been removed in https://github.com/ransome1/sleek/releases/tag/v1.0.5
Could you please switch to this version, check it and close this issue, if it is fixed?

I confirm that the automated creation is inactive as expected, but there is an issue with marking the tasks as done: when I create a task with due date today, recurring daily and mark it as done, sometimes it works well, sometimes it creates a task with different content. When I mark tomorrow's recurrence as done, nothing happens, or it creates a new task with the same due date (tomorrow). Sometimes it is impossible do delete an existing task, etc.

I'll look into it

@rzelnik could you please do me a favor and describe this issue with examples of what you do, what is being created and what actually happens?
I can't find any bug but I have the feeling that you might have a different expectation on how the recurence function is suppose to work than it is actually been implemented.
As for you tomorrow example: If you complete a todo with a recurrence (rec:d), the recurrence time frame of 1 day will be added to the date of completion (which is now). So the first todo will create a new todo with due date set to tomorrow. And if you complete this one now, sleek will again create a todo with rec:d (one day) and add this time frame to the date of completion (now).
I tried to describe how this works in the Wiki: https://github.com/ransome1/sleek/wiki/Recurring-todos/_edit

I created some testing tasks in a new todo.txt file and tried to mark them done and back, but I can't reproduce any of the previously described issues. I think we can consider it to be working well. If some new issue arises, I will create a separate ticket with a step-by-step sequence how to reproduce the issue.

Thanks @rzelnik !


Timeout with base/satellite setup

I just tried to re-activate my base/satellite setup, but when sending an intent from the satellite to the base, I get an timeout error:
[WARNING:2021-07-23 02:27:18,532] rhasspyserver_hermes: Message queue missing (topic=hermes/error/nlu, id=c6c5b756-6a47-4531-9f02-a226f391341f)
[DEBUG:2021-07-23 02:27:18,530] rhasspyserver_hermes: Handling NluError (topic=hermes/error/nlu, id=c6c5b756-6a47-4531-9f02-a226f391341f)
[ERROR:2021-07-23 02:27:18,488] rhasspyserver_hermes:
Traceback (most recent call last):
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1869, in dispatch_request
    return await handler(**request_.view_args)
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 1452, in api_text_to_intent
    site_id=site_id,
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 2706, in text_to_intent_dict
    text, intent_filter=intent_filter, site_id=site_id
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 524, in recognize_intent
    handle_intent(), messages, message_types
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 995, in publish_wait
    result_awaitable, timeout=timeout_seconds
  File "/usr/lib/python3.7/asyncio/tasks.py", line 423, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
[DEBUG:2021-07-23 02:26:48,457] rhasspyserver_hermes: Publishing 273 bytes(s) to hermes/nlu/query
[DEBUG:2021-07-23 02:26:48,456] rhasspyserver_hermes: -> NluQuery(input='Lautsprecher an', site_id='satellite', id='93bc81a6-1831-485b-8c94-913e302afcd8', intent_filter=None, session_id='93bc81a6-1831-485b-8c94-913e302afcd8', wakeword_id=None, lang=None, custom_data=None, asr_confidence=None, custom_entities=None)

Setting of the base (running on proxmox):

{
    "handle": {
        "system": "hass"
    },
    "home_assistant": {
        "access_token": "xxx",
        "url": "https://<IP_ADDRESS>:8123"
    },
    "intent": {
        "system": "fuzzywuzzy"
    },
    "mqtt": {
        "site_id": "base"
    },
    "speech_to_text": {
        "system": "kaldi"
    },
    "text_to_speech": {
        "system": "picotts"
    }
}

Setting of the satellite (running on a raspberry 3B):

{
    "home_assistant": {
        "url": ""
    },
    "intent": {
        "remote": {
            "url": "http://<IP_ADDRESS>:12101/api/text-to-intent"
        },
        "system": "remote"
    },
    "microphone": {
        "pyaudio": {
            "device": "1"
        },
        "system": "pyaudio"
    },
    "mqtt": {
        "site_id": "satellite"
    },
    "sounds": {
        "aplay": {
            "device": "default:CARD=Headphones"
        },
        "system": "aplay"
    },
    "speech_to_text": {
        "remote": {
            "url": "http://<IP_ADDRESS>:12101/api/speech-to-text"
        },
        "system": "remote"
    },
    "text_to_speech": {
        "espeak": {
            "voice": "de"
        },
        "remote": {
            "url": "http://<IP_ADDRESS>:12101/api/text-to-speech"
        },
        "system": "remote"
    },
    "wake": {
        "pocketsphinx": {
            "keyphrase": "okay snips"
        },
        "porcupine": {
            "keyword_path": "computer.ppn"
        },
        "raven": {
            "keywords": {
                "default": {
                    "enabled": true
                }
            }
        },
        "snowboy": {
            "model": "Hey_computer.pmdl"
        },
        "system": "snowboy"
    }
}

When I ping the base from the satellite I get positive results:
pi@raspberrypi:~ $ ping <IP_ADDRESS> -p 12101
PATTERN: 0x121001
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=64 time=1.79 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=64 time=1.72 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=64 time=1.75 ms
64 bytes from <IP_ADDRESS>: icmp_seq=4 ttl=64 time=2.15 ms
64 bytes from <IP_ADDRESS>: icmp_seq=5 ttl=64 time=1.91 ms
64 bytes from <IP_ADDRESS>: icmp_seq=6 ttl=64 time=1.41 ms
64 bytes from <IP_ADDRESS>: icmp_seq=7 ttl=64 time=1.53 ms
--- <IP_ADDRESS> ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 15ms
rtt min/avg/max/mdev = 1.410/1.751/2.152/0.229 ms

Any idea why I get this timeout? How can I solve this? Thanks!

Don't know, if this is related to your problem. But you have to define the satellites siteId in the base-configuration. See https://rhasspy.readthedocs.io/en/latest/tutorials/#shared-mqtt-broker

Hmm I don't use MQTT, so I'm not sure if this is the issue 🤔

For anybody who also struggles with this issue:
@drhirn was right, you need to set the satellite siteIds for every remote action, even if it is not MQTT (maybe the docs should be updated to reflect this)


when using asNavFor or pagination main slider doesnt update if fade/slide isnt finished.

====================================================================
If I have two sliders one main slider and one thumbnail slider say and I use asNavFor , everything works fine unless I click on one of the thumbnails and then quickly click another one, the thumbnail slider will update to the correct slide but the main slider it is controlling does NOT change . I slowed down the fade in the fiddle to emphasize the issue.
https://jsfiddle.net/qxyh0jp6/3/

Click on one of the thumbnails
Click on another thumbnail quickly before the main slide has finished fading.

====================================================================
Note this also happens when using the pagination method, which can be seen in this fiddle
http://jsfiddle.net/Virtual/webyrb9d/

Hey @nickfmc interesting! I think you're right that this is not ideal, but I don't know how much support we can expect for a fix in the near term. Happens with dots too. I'll add it as an enhancement to consider, and perhaps it will be attended to in v2.

ya, I always crank up the speed as a workaround when using asNavFor but again not ideal when you want to have a slower transition.

Agreed. I'll definitely leave it open so we don't lose track of it. Thanks man.

Try out this fiddle:
Fiddle
waitForAnimate: false,
Is this what you want?

Awesome @mazzaker exactly the problem I was having, updated my production sites.  Working like a charm, haha it was right there the whole time, Thanks!


antimeridian.bbox possibly returning wrong result

I have a set of features in the attached file which are on both sides the antimeridian, but do not span it. However, antimeridian.bbox appears to be returning their values in the wrong order. Or, is the the desired behavior, as none of the features cross the antimeridian itself?
import antimeridian
import geopands
antimeridian.bbox(antimeridian.fix_multi_polygon(geopandas.read_file('tricky.zip').geometry.loc[0]))

produces:
[-179.92643501801794, -19.044135782844712,<PHONE_NUMBER>6944076, -18.555752850452095]
tricky.zip

Looks like a bug, thanks for reporting!

After digging in, it looks like tricky.zip was not being picked up as antimeridian-crossing because it doesn't have any points coincident to the antimeridian:

I could implement a heuristic to guess that a given set of polygons are "close enough" to the antimeridian to consider the multipolygon antimeridian crossing, but I'm not quite sure what that would be. I'll think about it.

Do you see the shapes on the other side (at about -19, -179.85)? That was why I considered the dataset to be crossing. My naive thought was that two bounding boxes could be calculated: one that crossed -180 and one that  crossed 0 and the smaller one could be used.

Do you see the shapes on the other side (at about -19, -179.85)?

I did.

two bounding boxes could be calculated: one that crossed -180 and one that crossed 0 and the smaller one could be used.

Yup, that's no a bad heuristic but it could be wrong, e.g. for a bbox that covers New Zealand, Madagascar, and Argentina.
I'm thinking we need more info from the caller, so maybe a force_over_antimeridian argument is the answer, which will calculate the antimeridian-crossing bbox no matter what.

@jessjaco I added a force_over_antimeridian argument to bbox, available in v0.3.9: https://github.com/gadomski/antimeridian/releases/tag/v0.3.9


Simplify and unify model configuration files

New configs:

Provide a config argument string as {model}-{task}-{split}. Example: --config=sfarinet-is2re-all
The main (almost only) code change is in utils.py with the new load_config(config_str) function which processes the files this way:

Load configs/tasks/{task}.yaml

Read its default dict as initial config
Update config with the task's {split} dict

Load configs/{model}.yaml

Update config from the model's default dict
Update config from the model's {task}.default dict if it exists
Update config from the model's {task}.{split} dict

Tada 🎉

Mode is train by default (in {task}.yaml)
All previous configs are moved to a configs/legacy/ folder

Before merging, do you think we shall also change ase_utils.py, OCPCalculator ? It leverages (the "old") config_yml, which may cause troubles if the function is called, wouldn't it ?
Otherwise, it's great !

Done :)


Merge problem with profiles and erl_opts

Trying out profiles and I ran into an issue where the option merging for list values uses lists:keysort, but when specifying erl_opts there are several valid options that are single atoms. This causes things to blow up like this:
23:02:56:$ DEBUG=1 ./rebar3 as test eunit
DEBUG=1 ./rebar3 as test eunit
===> Consult config file "rebar.config"
===> Consult config file "rebar.lock"
===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace
===> Uncaught error: {'EXIT',
                             {badarg,
                              [{lists,keysort,2,
                                [{file,"lists.erl"},{line,755}]},
                               {rebar_state,'-merge_opts/3-fun-0-',3,
                                [{file,"src/rebar_state.erl"},{line,211}]},
                               {dict,update_bkt,4,
                                [{file,"dict.erl"},{line,318}]},
                               {dict,update_bkt,4,
                                [{file,"dict.erl"},{line,320}]},
                               {dict,on_bucket,3,
                                [{file,"dict.erl"},{line,408}]},
                               {dict,update,4,[{file,"dict.erl"},{line,313}]},
                               {dict,fold_bucket,3,
                                [{file,"dict.erl"},{line,434}]},
                               {dict,fold_seg,4,
                                [{file,"dict.erl"},{line,430}]}]}}
===> When submitting a bug report, please include the output of `rebar3 wtf "your command"`

Here is the relevant snippet of my rebar.config:
{erl_opts, [warnings_as_errors,
            warn_untyped_record,
            {parse_transform, lager_transform}]}.

{profiles, [
            {default, [

                      ]},
            {prod, [
                    {relx, [{dev_mode, false}]}
                   ]},

            {test, [
                    {erl_opts, [debug_info]}
                   ]}
]}.

If I remove the atoms from the each erl_opts list then the command runs without error.
Here is the wtf output:
23:05:12:$ ./rebar3 wtf as test eunit
./rebar3 wtf as test eunit
Rebar3 wtf report
 version 3.0.0-pre_alpha
 generated at 2015-02-24T06:09:30+00:00
=================
Please submit this along with your issue at https://github.com/rebar/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: astesteunit
Entered as:
  astesteunit
-----------------
Operating System: x86_64-unknown-linux-gnu
ERTS: Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:0] [kernel-poll:false]
Root Directory: /home/XXX/erlang/17.4-64/lib/erlang
Library directory: /home/XXX/erlang/17.4-64/lib/erlang/lib
-----------------
Loaded Applications:
[{inets,"INETS  CXC 138 49","5.10.4"},
 {ssl,"Erlang/OTP SSL application","5.3.8"},
 {public_key,"Public key infrastructure","0.22.1"},
 {asn1,"The Erlang ASN1 compiler version 3.0.3","3.0.3"},
 {crypto,"CRYPTO","3.4.2"},
 {stdlib,"ERTS  CXC 138 10","2.3"},
 {kernel,"ERTS  CXC 138 10","3.1"}]
-----------------
Escript path: /home/XXX/rebar3
Providers:
  app_discovery as clean compile compile ct deps dialyzer do eunit help install_deps lock new pkgs release shell tar update upgrade version wtf xref

code to blame: https://github.com/rebar/rebar3/blob/master/src/rebar_state.erl#L210-L212
 lists:keymerge(1
               ,lists:keysort(1, OldValue)
               ,lists:keysort(1, NewValue))

So my guess is we'll need to implement our own new fancypants function. We're gonna need a stable sort that works on the key of tuples and will treat atoms as keys, and will then need a merge that works the same way. Unless @tsloughter has another idea on how to go about it, it seems that's what we'll need.
Also we're gonna need test cases for that I guess.

Since we are sorting and merging on element 1 could we just do a merge and sort?
lists:merge(lists:sort(OldValue)
                  ,lists:sort(NewValue))

Ugh, I'm dumb, just finished my coffee and realized that makes no sense.

But maybe we'd be good with just having our own merge that is just a copy of keymerge but doesn't do element(Index, H2) if it isn't a tuple of size > 0?

Do we need the sort to preserve on tuples with the same key? [{a,x},{a,e}] in an option list might want to keep {a,x} first when sorted by key.

I think it should just combine the lists, https://github.com/rebar/rebar3/pull/182

Thanks for the fix!


falsecolor function dose not work

In this example link:
https://rawgit.com/chafey/cornerstone/master/example/falseColorMapping/index.html
Whichever colormap i choose, image always black.

When i install "cornerstone-core": "2.2.6"
It's work fine.


ix-select testing with JEST

Prerequisites

[X] I have read the Contributing Guidelines.
[X] I have not leaked any internal/restricted information like screenshots, videos, code snippets, links etc.

What happened?
I'm writing test cases for a react application. The following behaviour is expected:

A dropdown is being populated from an API call's response being set as a state variable
There is a refresh button which calls the api on click
Whenever the button is clicked the state is updated and so is the dropdown

Issue:
On writing test case for this scenario:

The button successfully makes the API call
The state variable gets updated
The dropdown does not reflect the updated state

What type of frontend framework are you seeing the problem on?
React
Which version of iX do you use?
v2.0.0
Other dependencies:
"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",
"@testing-library/jest-dom": "^6.4.6",
"@testing-library/react": "^16.0.0",
"@testing-library/user-event": "^14.5.2",
Code to produce this issue.
// For the code I've not been able to make the testcase run on stackblitz, but you can get an idea what I'm trying to do

https://stackblitz.com/edit/1zkw8s-96beuv?file=src%2Fdropdown.test.tsx

Test not running at all in example
 FAIL  src/dropdown.test.tsx
  ● Test suite failed to run

    TypeError: Cannot read properties of undefined (reading 'html')

      at new JSDOMEnvironment (file:/home/projects/1zkw8s-96beuv/node_modules/jest-environment-jsdom/build/index.js#cjs:72:44)

Please prepare a Github repo which shows the issue.

@danielleroux please check the following stackblitz code
Github repo: https://github.com/umeWaheed/ix-select-testing

I've been able to resolve this issue by using
fireEvent(<dropdown element>, new CustomEvent('valueChange', { detail: <text of the option> }));


Add AutoTracker

Codecov Report
Patch coverage: 100.00% and no project coverage change.

Comparison is base (0baa2cb) 100.00% compared to head (569f20f) 100.00%.

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #68   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           25        26    +1
  Lines          692       736   +44
  Branches        80        89    +9
=========================================
+ Hits           692       736   +44

Impacted Files
Coverage Δ

src/arctix/stats/__init__.py
100.00% <100.00%> (ø)

src/arctix/stats/auto.py
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Use agentkeepalive to avoid no more living connections error

Related to this problem https://github.com/elastic/elasticsearch-js/issues/196 I found this fix to avoid the 'no more living connections error'.

Thank you for your contribution! new version 1.1.15 is available on npm


Automatically build Perseus Docker container images and push to Docker Hub when code changes in GitHub

Automatically build Perseus Docker container images and push them to Docker Hub when code changes are pushed to the OHDSI Perseus github repo. Tag the Docker image with a version number.
Benefits of this change:

It will ensure that we always have the latest Perseus Docker images in Docker Hub.
It is much faster to deploy Perseus using pre-built Docker Hub images instead of building them locally

One way to accomplish this is to develop GitHub Actions

Done


belgian keyboard issue

we need to include keymap for this keyboard
xrdp-genkeymap /etc/xrdp/km-080c.ini
bug reported in sourceforge.
https://sourceforge.net/p/xrdp/bugs/58/
dears,
i have a belgian keyboard (similar of azerty french but some difference)
i try everything to adapt the keyboard but it' impossible....
i try sudo xrdp-genkeymap /etc/xrdp/km-0409.ini to pu in fr but when i push on a letter, i see number 7?
What can i do ?
regards
Marin

FYI, I managed to make xrdp work on my Pi with Belgian keyboard.
First log into your system with a real screen, not with xrdp. Then you need to copy km-0409.ini to km-0409.ini.bak and generate the correct ini.
You can find the country codes here: http://xrdp.sourceforge.net/documents/keymap/rfc1766.html
For me I needed to:
sudo xrdp-genkeymap /etc/xrdp/km-0813.ini

Then I copied km-0813.ini to km-0409.ini
sudo cp km-0813.ini km-0409.ini

And rebooted, after which I was able to use azerty keyboard.

Do
nano /etc/xrdp/xrdp_keyboard.ini

and edit:
[default_rdp_layouts]
rdp_layout_be=0x00000813
[default_layouts_map]
rdp_layout_be=be

It should work without problems now. The km-00000813.ini is there... but not in the xrdp_keyboard.ini file.

I've created #858.

Merged.


Take a list of loci to report

Take an input file that describes some STR loci to be tested. These will be added to the bounds.
Format something like:
chr    start    stop    repeatunit [name]

Started working on this here: https://github.com/hdashnow/str-dev/tree/locusfile
Remaining issues/todos:

[ ] Allow loci to have labels
[ ] Check for matches (overlap??) between discovered bounds and provided loci to avoid adding duplicates OR just use something to distinguish them.
[ ] Assign str reads from the discovery stage to provided loci (currently only getting spanning pairs/reads for these)
[ ] Reduce the amount of duplicated code (probably will happen naturally once duplicate bounds are avoided).

Seems inefficient?
afd4bafd3ef214291a572df527c6b027bad0fa7d

leave it and I'll implement with this: https://github.com/brentp/nim-lapper


Gist Support

Is anyone implementing Gist/Web support at this time? If not, I can do it.

There's no-one working on this as far as I know.
Just one thing to be aware of: the reason I've avoided doing this up to now is that I have a pathological hatred of crates that require dicking about with the environment to get them to build.  I'm yet to find a library that does HTTPS requests that actually builds on Windows without having to set up a fake UNIX environment or do global installs that break other software I'm using.
I actually managed to get hyper to work with SSL once upon a time.  The very next update broke it for no readily explainable reason, and I've just had a zero-tolerance policy for such crates since.
I'm not saying I will reject something that adds a dependency on openssl or curl, but I'm significantly more likely to merge and enable by default if I can just do cargo build and have it work.
I do want this (being able to run a playpen URL locally would be sweet), I'm just not looking forward to the tremendous hassle HTTPS inevitably seems to bring...

I can't really speak for Windows support as I haven't used Windows in many years. All of the latest programming languages have limited support for Windows, including the popular Node language, so it's no surprise that getting Windows configured as a development environment for Rust is hard. It's hard to get Windows configured as a development environment in general -- it's just not a good place for developing software.
Submitted an implementation that uses the curl crate: https://github.com/DanielKeep/cargo-script/pull/21


call context cancel func

I think this should be revised.

Ah, yes! it's definitely a memory leak. Thanks for your review!


Optimizing multi_gpu_model

Opening this to discuss possible optimizations to multi_gpu_model.
Based on the code, heres a few ideas I could think of:

Limiting the number of GPUs to a fewer number, in case the model is small (<400k params, as described in the analysis doc). However, this might not fit well for users who still want to try out parallelization (maybe provide an optimize param?)

As far as I understand, the ops created in def get_slice(data, i, parts) get added to individual GPUs as a result of with tf.device. Would it make sense to place them on the CPU instead? (and only do the core model running on the GPUs)

Are there any other ideas on how to go about this?
There's also the idea of auditing TF behavior to check if sub-batches indeed run on the devices they were assigned to. I could not find any documentation on how/why to achieve this.
I am kinda new to Keras, so I might be missing something trivial :-)

@fchollet FYI

For analysis of time series unsing LSTM's and ,  get_slice() does work as intended when >1 GPU is used (which is sort of the idea...). Sequence bucketing might be a way to solve this (https://arxiv.org/ftp/arxiv/papers/1708/1708.05604.pdf) but is this is an optimal way of doing this requires more analysis.


Add links to the sum and multiply plugins to the docs website

Story
As a user I want to be able to find plugin documentation easily.
Rationale
We link out to the plugin README docs from the docs website. These new plugins are not yet linked.
Implementation details
Add links to the plugins page on the docs website
Priority
4/5
Scope
docs only
Size
1 hour
What does "done" look like?
links to both plugins available on the plugins page of docs website
Does this require updaes to documentation or other materials??
docs only
What testing is required?
none
Is this a known/expected update?
expected in sprint 7

Closing as complete


Double Voting

When I ran a test vote in Discord, one of the members voted twice; thumbs up and thumbs down. Both of his votes were recognized. Seems like it would be wise to only let one of the votes count. Or, in order to maintain 100% quorum for those who do not have an affirmative or negative vote, maybe allow for an "abstain" button (i.e. 🤷♀).

Hey @KendyLand thanks for the suggestion!
How's someone's double vote different from not voting at all?

@aesedepece
As far as "in favor" or "opposed" vote counts go, it's the same thing. I've been a member in orgs that use Robert's Rules, and it was helpful to have an "abstain" option. This way you can encourage quorum by giving members a clear option if they don't wish to vote in a positive or negative way, rather than having them vote positive and negative, to accomplish their intention. It's mostly a UX/UI suggestion, to offer an "abstain" emoji/button, to encourage as many members as possible to participate, in order to obtain quorum. Overall, not a huge issue, but will make the process more user-friendly.


Add arguments property to the script process

What would you like to be added:
Add an arguments property to the script process.
Why is this needed:
Adding an arguments property to the script process is needed to allow scripts to access the arguments passed to them directly within the script's execution context.

Fixed by #960


Redefined protection

In your opinion wouldn't in be better if this new REDEFS function operate on an authenticate stack, As other functions that impact of the configuration ?

Nope, as a stack can only be authenticated once and as it would force to use AUTHENTICATE in secure scripts thus including expiring tokens in them.


Find-DbaUnusedIndex fails in capitalization-sensitive DBs

Before submitting a bug report:

[X] Ensure you are able to reproduce it on the latest released version (we release often)
[X] Verified this bug is not already reported in an issue
[X] Verified errors are not related to permissions
[X] Can reproduce in a clean PowerShell session (clean = powershell -NoProfile)

Steps to Reproduce
The problem exists in the current development branch.
Basically, you just need to run the cmdlet against any case-sensitive DB that has been up for more than a week or so.
If you look at the query that Find-DbaUnusedIndex runs, you can clearly see that some of the objects in the FROM and JOIN clauses are capitalized and some are not. They should be lower case.
<#
    Import-Module DbaTools
    Find-DbaUnusedIndex -sqlinstance 'server_with_one_or_more_case_senstiive_dbs'
#>

Expected Behavior
The expected behavior is that the case-sensitive DB would be searched for unused indexes.

Actual Behavior
The cmdlet uses TSQL to look for unused indexes. This TSQL query uses a mishmash of mixed case in the FROM and JOIN clauses.  This doesn't matter in non-case-sensitive DBs so it runs OK in the majority of cases.
BUT, in a case-sensitive DB the query fails because there is no such object as 'SYS.TABLES' (only 'sys.tables').
I guess that this hasn't cropped up in testing because:

Not a lot of people have case-sensitive DBs

The cmdlet doesn't generate a hard error. It just says something (in yellow text) about the query failing.  This is easy to overlook, especially when testing dozens of DBs.

The cmdlet doesn't run against server instances that have only been up for less than a week or so because they have not been up "long enough" to accumulate useful information. If you are spinning up a test infrastructure via AppVeyor, it's never going to be up "long enough" for a good test.

Environmental data

PowerShell:

Name                           Value

PSVersion                      5.1.17134.228
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17134.228
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

SQL Server:
Microsoft SQL Server 2016 (RTM-GDR) (KB4019088) - 13.0.1742.0 (X64)
Jul  5 2017 23:41:17
Copyright (c) Microsoft Corporation
Developer Edition (64-bit) on Windows 10 Pro 6.3  (Build 17134: ) (Hypervisor)

I already have a fix coded up in a branch on my fork. Let me know if you want a PR. Thx.

Yes please! I'd love a fix

PR is #4534. It's just five lines that need to have their capitalization
adjusted slightly.
On Wed, Nov 14, 2018 at 4:32 PM Chrissy LeMaire<EMAIL_ADDRESS>wrote:

Yes please! I'd love a fix
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/sqlcollaborative/dbatools/issues/4389#issuecomment-438826288,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AK5fUSBO4mygmbDeaqOiWY_znBy1v77jks5uvIv0gaJpZM4YFicK
.

--
Ashdar Partners
Specializing in Performance Engineering (and Re-engineering)
Blog http://blog.ashdar-partners.com/, LinkedIn
http://www.linkedin.com/in/darinstrait, GitHub
https://github.com/AshdarPartners, Twitter
https://twitter.com/ashdarpartners


⚠ Bofrost.at has degraded performance

In c572920, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1335 ms

Resolved: Bofrost.at performance has improved in 4915046 after 7 minutes.


Toolbar header not draggable

When you make a toolbar-header div, it's not draggable. It needs the -webkit-app-region: drag property in its CSS

There is helper class .draggable which you can use.

Here's an example:

Photon

Note: Look at header class: draggable

thanks for the hint - maybe you can add it to the example?

I want to work on this is anyone working on it or i can start


[Feature] set Download directory for models while pulling/downloading from ollama

Hey,
In Ubuntu 23.10,
Previously, Ollama used to download the models into the root directory.
Now, it is downloading in the Home directory.  How do you control this?
I suggest a directory flag to let the user decide in which folder the model is supposed to go.

Hi @tikendraw thanks for opening the issue, you should be able to set the directory that models are stored in using the OLLAMA_MODELS environment variable. There's a bit more detail on that here:
https://github.com/jmorganca/ollama/blob/main/docs/faq.md#how-do-i-set-them-to-a-different-location
Let me know if you have any more follow up questions, I'm gonna resolve this one for now as it seems to be covered. n

Setting the env variable, globally, permanently, and on the service itself, doesnt take effect, it keeps looking for space on /usr/share, is there something I could check?

Here is what i did that worked for me. Try this if you have not.
in a etc/systemd/system/ollama.service.d/environment.conf file (if doesn't exist, then make one) and
write this or adjust accordingly
[Service]
Environment="OLLAMA_HOST=<IP_ADDRESS>:11434"
Environment="OLLAMA_MODELS=/usr/share/ollama/.ollama/models"

restart maybe.

It works, thanks