opentapioca long-due contributions

Hello @wetneb
I worked intensively on opentapioca in the last days and I wanted to summarize the major points. The general aim is to add proper multilingual support to OpenTapioca, improve its performance (in relations to its bootstrap, like creating the index) and including more entities into the indexing by relying on the type inheritance.
As I started this work a long time ago and only recently got back to it, some of the things might not make 100% sense :-)
In short:

Multiple language support (Solr configset and document structure)
Performance (train-bow and index-dump)
Wikidata type inheritance

Multiple language support
In order to properly support multiple languages I created a different structure for doc where the labels would be stored in tag_{language}name and the aliases in tag{language}_alias.
This is fundamental to ensure that I can tag targeting a specific language (set as a parameter to the annotate api).
In order to enable this I created a custom solr image (based on the official one) which includes the required extra libs and support files:
https://github.com/wordlift/docker-solr-opentapioca
You can see the managed-schema configuration here:
https://github.com/wordlift/docker-solr-opentapioca/blob/master/tapioca/conf/managed-schema
The image is published to docker hub as well:
https://hub.docker.com/r/wordlift/solr-opentapioca/tags
You can spot the updated structure in my own fork:

Indexing: https://github.com/ziodave/opentapioca/blob/e44fa870ca12373a5bb0f51694a756974c600799/opentapioca/indexingprofile.py#L204-L211
Tagging: https://github.com/ziodave/opentapioca/blob/e44fa870ca12373a5bb0f51694a756974c600799/opentapioca/tagger.py#L47-L62

Performance (train-bow and index-dump)
I tried to boost as much as possible the performance of train-bow and index-dump since I would like to be able to run many tests and I can't wait one week before I can recreate an environment.
I used cProfile to profile the app and remove bottlenecks, working in particular on the readers (removing the json loads from there) and replacing the system json.loads with the more efficient msgspecs package, while defining the schema that we need.
I also added rxpy and added multithreading support for index-dump (i.e. not waiting for Solr to reply while moving forward processing new docs):
https://github.com/ziodave/opentapioca/blob/master/opentapioca/taggerfactory.py#L146-L159
The result is that I believe that I would be able to index the full wikidata dump in ~4 hours (compared to the 3-4 days I needed before) on my M1.
This is in conjuction with the type inheritance cache (more on this on the next paragraph).
Wikidata type inheritance
More than a year ago I noticed that some of the entities I needed where not in the index (e.g. ingredients). By investigating I found out that we needed to do some sparql queries to Wikidata for the type inheritance. TBH I don't recall how much was already in opentapioca and what I contributed :-)
Anyway as sparql queries are what they are, I added here a @cache hof https://github.com/ziodave/opentapioca/blob/e44fa870ca12373a5bb0f51694a756974c600799/opentapioca/sparqlwikidata.py#L9 which stores the results to disk for future references.
And then I created a GH repo holding this cache:
https://github.com/wordlift/opentapioca-wikidata-query-cache
The increase of speed is drastic and I think the Wikidata folks may also be happy that I don't repeat the same queries.
Summary
In short I really like OpenTapioca and I am trying to grow it according to my needs (and from some conversation I had, also other people needs).
My aim now would be to share this work with you, see if you have the time to review it and eventually merge it upstream.
Thanks for your time 🙏

Hey @ziodave, that is really impressive work! Thanks a lot for offering to upstream it.
Here are some broad comments about your points:

Multiple language support (Solr configset and document structure)

That's really nice! The managed-schema configuration is really impressive - getting the pipeline right for so many languages is a ton of work.
The only issue I could think of is that those pipelines rely on external libs, so we cannot run tapioca off an off-the-shelf Solr instance. But that's a reasonable requirement to introduce given the benefit the change can bring. And your Docker image seems to be a great solution for it (especially since you already include the tapioca configset, which makes set up even easier).
If I understand correctly, the set of languages to be used for the indexing is configurable, so one should still be able to use a stock Solr instance if we only enable English, right?

Performance (train-bow and index-dump)

It's really nice too and I do not see any reason not to have this upstream. The extra dependencies are worth it.

Wikidata type inheritance

We already have a mechanism to fetch all the subclasses of a given type (well, all the ones we can get via a SPARQL query without timeout…) and caching those more aggressively can make sense in some situations.
But for the real-time update of the index, it is still worth refreshing those type hierarchies relatively frequently. When debugging the outcome of a tagging, for instance to understand why a particular entity is picked up or not, it is fairly common to have to fix the type hierarchy. For this to circle back to tapioca, the type hierarchy needs to be refreshed in some way.
How to upstream your changes
I am keen to review and merge any PR you make around those topics.
I don't have a good overview of your fork - I do not know to what extent it contains other changes which you might not be ready to upstream yet, or changes which are too specific to your own use case to be worth including upstream.
I can imagine that it is significant work for you to break down the changes you made in your fork into topical PRs. Perhaps it is a useful exercise to ensure that the changes are well documented, tested, and that the CI remains green. But if you judge that your fork is generic enough and free from such "un-upstreamable" changes, we could simply decide to make your fork the official thing. I'd be happy to tag along as a co-maintainer of course. The way I generally do this is creating a GitHub org for the project, transfer the project from my account to the org and add you as an owner. Then you can just push the changes from your fork there, and done.

If I understand correctly, the set of languages to be used for the indexing is configurable, so one should still be able to use a stock Solr instance if we only enable English, right?

Right now the list of languages is hard-coded but it's easy to move it to a configuration file, so that one may run only english (or any other preferred language):
https://github.com/ziodave/opentapioca/blob/e44fa870ca12373a5bb0f51694a756974c600799/opentapioca/indexingprofile.py#L290-L336
BTW in the process I also moved the configuration to an AppConfig class which makes it easier to deploy on managed environments (e.g. K8s), by using environment variables (or an .env file).

When debugging the outcome of a tagging, for instance to understand why a particular entity is picked up or not, it is fairly common to have to fix the type hierarchy.

Agree, I was thinking that we can also provide an API that, given a Q will tell you whether it'll be indexed or not along with some helpful information.

I can imagine that it is significant work for you to break down the changes you made in your fork into topical PRs. Perhaps it is a useful exercise to ensure that the changes are well documented, tested, and that the CI remains green. But if you judge that your fork is generic enough and free from such "un-upstreamable" changes, we could simply decide to make your fork the official thing. I'd be happy to tag along as a co-maintainer of course. The way I generally do this is creating a GitHub org for the project, transfer the project from my account to the org and add you as an owner. Then you can just push the changes from your fork there, and done.

I ran the UTs and CI is still green with 2 exceptions:

I removed extra_aliases since we have the aliases per language, so I think we can remove this test: https://github.com/ziodave/opentapioca/blob/e44fa870ca12373a5bb0f51694a756974c600799/opentapioca/tests/test_indexingprofile.py#L108-L116
This test yields 1 mention instead of 2, but I still didn't look thoroughly into it: https://github.com/ziodave/opentapioca/blob/e44fa870ca12373a5bb0f51694a756974c600799/opentapioca/tests/test_classifier.py#L59-L62

If I try to open a PR now, there are quite some changes and I am afraid I am to the point where I wouldn't be able to split it into smaller PRs:

If you agree, I would like to get to a point where I can create a copy of opentapioca.org adding a language selector, so that we can run some e2e tests.
And if we like it, we can then move to a separate org 🚀

Sounds like a plan, looking forward to it! :)

Just a quick update: I have the instance ready, I am performing the latest task to prepare and then train the classifier (as a matter of fact it already works, but it doesn't populate the best_qid as I am missing the classifier).

[ ] Use the sample classifier
[ ] To use the classifier, I need to fix the image with the right version of scikit-learn (0.20.1, currently not building in my environment), or investigate whether to upgrade to 1.x see https://github.com/opentapioca/opentapioca/issues/4#issuecomment-1382836587
[ ] At the same time I am preparing a new classifier by using an extensive (21 GB gzipped) model by dbpedia
[ ] I also need to tune the Solr schema (specifically remove the POS filter from the query handler)

@wetneb new logo ☝ I'll move forward with updating the repo 🙏

Hi everyone,
Thanks for this amazing project!
I have one question, while running @ziodave's fork and trying to annotate some text, I get AttributeError: 'WikidataGraph' object has no attribute 'pagerank' in file "/app/opentapioca/wikidatagraph.py", line 250, in get_pagerank. Is the sample PageRank wd_2019-02-24.pgrank.npy available here not compatible? Do I have to rebuild it again?
If that's relevant, I'm using scikit-learn 0.23.2 with the sample_classifier.pkl and wd_2019-02-24.bow.pkl BOW as well.


gkev2 - should be able to edit "network policy"

What kind of request is this (question/bug/enhancement/feature request): Bug
Steps to reproduce (least amount of steps as possible):

master-618c9efe52856daff6a534097bd927b69f03f4e4-head
Create GKE
Select  Network Policy
Network Policy Config (master only)
Create --> Active
Edit

Result:
UI lets users to disable / enable Network Policy Config (master only), but not Network Policy.
If a user disable Network Policy Config (master only), without disabling Network Policy from GCP (Rancher UI doesn't allow it), then
Googleapi: Error 400: Network policy addon cannot be removed until network policy has been disabled on the nodes., badRequest

Either, we make "network policy" editable or we should make "Network Policy Config (master only)" un-editable as well.
If we make "network policy" editable, then "Network Policy Config (master only)" should become editable only when user has disabled "network policy" (to avoid the error ^)
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI):
Installation option (single install/HA):

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported):
Machine type (cloud/VM/metal) and specifications (CPU/memory):
Kubernetes version (use kubectl version):

(paste the output here)

Docker version (use docker version):

(paste the output here)

@prabalsharma I've opted to make the option uneditable because one is and one is not according to comment. I believe this route offers the least amount of confusion and if they need to edit these params they can from the API.

@prabalsharma I've opted to make the option uneditable because one is and one is not according to comment. I believe this route offers the least amount of confusion and if they need to edit these params they can from the API.

anh interesting. I can edit both from GKE UI, just have to be in a specific sequence. I guess editing from API should be okay.

@westlywright In the current situation, I get the error (as mentioned above). Should we not make "Network Policy Config (master only)" un-editable as well?

the new implementation is working for almost every combination of creation and edit. Except;
If a user creates a GKE cluster with only Network Policy for Nodes and does not check Network Policy Config (master only), GKE cluster will get created with Network Policy enabled (as expected). However, Rancher will throw this error after cluster creation: Waiting for API to be available; googleapi: Error 400: Network policy addon cannot be removed until network policy has been disabled on the nodes., badRequest
An easy fix will be; on cluster creation if a user checks Network Policy for Nodes, UI should enable Network Policy Config (master only).
It is important that this fix is done only for creation, as Edit follows a specific sequence (which has already been implemented).

@codyrancher - Is this something that the UI team is planning to tackle for stabilization in 2.6.1 or is there some backend work here needed? Just trying to get an idea as I groom through v2.6.1

@Jono-SUSE-Rancher  Without digging in too deep I believe this is just UI for right now. I'll update if anything is uncovered while trying to fix.

Great, thanks for updating me - I'll keep this in 2.6.1 for now and sync up with @gaktive and @nwmac to determine timing.


[BUG] Combat Maneuvers disappear from Combat Tracker when using Set Maneuver button

When right-clicking on a token and selecting 'Set Maneuver', choosing a Maneuver causes the icon to appear next to that token on the Combat Tracker. However, when moving to the next token in initiative order, the Maneuver icon disappears from the previous token. Also, when attempting to use the 'Set Maneuver' button to set Maneuvers for multiple tokens without cycling through initiative, the Maneuver icon for the previous token disappears whenever a Maneuver is set on the currently selected token. Double-clicking on the token to open its character sheet shows that the Maneuver did set properly, but does not display the icon on the Combat Tracker.
This behavior only affects tokens that had their Maneuvers set via the 'Set Maneuver' button. If Maneuvers are set by opening the token's character sheet and setting the Maneuver from the Maneuver drop-down in the Conditions section, the Maneuver icons display properly in the Combat Tracker.
Tested on two different systems.
System 1
Foundry Virtual Tabletop Version 9 Build 249
GURPS 4th Ed. Game Aid (Unofficial) v. 0.13.4
Modules Installed:
Dice So Nice! v. 4.4.0
Foundry VTT Arms Reach v.2.1.18
libWrapper v. <IP_ADDRESS>
PDFoundry v. 0.9.1
Stairways (Teleporter) v. 0.8.6
System 2
Foundry Virtual Tabletop Version 9 Build 249
GURPS 4th Ed. Game Aid (Unofficial) v. 0.13.5
Modules Installed: None

Well darn, that is a bummer... I cannot recreate this issue.
When I start a combat, and use the Set Maneuver button, the maneuver continues to be displayed even after I move to the next token in the combat order.

I went through 8 rounds of changing maneuvers through various means, and the icons will always displayed.
Gamemaster
Modules:
About Face
Chat Portrait
CodeMirror
Compendium Folders
Dice So Nice!
Forien'd Quest Log
Forien's Unidentified Items
FXMaster
GM Notes
Image Previewer
lib - Color settings
libWrapper
Math.js
Not Your Turn!
NPC Chatter
Patrol
PDFoundry
Permission Viewer
Pin Cushion
Sequencer
Settings Extender
Sidebar Expander
socketlib
SortableJS
Splatter
Target Recall
Tidy UI
Token Attacher
Token Chat link
Token Magic FX
Token Tooltip Alt
TriggerHappy
Turn Alert
Turn Marker
Wow, I never knew I so many modules enabled... I will try a new world with no modules to see if I can recreate.

Nope, sorry.   I created a world with NO modules, and (once I set the system setting for "Maneuver Icon Visibility" to "everyone") I always saw the maneuver icons.

That helped! I don't recall changing "Maneuver Icon Visibility", but I checked that setting and it was set to "No One". Changing it to "GM and Owner Only" restored the behavior I was expecting to see on my end.
That being the case, I guess the actual bug I'm experiencing is that if "Maneuver Icon Visibility" is set to "No One", the "Set Maneuver" button works properly, but the "Maneuver" drop down on the character sheet causes the Maneuver to appear for all players when it should not. Fortunately, that's a bug I can deal with, though I'm sure for more secretive GMs that could be a potential issue.


Show edit and delete buttons in budget view

Add edit and delete buttons to budget cell
update budget table view when we insert/edit a budget
Add RefreshControl to the Budget table view

I add the same change to the Transaction table view


The custom effect from the setting is not available anymore

I have lost all my custom effects, and can not see the custom effect setting to create new custom effects in the settings part of  Foundry.
This happens even with only this module and library on.

May have posted in the wrong module page. sorry!


I think this is a really bad bug.
I have installed freerdp 2.0.0-rc4 on my Centos 7 via yum
$ xfreerdp --version
This is FreeRDP version 2.0.0-rc4 (n/a)

I connect to Centos via VNC. Inside that, I launch xfreerdp from terminal to access a remote windows machine. Problem is that, when I close the freerdp window (clicking on cross symbol), the VNC service is terminated! I have tried multiple times.
Please see the output of terminal below, where Xvnc process is running, but after the second process query, there is no Xvnc process.
[mahmood@hpc ~]$ ps ux | grep Xvnc
mahmood   6357  0.5  0.1<PHONE_NUMBER>8 pts/2    Sl   Oct09   0:07 /usr/bin/Xvnc :1 -auth /home/mahmood/.Xauthority -desktop hpc.scu.ac.ir:1 (mahmood) -fp catalogue:/etc/X11/fontpath.d -geometry 1300x900 -pn -rfbauth /home/mahmood/.vnc/passwd -rfbport 5901 -rfbwait 30000
mahmood  10845  0.0  0.0 112720   956 pts/2    S+   00:11   0:00 grep --color=auto Xvnc
[mahmood@hpc ~]$
[mahmood@hpc ~]$
[mahmood@hpc ~]$ ps ux | grep Xvnc
mahmood  11047  0.0  0.0 112716   960 pts/2    S+   00:13   0:00 grep --color=auto Xvnc
[mahmood@hpc ~]$

Also, you see the picture below, then right after clicking the close button (see that button is pressed), the VNC viewer is terminated.

Checking system log, at the time I press the close button, I see these messages.
Oct 10 00:14:32 hpc abrt-hook-ccpp: Process 11309 (Xvnc) of user 1000 killed by SIGABRT - dumping core
Oct 10 00:14:40 hpc journal: gnome-shell: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc org.a11y.atspi.Registry: X connection to :1 broken (explicit kill or server shutdown).
Oct 10 00:14:40 hpc journal: gsd-media-keys: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc org.gtk.vfs.Daemon: A connection to the bus can't be made
Oct 10 00:14:40 hpc journal: gnome-terminal-server: Fatal IO error 104 (Connection reset by peer) on X server :1.
Oct 10 00:14:40 hpc com.redhat.imsettings: [<PHONE_NUMBER>.082072]: IMSettings-Daemon[11373]: INFO: Release the ownership of com .redhat.imsettings
Oct 10 00:14:40 hpc com.redhat.imsettings: Exiting...
Oct 10 00:14:40 hpc com.redhat.imsettings: [<PHONE_NUMBER>.082578]: GLib-GIO[11373]: CRITICAL **: Error while sending AddMatch()  message: The connection is closed
Oct 10 00:14:40 hpc com.redhat.imsettings: [<PHONE_NUMBER>.082772]: GLib-GIO[11373]: CRITICAL **: Error while sending AddMatch()  message: The connection is closed
Oct 10 00:14:40 hpc com.redhat.imsettings: [<PHONE_NUMBER>.083094]: IMSettings-Daemon[11373]: INFO: Unloading imesttings module:  qt
Oct 10 00:14:40 hpc com.redhat.imsettings: [<PHONE_NUMBER>.083320]: IMSettings-Daemon[11373]: INFO: Unloading imesttings module:  gsettings
Oct 10 00:14:40 hpc com.redhat.imsettings: [<PHONE_NUMBER>.083511]: IMSettings-Daemon[11373]: INFO: imsettings-daemon is shut do wn.
Oct 10 00:14:40 hpc journal: gsd-keyboard: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Wacom.desktop' exited with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Wacom.desktop' exited  with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Clipboard.desktop' exi ted with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Lost name on bus: org.gnome.SessionManager
Oct 10 00:14:40 hpc journal: gsd-wacom: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Clipboard.desktop' exited with code 1
Oct 10 00:14:40 hpc org.gtk.vfs.Daemon: A connection to the bus can't be made
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Power.desktop' exited  with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.XSettings.desktop' exi ted with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Color.desktop' exited  with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Keyboard.desktop' exit ed with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.MediaKeys.desktop' exi ted with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: App 'nautilus-classic.desktop' exited with code 1
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Rfkill.desktop ' killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.ScreensaverPro xy.desktop' killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.PrintNotificat ions.desktop' killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Account.deskto p' killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Sharing.deskto p' killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Smartcard.desk top' killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.A11ySettings.d esktop' killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Sound.desktop'  killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Housekeeping.d esktop' killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Mouse.desktop'  killed by signal 15
Oct 10 00:14:40 hpc gnome-session: gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Datetime.deskt op' killed by signal 15
Oct 10 00:14:40 hpc org.gnome.Shell.CalendarServer: gnome-shell-calendar-server[11626]: Lost (or failed to acquire) the name  org.gnome.Shell.CalendarServer - exiting
Oct 10 00:14:40 hpc tracker-miner-user-guides.desktop: Received signal:15->'Terminated'
Oct 10 00:14:40 hpc tracker-miner-user-guides.desktop: OK
Oct 10 00:14:40 hpc tracker-miner-fs.desktop: Received signal:15->'Terminated'
Oct 10 00:14:40 hpc tracker-miner-fs.desktop: OK
Oct 10 00:14:40 hpc org.freedesktop.Tracker1: Received signal:15->'Terminated'
Oct 10 00:14:40 hpc org.freedesktop.Tracker1: OK
Oct 10 00:14:40 hpc tracker-extract.desktop: Received signal:15->'Terminated'
Oct 10 00:14:40 hpc tracker-miner-apps.desktop: Received signal:15->'Terminated'
Oct 10 00:14:40 hpc tracker-miner-apps.desktop: OK
Oct 10 00:14:40 hpc journal: nautilus-desktop: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Lost name on bus: org.gnome.SessionManager
Oct 10 00:14:40 hpc org.gtk.vfs.Daemon: A connection to the bus can't be made
Oct 10 00:14:40 hpc journal: gsd-clipboard: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Power.desktop' exited with code 1
Oct 10 00:14:40 hpc journal: gsd-xsettings: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.XSettings.desktop' exited with code 1
Oct 10 00:14:40 hpc journal: gsd-color: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Color.desktop' exited with code 1
Oct 10 00:14:40 hpc journal: abrt: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.Keyboard.desktop' exited with code 1
Oct 10 00:14:40 hpc journal: gsd-power: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: App 'org.gnome.SettingsDaemon.MediaKeys.desktop' exited with code 1
Oct 10 00:14:40 hpc journal: Error releasing name org.freedesktop.portal.IBus: The connection is closed
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: App 'nautilus-classic.desktop' exited with code 1
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.UDisks2VolumeMonitor disconnected from the bus; removing dri ves/volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Rfkill.desktop' killed by sig nal 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.AfcVolumeMonitor disconnected from the bus; removing drives/ volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.ScreensaverProxy.desktop' kil led by signal 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.UDisks2VolumeMonitor disconnected from the bus; removing dri ves/volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.PrintNotifications.desktop' k illed by signal 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.AfcVolumeMonitor disconnected from the bus; removing drives/ volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Account.desktop' killed by si gnal 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.GPhoto2VolumeMonitor disconnected from the bus; removing dri ves/volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Sharing.desktop' killed by si gnal 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.GPhoto2VolumeMonitor disconnected from the bus; removing dri ves/volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Smartcard.desktop' killed by  signal 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.MTPVolumeMonitor disconnected from the bus; removing drives/ volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.A11ySettings.desktop' killed  by signal 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.MTPVolumeMonitor disconnected from the bus; removing drives/ volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Sound.desktop' killed by sign al 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.GoaVolumeMonitor disconnected from the bus; removing drives/ volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Housekeeping.desktop' killed  by signal 15
Oct 10 00:14:40 hpc journal: Owner of volume monitor org.gtk.vfs.GoaVolumeMonitor disconnected from the bus; removing drives/ volumes/mounts
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Mouse.desktop' killed by sign al 15
Oct 10 00:14:40 hpc journal: Error while sending AddMatch() message: The connection is closed
Oct 10 00:14:40 hpc gnome-session-binary[11340]: WARNING: Application 'org.gnome.SettingsDaemon.Datetime.desktop' killed by s ignal 15
Oct 10 00:14:40 hpc journal: Error while sending AddMatch() message: The connection is closed
Oct 10 00:14:40 hpc journal: Error while sending AddMatch() message: The connection is closed
Oct 10 00:14:40 hpc journal: Error releasing name org.gnome.evolution.dataserver.Subprocess.Backend.Calendarx12016x2: The con nection is closed
Oct 10 00:14:40 hpc journal: e_intervaltree_destroy: assertion 'E_IS_INTERVALTREE (tree)' failed
Oct 10 00:14:40 hpc journal: Cannot open display:
Oct 10 00:14:40 hpc abrt-server: Generating core_backtrace
Oct 10 00:14:41 hpc abrt-server: Duplicate: core backtrace
Oct 10 00:14:41 hpc abrt-server: DUP_OF_DIR: /var/spool/abrt/ccpp-2019-10-09-23:26:46-6111
Oct 10 00:14:41 hpc abrt-server: Deleting problem directory ccpp-2019-10-10-00:14:32-11309 (dup of ccpp-2019-10-09-23:26:46-6 111)
Oct 10 00:14:41 hpc abrt-applet.desktop: abrt-applet: repeated problem in tigervnc-server-minimal-1.8.0-17.el7, not showing t he notification
Oct 10 00:14:41 hpc abrt-server: Email address of sender was not specified. Would you like to do so now? If not, 'user@localh ost' is to be used [y/N]
Oct 10 00:14:41 hpc abrt-server: Email address of receiver was not specified. Would you like to do so now? If not, 'root@loca lhost' is to be used [y/N]
Oct 10 00:14:41 hpc abrt-server: Sending an email...
Oct 10 00:14:41 hpc abrt-server: Sending a notification email to: root@localhost
Oct 10 00:14:41 hpc abrt-server: Email was sent to: root@localhost

Can someone explain is that really a bug or there is any workaround for that?

For freerdp I can only suggest you try a newer build.
But since it seems to trigger a bug in xvnc (or the x11 server?) it may help searching for related issues / ask the respective projects, maybe they already know something.

any update on this?

Current position: Trying to build from git version
https://github.com/FreeRDP/FreeRDP/issues/5654

Ok, closing as 'nothing we can do' (we're not xvnc or X11 devs)
Hope you had some luck in tracking down someone who can help more.

I stuck at building freerdp from source. I posted on mailing list and got nothing.
That means I can not test the dev branch at the moment.

Did you figure out a solution?  I'm having the exact same issue (CentOS 7, tigerVNC, FreeRDP).  I've tried rebuilding FreeRDP, but I still have segfaults when closing FreeRDP in a VNC session.

@CJRMoore try to update vnc, it is the vnc server crashing, not freerdp. (Assuming you have the same issue as reported here)

@CJRMoore
I wasn't able to built from source. Still I am facing that issue.


asterisk: 13.6.0 -> 13.13.1

Motivation for this change
asterisk was marked broken in 655017d. This version is the latest release on the same LTS branch and does not have that CVE.
(cc: @grahamc)
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] Linux

[x] Tested execution of all binary files (usually in ./result/bin/) (some of them depend on asterisk being in the $PATH)
[x] Tested usage on my server. It didn't get worse than it was before.
[x] Fits CONTRIBUTING.md.

👍 I'll merge in the AM. @fpletz this is being merged to 16.09 directly, as unstable has a newer version of asterisk.

Note: someone else could merge this too, but I'm not merging now because I haven't reviewed the code.

Thank you!


remove filter from complex file

I think this info (Complex filters), is now in a config  file, but I can't remember where.
Id like to remove
proteasome complex (GO:0000502)
from the filtered complexes.
rationale, for  the record
This is because at the moment the  sub parts alpha, beta, lid and  base all float around independanty in network figures. Since they are always a functional unit they should  be grouped. I tried to track down physical interactions to group them, but none were clearly direct  (co- affinity pull down).

If you mean you want GO:0000502 to be included in the downloadable complex file, done. (If not, I'd need more clues.)

Ah it used to be a filter of complex terms to omit. Has that changed to a list of included IDs instead ?
Anyway  yes I want  GO:0000502 to be included in the complex file. Its a bit of redundancy but it will look better in the networks.

No, the config still lists IDs to exclude. My question was meant to clarify the desired outcome.

I think this info (Complex filters), is now in a config file, but I can't remember where.

GO:0000502 is in this file: supporting_files/GO_terms_excluded_from_pombase.txt
I think that's the only config file it appears in.  Is that what you were after?

no Midori sorted it. This used to be in a complex filter file, but at some point the way we do this changed. It's now done another way in yaml. Now sorted.

I'll keep this open until GO:0000502 appear here:
ftp://ftp.pombase.org/pombe/annotations/Gene_ontology/GO_complexes/Complex_annotation.tsv
and here
http://www.esyn.org//builder.php?type=Graph&term=GO:0000070&interactionType=physical&source=pombase&includeInteractors=false

GO:0000502 is in this file: supporting_files/GO_terms_excluded_from_pombase.txt

It also used to be in the part of pombase_v2_config.json that lists GO CC terms to be excluded from the downloadable macromolecular complex file, until I deleted it for this very ticket.

Now I am confused.
Do we still use
GO:0000502 is in this file: supporting_files/GO_terms_excluded_from_pombase.txt
I thought it was replaced by some yaml. But this is what I want to check, if everything resolves as expected.

Do we still use ... supporting_files/GO_terms_excluded_from_pombase.txt

Yes, but for a different purpose - this file is for terms that we don't want to use for direct annotation (but aren't tagged in the ontology for whatever reason). Any annotations to terms on this list get reported in the excluded_go_terms_softcheck log.
Filtering for the complex file (ftp://ftp.pombase.org/pombe/annotations/Gene_ontology/GO_complexes/Complex_annotation.tsv) uses the pombase_v2_config.json file. On Monday I removed GO:0000502 from that config file, so genes annotated to it by transitivity will be associated with it in the complex_annotation file.

Yes ignore. I wasn't reading it properly. Different file.
I'm just waiting for the changes to filter through.

This didn't filter through to the esyN networks yet. Need to figure where the block is.

GO:0000502
isn't in
ftp://ftp.pombase.org/pombe/annotations/Gene_ontology/GO_complexes/Complex_annotation.tsv
yet....

GO:0000502 isn't in
ftp://ftp.pombase.org/pombe/annotations/Gene_ontology/GO_complexes/Complex_annotation.tsv
yet....

There are some child terms of GO:0000502 in that file, like GO:0031595.  Are there some annotations directly to GO:0000502 that are missing?

There are some child terms of GO:0000502 in that file, like GO:0031595. Are there some annotations directly to GO:0000502 that are missing?

No, GO:0000502 is on our internal don't-annotate-directly list. But it appears that whatever generates Complex_annotation.tsv doesn't work the way I thought it did. It looks like what actually happens is that if a GO CC term has no direct annotations, it's not being included in the Complex_annotation.tsv file via annotations to subclasses or parts. We thought that removing GO:0000502 from the exclusion list in the pombase_v2_config.json file would get it included in the complex file, associated with the genes annotated to its descendants, but nope.
GO:0000502 is not the only affected term; GO:0000798 is an older example that is also affected. In both cases, the term is not on the exclusion list, but also is not in the Complex_annotation.tsv file. I have not investigated enough to tell whether all affected terms are in one or more do-not-annotate lists (the GO subset or our additional file), but these two examples are.
Does a term have to have direct annotations to get into Complex_annotation.tsv? Maybe it shouldn't, in light of what Val and I thought would happen when we changed the config file.
In any case, could you clarify what is actually happening, and see if that is indeed as intended?

Let's chat about this tomorrow cos I can't remember what we do.... I put it on the list...

The Complex_annotation.tsv file contains only direct annotations.  It includes all annotations for GO:0032991 and descendants except for those configured as "excluded_terms" in the config file.  Duplicates are removed before writing the file.
   "file_exports": {
     "macromolecular_complexes": {
       "parent_complex_termid": "GO:0032991",
       "excluded_terms": [
         "GO:0099023", "GO:0022624", "GO:0005834", "GO:0000153", "GO:0000835",
         "GO:1990904", "GO:0032806", "GO:0000307", "GO:0016461", "GO:1990421",
         "GO:0008023", "GO:0017102", "GO:0044796", "GO:0032777", "GO:0033100",
         ...

Ah OK, then this data won't get  into the file. It would be a bit overkill to annotate everything additionally to GO:0000502
I will go back to my original plan and try to connect the subcomplexes together using a direct physical interaction. I tried this way first of all, but I could not find any suitable data. Hopefully there is a high-quality binary interaction somewhere....


Enhancement (js): Reduce lag between books of the same collection

Resolves #61

Prefer https://github.com/touchtypie/touchtypie/pull/70


Feature: Ability to choose different transform heuristics for columns in continuous values component

We want to allow users to select different types of data transformation heuristics (from those available) for the columns of a continuous values category (i.e. 'Age').

UI changes must be made in the in annot-continuous-values component for this to occur. The suggested solution is a mult-tabbed continuous values component where each tab represents a column and where within each tab there is a dropdown for the column's transform heuristic – somewhat similar to the current implementation that selects a heuristic for all columns.
Once the choice is made a mutation must be written to alter the annotated data dictionary to register the newly selected heuristic for its respective column.

Tied with #270 and #274

@surchs Does this sound about right to you?

Makes sense to me @jarmoza. What I would add is that both the "user-selected-a-heuristic-for-this-column-please-store" mutation and the "what-heuristic-is-currently-stored-for-this-column" getter have to be updated for this to work. And, as we discussed: the choice of available heuristics (i.e. a list of possible string names of heuristics the user can choose from) is still going to be tied to the category.

@jarmoza can we close this?

@surchs No, this is the new feature that will be my upcoming PR.

Closed by #372


Fixed country_code validation to properly handle strings

Fixes Or Enhances
This PR fixes #860. I have modified existing tests and also added a test for country_code to ensure that my modifications have fixed the issue and improve upon the validation.

[x] Tests exist or have been written that cover this particular change.

@go-playground/validator-maintainers

Coverage decreased (-0.004%) to 75.096% when pulling da1902de14302605e7c982eafe14a73816177275 on uberswe:country_code into ce34f361cca51e25715399ab801c5e1c01d02d89 on go-playground:master.

Coverage decreased (-0.004%) to 75.096% when pulling da1902de14302605e7c982eafe14a73816177275 on uberswe:country_code into ce34f361cca51e25715399ab801c5e1c01d02d89 on go-playground:master.


[FEATURE REQUEST] Add position animation blending

Original animation (Using Yes Steve Model (GeckoLib core)):
https://github.com/GrandpaScout/GSAnimBlend/assets/95112579/3e0d1aa0-e79e-43ad-97ed-34324aff8def
Using Figura:
animations.main["sprint"]:blendTime(2.5)

https://github.com/GrandpaScout/GSAnimBlend/assets/95112579/67bc74ad-c57f-47e6-9a2e-8e77139427ae

Animation blending accounts for everything an animation can do already. It uses the built-in :setBlend() method created by Figura to handle blending in and out. This seems more like an issue with the animation itself.


DRILL-4583: Test fails on TestNewDateFunctions#testIsDate

DRILL-4583: Test fails on TestNewDateFunctions#testIsDate

changed constructor parameters of joda datetime

This issue was already addressed in https://github.com/apache/drill/pull/1247. So closing this.


create_token_perm permission might be a vulnerability

This permission allows creating tokens with any other permissions.
Possible solutions:

Remove the permission entirely, use master token or admin permission instead.
Only allow creating tokens with permissions that are enabled for the token that created it.

Removed the permission entirely


Added the possibility to use jshint reporter without running jshint

I love this little plugin, but found something missing for our use case.
I have added a little method for the case you want to run jscs without running jshint but still want to use jshint reporters.

I don't see why we'd need an extra method here. Wouldn't it be possible to you use the combineWithHintResults method if this was added at the beginning of the method body?
file.jshint = file.jshint || {};

Well, sure, it could, just the name combineWithHintResults, suppose you have hint results already. This is why I made a separate method.
But either way I am fine with adding it in the existing method, as long as it's possible to do, I'm happy :)

You're right about the naming - i'm still not convinced adding another method only for the purpose of naming is a good idea. Thanks for the updated pull request, will merge this today.

Published to npm as 1.1.0.
Thanks for your efforts!


Difference between realtime and realtime-js documentation ... token on connect

Bug report
In realtime documentation it shows:
var channel = socket.channel('realtime:public:rls_table', { user_token: JWT })
In realtime-js documentation it shows:
var databaseChanges = client.channel('realtime:*')
And there is no discussion of providing a user_token.
I don't use realtime-js directly, but if it requires the user_token then documentation should reflect that.
Expected behavior
Documentation should reflect the correct way of doing things in both repositories.

@GaryAustin1 Thanks! Will update realtime-js docs.

I was just working with a user in Discord getting deletes and not inserts/updates using realtime-js directly.   Went thru all the normal stuff, back and forth as looks like an RLS issue, finally suggested they contact support as out of ideas...  Then I dug deeper and found this, which they did not have.  Let them know, but they are offline now...  Not sure if that is their issue or not...

@GaryAustin1 Thanks for helping said user. I'm currently debugging with the user and I think you may be right about the token.

@GaryAustin1 That was an odd one. User created a new project and everything is fully working. There was nothing out of the ordinary so it was quite strange it wasn't working before. Anyways, appreciate you helping out the user!

Hate those... Need to be watching for something a user can do to grants/permissions or such.  realtime schema mainly has the subscription table which seems to go empty between connections and certainly server resets, which the user said they did.  That leaves migrations, but I assume you can see those.  I guess something strange could be going on with replications getting out of whack on tables...


editor.bracketPairColorization.enabled working not correctly in this case with html template (vue 3.2)

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: Version: 1.60.0
OS Version: MacOs 11.5.2 (mac mini m1)

Steps to Reproduce:

enable bracketPairColorization: "editor.bracketPairColorization.enabled": true
In template vue

This also occurs in rust in situations where you create an arrow => (e.g. match statements):
match an_optional {
    Some(inner) => node = inner,
    _ => break,
}

There is an error with the colorization like in Ruby string interpolation where the opening curly bracket is attach to the number sign (#). The opening curly bracket does not get the colorization but the closing bracket get it

Html too?

I am experiencing the same issue.
For most languages it doesn't even makes sense to colorize angled brackets... 🤦♂
I think this feature request would solve this problem! https://github.com/microsoft/vscode/issues/132336

Every language extension can configure which brackets should be highlighted.
By default, the language brackets are highlighted.
If there colorizedBracketPairs propery, only those bracket pairs are highlighted.
The documentation for this new property is still missing.
Please reach out to the extension authors to configure this propery, if the default has issues!
@HoTKreeZy The underlying problem is this. Unfortunately, this seems to be a restriction by chrome/electron (and even firefox). If you disable ligatures, it should work.

I'm a little confused. this was not happening with the bracket pair colorizer extension. even if the extension had a per-language config, it has been working out of the box for me just fine without having to touch such a config.

this was not happening with the bracket pair colorizer extension

Because the bracket pair colorizer extension explicitly does not colorize < and > - this exception is baked into the extension and even languages that want to colorize these brackets cannot do so.
We introduced a per language config (rather than assuming that < ... > is the only problematic pair and not colorizing it for any language).
If a language does not configure which brackets to colorize, VS Code colorizes all configured brackets. If you think we should, by default, remove <...> from this list when using it as default for colorized bracket pairs, please open a separate issue. While it would fix the issues mentioned here, it would only do so because <...> should not be colorized in most languages. That's why I'm unsure if this is a good default.

@hediet Not really a good default to have angled brackets < > colorized in all languages, it probably generates more problems than it solves. I opened a new issue for this #132476


Add webpack profiling task

webpack --progress -c --profile --json > stats.json
-> can be uploaded to get a nice dependency tree
add documentation about how to use the profiling

closed by ce09c3c7948a0c09ef39e4b730ccc202a2427e92


Revert "Bump selenium-webdriver from 4.14.0 to 4.15.0"

Reverts alphagov/forms-runner#477
The selenium-webdriver update broke our deployments, so reverting to unblock the pipeline.

This wasn't actually the cause of the pipeline breaking, so this isn't necessary any more


Parcel fixes

These don't quite fix  the end-to-end tests hanging in CI, but they do get the MacOS runs to succeed again - before, they'd crash due to error messages thrown onto the page by Parcel.

@NSeydoux I added a5494b9c4887e3f17403a5d405cfaf61c36f6f85 to rely on just the MacOS tests for now, could you take a look whether that makes sense? I'll add a ticket to investigate this in the future.

The change makes sense to me, even if it's still unclear what triggered the failure.


Fuselage loft building fails sometimes

In certain cases the loft building routine for fuselages fails
e.g. for the file
https://github.com/DLR-SC/tigl/blob/fixIssue123/tests/TestData/fuselageBuildLoftFail.xml

The reason for this is a buggy Opencascade BRepOffsetAPI_ThruSections routine with smooth surfaces. Possible workaround is the explicit setting of C0 continuity in the first fuselage segment

Due to new lofting, this is no longer an issue.


Multi Manifest for GlassWire.GlassWire version 2.3.321.0

Closes #16256
Closes #16257
Closes #16048
Closes #14027
Closes #15686

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[X] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

nvm gotta edit more, fixing later


[BEAM-2571] CombineTest.testSlidingWindowsCombineWithContext

R: @kennknowles @JingsongLi

Run Flink ValidatesRunner

Run Flink ValidatesRunner

Coverage increased (+0.003%) to 70.684% when pulling d05246c8681ff6a58c3ed881ff9bcb01963efed9 on aljoscha:jira-2571-fix-flink-watermark-issues into 91c7d3d1f7d72e84e773c1adbffed063aefdff3b on apache:master.

Run Flink ValidatesRunner

Coverage remained the same at 70.68% when pulling 8f1df14f9b2adb012bfbe16f565bd62cee8eb802 on aljoscha:jira-2571-fix-flink-watermark-issues into 91c7d3d1f7d72e84e773c1adbffed063aefdff3b on apache:master.

Small bump, I think this is quite important because it gives us back a good signal from Jenkins.

Run Flink ValidatesRunner

@peihe I tried and addressed your comments.

Coverage decreased (-0.05%) to 70.632% when pulling 26a107ef75750323ee64c5fc85d1635173f12d53 on aljoscha:jira-2571-fix-flink-watermark-issues into 91c7d3d1f7d72e84e773c1adbffed063aefdff3b on apache:master.

@peihe I opened a rebased version of this with all fixes: #3600


Able to link to another file within the project in comment.

Add the functionality of referring to another file within the project in the comment of the source code, I believe that would do a lot of help. For exampe, when I read a config-parsing function, I want to quickly jump to the corresponding file.

I think this should be done by the extension that adds commenting. For example, this is how it could work with the GitHub Pull Request extension:

User copies a permalink to a file or line
User pastes that into a PR comment
When the comment is rendered, the GitHub Pull Request extension can re-write that permalink to be a local file link.
When the link in the comment is clicked on, the local file is opened.

@Mulf if you you're using GitHub Pull Request and Issues to make comments then I will take this as a feature request to that extension.


#194 still remain

#194
lapin.tar.gz

Ok, so there were two (rather similar) problems there, fixing the second one

Should be fixed in 0.26.4

Btw, while we sure can fix this in the client library (still digging to make sure I did it correctly), channels are not supposed to be used across multiple threads, it's recommended to create at least one channel per thread


[BUG] URL generation bug, plus WORKAROUND

WhatsApp version: any
OS: Android
Platform: any
Branch and version: 0.10.0

Describe the bug
The program makes URLs clickable as per #69 . However, there is a bug of sorts involving messages that involve new-lines inside the message, where the link includes text and
 elements.
To Reproduce
Steps to reproduce the behavior:

With a message like:
"https://example.com/webpage.html
Hey check out this link"

the generated HTML will look like:

The same if there are two-new lines, you would get html like

Workaround
The issue is caused by the
 insertions in the if "\r\n" in msg: and if "\n" in msg: sections in android_handler.py (line 446 and 461).
If you comment out those code segments to NOT add
, then the URLs are handled correctly (e.g. ). However, that is at the expense of not creating a new line.
There must be some incompatibility with the addition of the
 elements to the message and the subsequent "urlification" of the message in create_html() (the urlize() function in whatsapp.html?)

The issue should be resolved in commit f35bf24 by adding a space before the
 tag, although it is essentially an alteration to the message.

I tested this myself, and seems to work perfectly now. As for the message being slightly "altered", that is a small price to pay to have the message show up correctly!

The issue should be resolved in commit f35bf24 by adding a space before the
 tag, although it is essentially an alteration to the message.


Waveforms in Chart Editor is not showing properly for mono sounds

Describe your bug here.
https://user-images.githubusercontent.com/78720179/159449189-51ba79fd-6b25-4317-a354-9c1707911100.mp4
^
Command Prompt/Terminal logs (if existing)
No response
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows
Did you edit anything in this build? If so, mention or summarize your changes.
No

bro you activated the waveform for the instrumental

bro you activated the waveform for the instrumental

Yes i did, and im supposed to chart the instrumental, and because the instrumental audio is mono (not stereo), it doesnt display properly

make it stereo then


Bad packet handling

This is an outline for an approach to #269.
It's still incomplete, and needs polish, but shows the direction I had in mind. I'd appreciate thoughts.

general approach looks good to me, main thing is to remove any panic calls from my side

general approach looks good to me, main thing is to remove any panic calls from my side

Great, thanks. I'll cover some more cases that should also go into the "Unknown but ok" bucket, and will clean up panics after.

CI complains about code in src/armor/reader.rs (see https://github.com/rpgp/rpgp/actions/runs/7702519672/job/20990984070?pr=271):
error: read amount is not handled
   --> src/armor/reader.rs:391:13
    |
391 |             base_decoder.read(into)?
    |             ^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: use `Read::read_exact` instead, or handle partial reads
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unused_io_amount

Looks like a regression in clippy. Probably this one: https://github.com/rust-lang/rust-clippy/issues/12208

I've processed your feedback (note that I wrote two replies that are still open, we might want to continue discussing those two threads).
I've made one more substantial change to the logic of this RP: Two more error types are now considered "soft packet parser failures" (see 5491bda201f66e0bce8f3b134d940c6b5e7d97ca).
I think these errors happen with "weird packets" that can be safely skipped, and can't imagine a case where it's necessary to hard-error on them.
With these changes, CI is green again, after bumping the numbers of expected parsable packets (see 807bba718422f80f873145bf4e2f8eea2a07d916).
I have not verified which exact packets can now be parsed, but the increase in numbers feels plausible, and I don't know that it's worth investigating more deeply.
I was surprised that for test_parse_dumps_0 the test outcome for actual_count increased by one, and wonder what might have caused this.

I also added an Other  value to CompressionAlgorithm and SymmetricKeyAlgorithm, following the same pattern

any reason this is still in draft mode?

any reason this is still in draft mode?

no good reason. i un-drafted it.

rebased, and adjusted the expected test_parse_dumps numbers

some last notes, but overall looks good, thanks for working through this

Ok, I think I've done these bits of cleanup. Thanks for talking me through this :)

thanks alot for working though this


feature request

No heart_rate in gpx.
Time zone issues are not considered
Can I  raise a pr?

Yeah for sure! Do open up a PR. I didn’t use to record heart rate data in nrc so didn’t add that functionality in.

Is it okay for you to share a sample NRC activity file with heart-rate data? I don't have one myself.

You can download from my nrc json like this it contains heart rate data. https://raw.githubusercontent.com/yihong0618/blog/master/data/activities/1593858760016.json
I can take a pr maybe today or tomorrow in my time zone, Thank you.

@yasoob
You can download from my nrc json like below it contains heart rate data. https://raw.githubusercontent.com/yihong0618/blog/master/data/activities/1593858760016.json
I can take a pr maybe today or tomorrow in my time zone, Thank you.

Thanks!

time zone fix https://github.com/yasoob/nrc-exporter/pull/4

I already merged that in :)

@yasoob add heart rate https://github.com/yasoob/nrc-exporter/pull/5
But I don't know which lint do you prefer, black or pep8 or pylint, I didn't do the lint.


NetManager.PeersCount can reach negative number

as the title said, NetManager.PeersCount can reach negative number on version 0.8, it didn't happens in <IP_ADDRESS>

@davinpra possibly fixed. Hard to reproduce but found some bad places.


Can't Get Data On 'Frame' Event

Per my comment here, I am not able to get data on the frame event.
Code to reproduce this issue (on Raspberry Pi running node js)
const { StreamCamera, Codec } = require("pi-camera-connect");

const streamCamera = new StreamCamera({
  codec: Codec.MJPEG
});

const videoStream = streamCamera.createStream();

await streamCamera.startCapture();

videoStream.on("frame", data => console.log("New frame", data));

// NOTHING IN CONSOLE?

// Wait 5 seconds
await new Promise(resolve => setTimeout(() => resolve(), 5000));

await streamCamera.stopCapture();

Thanks

Added comment to existing frame issue


Add RemoteDepth to llamahub

Needed to build a reader where we download all the links available in a webpage (also links inside links).
Difficult to write tests without network requests.
Not sure if I should have extended the reader RemoteReader, modify existing  RemoteReader, or just create a new class from BaseClass.

Still on the 24 from PST!

Yep you're in the pool :)


[SMALL] Update In-App CORS instructions to match README

Currently, the "API Not Connected" Error does not give the complete list of origins that need to be added to the IPFS settings according to the README.
This PR brings the In-App CORS instructions up to par with the README.
Haven't tested.

@rafaelramalho19, can you please have a look?

Tested it out and lgtm, thank you @Evanito

👍


Add filtration to API and CLI

Should be able to filter by rarity and tags :-)

@piatrashkakanstantinass is assigned to this issue

Fixed in #62


Out shows blank

I get the following blank output. what is the possible reason?

This could be due to many things. We have to troubleshoot. Are you able to get back facial landmarks with other images? If not, then there may be an issue with the installation or version of DLIB.

ya checked with other images . Output is same

ya checked with other images ,the results are same

Can you please print what is the output of lines 29 and 30 from the example_frontalize_dlib_singleimage.py when you try with an image? For these 2 lines of code:
landmarks_raw = predictor(image, face)  # detect landmarks
landmarks = get_landmark_array(landmarks_raw)
specifically, check what is the "landmarks_raw" and "landmarks" variables. What type and what values they have. This will help us understand what's going on...


[eslint] enable linter rules for README js blocks

Enabled linting README.md for the following sdk:

[X] @azure/ai-anomaly-detector - sdk/anomalydetector/ai-anomaly-detector
[X] @azure/app-configuration - sdk/appconfiguration/app-configuration
[ ] @azure/attestation - sdk/attestation/attestation
[X] @azure/communication-chat - sdk/communication/communication-chat
[X] @azure/communication-common - sdk/communication/communication-common
[X] @azure/communication-identity - sdk/communication/communication-identity
[X] @azure/communication-network-traversal - sdk/communication/communication-network-traversal
[X] @azure/communication-phone-numbers - sdk/communication/communication-phone-numbers
[X] @azure-tools/communication-short-codes - sdk/communication/communication-short-codes
[X] @azure/communication-sms - sdk/communication/communication-sms
[X] @azure/container-registry - sdk/containerregistry/container-registry
[ ] @azure/cosmos - sdk/cosmosdb/cosmos
[X] @azure/digital-twins-core - sdk/digitaltwins/digital-twins-core
[ ] @azure/dtdl-parser - sdk/digitaltwins/dtdl-parser
[X] @azure/eventgrid - sdk/eventgrid/eventgrid
[X] @azure/event-hubs - sdk/eventhub/event-hubs
[ ] @azure/eventhubs-checkpointstore-blob - sdk/eventhub/eventhubs-checkpointstore-blob
[ ] @azure/eventhubs-checkpointstore-table - sdk/eventhub/eventhubs-checkpointstore-table
[X] @azure/ai-form-recognizer - sdk/formrecognizer/ai-form-recognizer
[ ] @azure/identity - sdk/identity/identity
[ ] @azure/identity-cache-persistence - sdk/identity/identity-cache-persistence
[ ] @azure/identity-vscode - sdk/identity/identity-vscode
[X] @azure/opentelemetry-instrumentation-azure-sdk - sdk/instrumentation/opentelemetry-instrumentation-azure-sdk
[ ] @azure/iot-modelsrepository - sdk/iot/iot-modelsrepository
[ ] @azure/keyvault-admin - sdk/keyvault/keyvault-admin
[ ] @azure/keyvault-certificates - sdk/keyvault/keyvault-certificates
[ ] @azure/keyvault-keys - sdk/keyvault/keyvault-keys
[ ] @azure/keyvault-secrets - sdk/keyvault/keyvault-secrets
[ ] @azure/ai-metrics-advisor - sdk/metricsadvisor/ai-metrics-advisor
[X] @azure/mixed-reality-authentication - sdk/mixedreality/mixed-reality-authentication
[ ] @azure/monitor-opentelemetry-exporter - sdk/monitor/monitor-opentelemetry-exporter
[ ] @azure/monitor-query - sdk/monitor/monitor-query
[X] @azure/quantum-jobs - sdk/quantum/quantum-jobs
[X] @azure/mixed-reality-remote-rendering - sdk/remoterendering/mixed-reality-remote-rendering
[X] @azure/schema-registry - sdk/schemaregistry/schema-registry
[ ] @azure/schema-registry-avro - sdk/schemaregistry/schema-registry-avro
[ ] @azure/search-documents - sdk/search/search-documents
[X] @azure/service-bus - sdk/servicebus/service-bus
[X] @azure/storage-blob - sdk/storage/storage-blob
[X] @azure/storage-blob-changefeed - sdk/storage/storage-blob-changefeed
[X] @azure/storage-file-datalake - sdk/storage/storage-file-datalake
[X] @azure/storage-file-share - sdk/storage/storage-file-share
[X] @azure/storage-queue - sdk/storage/storage-queue
[X] @azure/data-tables - sdk/tables/data-tables
[X] @azure/template - sdk/template/template
[X] @azure/ai-text-analytics - sdk/textanalytics/ai-text-analytics
[X] @azure/video-analyzer-edge - sdk/videoanalyzer/video-analyzer-edge
[X] @azure/web-pubsub - sdk/web-pubsub/web-pubsub
[X] @azure/web-pubsub-express - sdk/web-pubsub/web-pubsub-express

Skipped packages:

[X] @azure/iot-device-update - sdk/deviceupdate/iot-device-update
[X] @azure/synapse-access-control - sdk/synapse/synapse-access-control
[X] @azure/synapse-artifacts - sdk/synapse/synapse-artifacts
[X] @azure/synapse-managed-private-endpoints - sdk/synapse/synapse-managed-private-endpoints
[X] @azure/synapse-monitoring - sdk/synapse/synapse-monitoring
[X] @azure/synapse-spark - sdk/synapse/synapse-spark

@jeremymeng, this is my new PR identical to #20903

@jeremymeng, this is my new PR identical to #20903

Looks good to me!  Could you please make similar updates to other packages too in this PR?

@jeremymeng, some of the packages (e.g. synapse-artifacts) are using "lint": "echo skipped". How should I deal with it?

@WeiJun428 good question!  Let's skip packages that currently don't have the linter enabled.

@jeremymeng, it seems like the current eslint config will throw an error when there exists file of type *.ts, *.tsx, *.json in README.md:
Parsing error: "parserOptions.project" has been set for @typescript-eslint/parser.
The file does not match your project config: README.md\15_15.ts.
The file must be included in at least one of the projects provided

These file types are exactly what I include in the override of azure-sdk-base.ts. Can I get some help on this? (most of the packages left unresolved are associated with this same issue)

@jeremymeng, I have compiled a list of import errors besides logger here. (only for those sdk without filetype issue mentioned above)
One error:
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/schemaregistry/schema-registry-avro/README.md#L77
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/metricsadvisor/ai-metrics-advisor/README.md#L108
Two errors:
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/eventhub/eventhubs-checkpointstore-table/README.md#L75-L76
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/eventhub/eventhubs-checkpointstore-blob/README.md#L73-L74
Multiple errors:
@azure/identity-vscode
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/identity/identity-vscode/README.md#L10-L11
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/identity/identity-vscode/README.md#L43-L44
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/identity/identity-vscode/README.md#L62-L63
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/identity/identity-vscode/README.md#L86-L87
@azure/identity-cache-persistence
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/identity/identity-cache-persistence/README.md#L10-L11
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/identity/identity-cache-persistence/README.md#L42-L43
https://github.com/Azure/azure-sdk-for-js/blob/d9e5bfb5fc865a03b861ec9fd88e06fcabd42ad8/sdk/identity/identity-cache-persistence/README.md#L55-L56

@WeiJun428 Thank you very much for the list?  Can these import {...} from "..." errors be fixed by changing them to const { ... } = require("...")?

@jeremymeng I believe so. Now, I am just unsure if a direct translation from import to require will work. For example, in the case of logger, we change import { setLogLevel } from "@azure/logger"; to const { logger } = require("@azure/logger"); instead of const { setLogLevel } = require("@azure/logger");

@WeiJun428 Sorry it looks like the fix is incorrect.  We should have a direct transform for the logger one too.  I apologize for the error in the issue.

@jeremymeng I remember that some of the repo (not fixed by us) are also using the syntax const { logger } = require("@azure/logger"); Would that be an issue?

@jeremymeng I remember that some of the README.md (not fixed by us) are also using the syntax const { logger } = require("@azure/logger"); That is why in my first PR for this issue, I fixed the text analytics using const { logger } = require("@azure/logger"); Would that be an issue?

I'm a little confused, since the object destructuring doesn't make sense to me, I think it should either destructure the specific exports or alias the entire module to an identifier e.g. const logger = require("@azure/logger"):

@WeiJun428 it's correct if the code block requires logger then use logger.  Iike
const logger = require("@azure/logger")
logger.setLogLevel("info");

Most of direct transform should be like:
const { setLogLevel } = require("@azure/logger")
setLogLevel("info");

Closing this as we discussed offline. We will target a new branch that does not trigger build storms.

@WeiJun428 I created a new branch https://github.com/Azure/azure-sdk-for-js/tree/jeremymeng/enable-linting-readme which should be same as feature/enable-linting-readme.  Please create a new PR targeting that branch.  Thanks!

Sure! @jeremymeng


update README.md to fix #1336

This change is

Skipping PR - No APP related changes

@yaronyg please review

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Border radius?

Why don't you allow changing border radius? Is that super hard to add?

Nevermind, I have just reduced the font size and got needed border radius


Group tags issue

Hello,
I am using the latest version 2.1.1. At first I thought I did something wrong, but after doing a separate test it seems like a bug in the save function. Group tags are not written into the output file.
import {default as kdbxweb} from 'kdbxweb';
import { writeFileSync } from 'fs';

const credentials = new kdbxweb.Credentials(kdbxweb.ProtectedValue.fromString('test'));

const db = kdbxweb.Kdbx.create(credentials, 'title');

const group = db.createGroup(db.getDefaultGroup(), 'Tagged Group');
// will not be saved in groups
group.tags = ['tag1', 'tag2'];

const entry = db.createEntry(group, 'Tagged Entry');
// will be saved in entries
entry.tags = ['tag1', 'tag2'];

const data = await db.saveXml();
writeFileSync('test.xml', data);

Here is the XML output of the KDBX file I get:
https://gist.github.com/mstoecklein/e2686de4c22a263240e75db6745df6ac

new databases default to kdbx v4.0 and no automatic upgrade to v4.1 occurs when you use v4.1 features.
Thus, you should be able to fix a problem like this by setting the header minor version to 1 after creation:
db.header.versionMinor = 0;


Error with packets

I have a simple Gate proxy set up with 3 backend servers the ports are all configured correctly also what do I have to enable on the backend servers to work in proxy mode is it bungeecord or like velocity with the forwarding secret at the moment i have enabled bungeecord on the backend servers and the forwarding mode is set to legacy. When i then try to join it gives me an error on the backend server like this
[10:29:33 INFO]: UUID of player PandaDEV_ is 2dae5251-257a-4d28-b220-60fe24de72f0
[10:29:33 INFO]: [+] PandaDEV_ • Nextron founder
[10:29:33 INFO]: PandaDEV_[/<IP_ADDRESS>:36086] logged in with entity id 59 at ([world]-1382.516426950758, 18.0, -1477.6716488100947)
[10:29:33 INFO]: PandaDEV_ lost connection: Internal Exception: io.netty.handler.codec.DecoderException: java.io.IOException: Packet 0/23 (d) was larger than I expected, found 150 bytes extra whilst reading packet 23
[10:29:33 INFO]: [voicechat] Disconnecting client PandaDEV_

enable proxy protocol. see links https://gate.minekube.com/guide/lite.html#proxy-behind-proxy

let's continue in discord if you need more help :)


WTERMSIG (and others) are unsafe on macOS, but safe on Linux

WTERMSIG and related functions have been changed to no longer require unsafe, seemingly in commit 5a1df22a783ee0348a4c862e36480a50829a95f5. However on macOS these functions are still unsafe. See the definition on Linux https://docs.rs/libc/0.2.77/libc/fn.WTERMSIG.html (no unsafe) vs. macOS https://docs.rs/libc/0.2.77/x86_64-apple-darwin/libc/fn.WTERMSIG.html (requires unsafe). Was it intentional to leave macOS out?
Also this causes warnings, e.g. see https://github.com/rust-lang/rust/pull/76561/checks?check_run_id=1096958697#step:23:213 (https://github.com/rust-lang/rust/commit/7c3e1ffd7a945b4044e5e80d7d74ba944ff54d0f/checks/1096958697/logs).
/cc @joshtriplett who wrote the commit.

It's not that I intentionally left macOS out, I just didn't go changing any other targets systematically in that first commit. Now that the infrastructure is in place, by all means we should update various other functions to use it, including on other targets.
And yes, I'm not surprised that this resulted in "unused unsafe" warnings. We should prioritize adding this to matching functions for other targets, to avoid having problems where one target wants unsafe and another doesn't.

https://github.com/rust-lang/libc/commit/773f5562ba04ad8b7ba772e99645922ea908be76 should fix this on all platforms. The fix should be available in the next release of the libc crate.

@joshtriplett @Thomasdezeeuw rustc's Cargo.lock pins to 0.2.85 which was released on Feb. 2, 2021. The unsafe blocks & warning suppressions are still present in the latest rustc, are they still necessary?

@xmo-odoo @joshtriplett did that in https://github.com/rust-lang/rust/commit/16ebf750cfbc3de36cbf2e05eb50669d83c4dbfd.

@Thomasdezeeuw

did that in rust-lang/rust@16ebf75.

Uh.
Sorry about that.


Images from imgur-gallery not sorted correctly when viewing in feh

When opening a imgur-gallery in feh, the images are not saved in the same sequence as in the gallery.
Most probably because images are saved with the same name as in the gallery (seven random letters and numbers).
Tested with imgur-gallery in this submission.
I'm really not sure how the images are processed or saved, but if they are saved on disk, the filenames should be appended with a number first: https://i.imgur.com/KXrAJMx.jpg -> 01 - KXrAJMx.jpg

What's your mailcap entry look like? It appears to preserve the order of images for me using the default example:
image/x-imgur-album; feh -g 640x480 %s; test=test -n "$DISPLAY"

Ooooh, sorry, brainfart here:
I forgot I had changed the viewer to eog, so I had this setting:
image/x-imgur-album; eog %s; test=test -n "$DISPLAY"
When testing with the default setting with feh, it works like a charm.
As I was trying to use existing apps, any tips on how to make eog sort the images?

Looks to be a long-standing limitation with eog
https://www.bountysource.com/issues/1075428-eog-doesn-t-respect-the-way-nautilus-has-ordered-the-pictures
You could probably get creative with curl+xargs and write a command or even a small script that fetches the URLs, renames them, saves them to a temp directory, and opens that directory in eog.

I might have a look at it. Where is the code for fetching imgur-galleries?

https://github.com/michael-lazar/rtv/blob/master/rtv/mime_parsers.py#L214

Cheers, I'll see what I can do.

@michael-lazar After having a look at it, I have to admit that I'm not even close to understand how this script works, since I consider myself an amateur Pythoneer. My solution was installing feh.
But thanks for the replies and the pointers.


Delete package-lock.json

This removes the global package-lock.json that was added accidentally with #4517

Thanks for noticing @JonathanWbn - I removed manually so closing this 🙏


Suggestion: Span/ReadOnlySpan / Utf8String operators for ArraySegment

There are already operators:
    public static implicit operator ReadOnlySpan<T>(T[] array);
    public static implicit operator Span<T>(T[] array);

Use case: have loaded data over a network and split (parsed) the data into tokens that you want to expose upstream with zero copies; have class:
public class Customer {
    public Utf8String Name {get;}
    public Utf8String Address {get;}
}

Here we can't store a Span<T> or a Utf8String directly as fields because of the stack-only semantics; but we could store the array-segments:
public class Customer {
    private ArraySegment<byte> _name, _address;
    public Utf8String Name => new Utf8String(new ReadOnlySpan<byte>(_name.Array, _name.Offset, _name.Count));
    public Utf8String Address => new Utf8String(new ReadOnlySpan<byte>(_address.Array, _address.Offset, _address.Count));
}

With the addition of:
    public static implicit operator ReadOnlySpan<T>(ArraySegment<T> segment);
    public static implicit operator Span<T>(ArraySegment<T> segment);

this becomes much more convenient:
public class Customer {
    private ArraySegment<byte> _name, _address;
    public Utf8String Name => new Utf8String(_name);
    public Utf8String Address => new Utf8String(_address);
}

This pairs quite nicely with the TryGetArrayElseGetPointer method which already exposes ArraySegment<byte> as part of the API surface.

Although it also prompts the question: should Utf8String also have operators for this type of scenario, for example:
public class Customer {
    private ArraySegment<byte> _name, _address;
    public Utf8String Name => _name;
    public Utf8String Address => _address;
}

via:
public static implicit operator Utf8String(ArraySegment<byte> segment);

That might be a step too far, though.

If @KrzysztofCwalina agrees with the proposed changes I would be very happy to implement this for both corefxlab and coreclr Spans.

I like the implicit cast from ArraySegment.
As to Utf8String, i would do an explicit cast as not all sequences of bytes are UTF8 strings.
All these operators might be "too cute", but I think it's fine to experiment in corfxlab to see how they feel.


Add retries to pdc-agent key signing request

This PR adds exponential backoff to the PDC client, which is used for making key signing requests to the PDC API.
closes #29
We're using the default timings,  with a 1s initial wait, which doubles every time. Default MaxRetry is 4. The backoff behaviour looks like the following:
level=debug caller=client.go:203 msg="performing request" method=POST url=https://private-datasource-connect-api-dev-us-central-01.grafana-dev.net/pdc/api/v1/sign-public-key
level=debug caller=client.go:203 msg="retrying request" request="POST https://private-datasource-connect-api-dev-us-central-01.grafana-dev.net/pdc/api/v1/sign-public-key (status: 500)" timeout=1s remaining=4
level=debug caller=client.go:203 msg="retrying request" request="POST https://private-datasource-connect-api-dev-us-central-01.grafana-dev.net/pdc/api/v1/sign-public-key (status: 500)" timeout=2s remaining=3
level=debug caller=client.go:203 msg="retrying request" request="POST https://private-datasource-connect-api-dev-us-central-01.grafana-dev.net/pdc/api/v1/sign-public-key (status: 500)" timeout=4s remaining=2
level=debug caller=client.go:203 msg="retrying request" request="POST https://private-datasource-connect-api-dev-us-central-01.grafana-dev.net/pdc/api/v1/sign-public-key (status: 500)" timeout=8s remaining=1
level=error caller=client.go:174 msg="error making request to PDC API" err="Post \"https://private-datasource-connect-api-dev-us-central-01.grafana-dev.net/pdc/api/v1/sign-public-key\": POST https://private-datasource-connect-api-dev-us-central-01.grafana-dev.net/pdc/api/v1/sign-public-key giving up after 5 attempt(s): unexpected HTTP status 500 Internal Server Error"
level=error caller=ssh.go:102 msg="could not check or generate certificate" error="failed to generate new certificate: key signing request failed: internal error"

Nice to see this fix -- we're running on an EC2 using latest PDC Docker image and getting
Connection to private-datasource-connect-prod-us-east-0.grafana.net closed by remote host.
Transferred: sent 12127452, received 4256284 bytes, in 239717.9 seconds
Bytes per second: sent 50.6, received 17.8
debug1: Exit status -1
level=error caller=ssh.go:122 msg="ssh client exited. restarting"
level=info caller=keymanager.go:52 msg="starting key manager"
level=info caller=keymanager.go:156 msg="new certificate required: certificate validity has expired"
level=info caller=keymanager.go:204 msg="generating new certificate"
level=error caller=client.go:177 msg="unknown response from PDC API" code=502
level=error caller=ssh.go:102 msg="could not check or generate certificate" error="failed to generate new certificate: key signing request failed: internal error"

regularly. Any timeline for rolled out to docker hub so that we can pull the latest and prevent our PDC from crashing?


Review 'to-be' Diagram of the Current Live Certificate Renewal Process

User Need
As a developer
I want a diagram of the final automated gandi.net certificate renewal process
so that I can understand all the different pieces of work that need to be done
Functional Requirements:

A diagram representing the user journey for the end goal of a majority automated cert renewal process

Acceptance Criteria:

A diagram representing the user journey for the end goal of a majority automated cert renewal process

Notes:
Can build on this diagram here https://app.diagrams.net/#G1qk4Wxv1wF_qKDG3ZnkmfzM0BKN137wfO#{"pageId"%3A"xBDcULKtSUn9UIfunSXo"}

Removing for now - certificates may be moving out of Ops Eng.


Fix bug in batch update and batch delete with User Defined Table Type parameter

Fix bug: when apply batch update or batch delete on a query with User Defined Table Type parameter, an exception will occur (System.ArgumentException : No mapping exists from object type System.Object[] to a known managed provider native type.).

Thx for contrib.


how many mem does you program need to run on cpu?

RuntimeError: [enforce fail at ..\c10\core\CPUAllocator.cpp:62] data. DefaultCPUAllocator: not enough memory: you tried to allocate %dGB. Buy new RAM!27

sry for the late reply. The used GPU is just 1080Ti, adjusting the batch_size may help


The es-proxy-deploy pods periodically stop responding and do not seem to recover

Issue Description

Describe the problem

The es-proxy-deploy pods periodically stop responding and do not seem to recover

Describe the application flow or what steps were being taken when the problem occurred

The deployed system is for functional testing. There is no special setup. The product is deployed and runs fine until the error occurs.

If the issue is reproducible provide a set of instructions describing how to reproduce it, including screenshots if they might assist

The issue occurs after the system has been deployed for a period of time. It looks like it is related to recovery from an unexpected network error. Details follow.

Include any error messages, stack traces and logs associated with the error, in full, in text format (ie not a screenshot)

Symptom
On startup, The es-proxy-deploy pods monitor the ConfigMap es-staging-ibm-es-proxy-cm using a Kubernetes k8s.Client. See detail (1) below. The k8s.Client watches the ConfigMap and periodically receives an unexpected, likely network, error. See detail (2). On perhaps some class of errors the watcher then does not recover. We see a retry attempt fail with a AuthError. See (3) below. The pods remain in a running state but the container never starts and hence no longer responds to client requests.
Details

(1) ConfigMap monitoring initializes
k8sclient.go:166] Attempting to watch for changes to config map (retry set for 5 seconds) : configmap name : es-staging-ibm-es-proxy-cm
(2) The k8s.Client watching the es-staging-ibm-es-proxy-cm ConfigMap receives an error.
{"ibm_datetime":"2020-10-24T03:01:48Z","logLevel":"ERROR","module":"KafkaProxy", "ibm_messageId":"IES","message":"IES: Kube watcher status change : http2: server sent GOAWAY and closed the connection; LastStreamID=187, ErrCode=NO_ERROR, debug="""}
(3) The retry to watch receives an auth error
{"ibm_datetime":"2020-10-24T03:02:14Z","logLevel":"ERROR","module":"KafkaProxy", "ibm_messageId":"IES","message":"IES: Error creating watcher : kubernetes api: Failure 401 Unauthorized"}

Run the log collection script available at https://github.com/IBM/charts/tree/master/stable/ibm-eventstreams-dev/ibm_cloud_pak/pak_extensions/support/get-logs.sh and attach the output to this issue

For those with access, logs have been loaded and the issue discussed here:  https://ibm-security.slack.com/archives/CB777ACRM/p1603733272446300.   We can repost here if necessary.

Describe the expected behaviour

The es-proxy-deploy pods should recover from unexpected kube issues, again likely network related.

Did this used to work? If so what has changed, for example was any maintenance of the system performed?

Environment

IBM Event Streams Version: 2019.4.2
IBM Cloud Private (ICP) Version: 3.2.4, running Openshift 4.3.35
Operating system of ICP install: Red Hat Enterprise Linux release 8.2
Browser (for UI issues):

kubectl version
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.2+7279a4a", GitCommit:"7279a4a", GitTreeState:"clean", BuildDate:"2020-08-31T17:24:39Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"

oc version
Client Version: 4.3.35
Server Version: 4.3.35
Kubernetes Version: v1.16.2+7279a4a

We've also seen the issue on
oc version
Client Version: 4.4.6
Server Version: 4.3.13
Kubernetes Version: v1.16.2
kubectl version
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.2", GitCommit:"a02f27a", GitTreeState:"clean", BuildDate:"2020-04-13T12:04:13Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"

FWIW, in this case we. see the GOWAY as in (2) above, followed by in (3) not by an auth error but instead but a refused connection.

Internal issue 6422 raised to track resolution.

Issue Description
As above
Issue Resolution
On a network disconnect the es-proxy-deploy proxy ( the "kafka proxy", or "edge proxy") only attempted to reconnect only once.  If this reconnect failed, processing stopped.
The kafka proxy has been updated to retry the reconnect.
Workaround
Delete the affected es-proxy-deploy pod.  This will restart the kafka proxy and remake the network connections.
Fix details
Internal issue: 6422
Fix target : 2019.4.4

A bit of follow up. We suspect that a primary reason for the issue is a network issue between a master node hosting the apiserver and a worker hosting eventstreams. We see this snippet in apiserver logs.
kube-apiserver-14 I1113 17:01:48.751387       1 node_authorizer.go:195] NODE DENY: unknown node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get secret es-staging/es-staging-ibm-es-access-controller-sa-token-wg5cq
kube-apiserver-master1.sys-test-04.cp.fyre.ibm.com kube-apiserver-14 I1113 17:01:48.752229       1 node_authorizer.go:195] NODE DENY: unknown node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get secret es-staging/sa-es-staging
kube-apiserver-master1.sys-test-04.cp.fyre.ibm.com kube-apiserver-14 I1113 17:01:48.757223       1 node_authorizer.go:195] NODE DENY: unknown node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get secret es-staging/insights-pull-secret
 I1113 17:01:48.757308       1 node_authorizer.go:195] NODE DENY: unknown node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get secret es-staging/es-staging-ibm-es-proxy-sa-token-wqz7n
kube-apiserver-14 I1113 17:01:48.757608       1 node_authorizer.go:195] NODE DENY: unknown node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get configmap es-staging/es-staging-ibm-es-metrics-cm
kube-apiserver-master1.sys-test-04.cp.fyre.ibm.com kube-apiserver-14 I1113 17:01:48.804691       1 node_authorizer.go:195] NODE DENY: node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get unknown secret es-staging/es-staging-ibm-es-proxy-sa-dockercfg-cxxh8
kube-apiserver-master1.sys-test-04.cp.fyre.ibm.com kube-apiserver-14 I1113 17:01:48.809901       1 node_authorizer.go:195] NODE DENY: node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get unknown configmap es-staging/es-staging-ibm-es-release-cm
kube-apiserver-master0.sys-test-04.cp.fyre.ibm.com kube-apiserver-master1.sys-test-04.cp.fyre.ibm.com kube-apiserver-14 I1113 17:01:48.810770       1 node_authorizer.go:195] NODE DENY: node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get unknown configmap es-staging/kafka-setup
kube-apiserver-master0.sys-test-04.cp.fyre.ibm.com kube-apiserver-master2.sys-test-04.cp.fyre.ibm.com kube-apiserver-14I1113 17:01:48.811074       1 node_authorizer.go:195] NODE DENY: node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get unknown secret es-staging/es-staging-ibm-es-ui-sa-token-dthdx
kube-apiserver-master1.sys-test-04.cp.fyre.ibm.com kube-apiserver-14 I1113 17:01:48.845529       1 node_authorizer.go:195] NODE DENY: node "worker3.sys-test-04.cp.fyre.ibm.com" cannot get unknown secret es-staging/es-staging-ibm-es-rest-proxy-sa-dockercfg-xkwr5

The resolution here would be on the apiserver end. The es code can loop asking for a connection to access a configmap, but until the apiserver issue is rectified, it seems es will be stuck.


Synchronisation status icon is not displayed

I would like to be able to see that my Syncthing folder is syncing with an animated icon, like in most sync apps. Is that possible with this app?

It will not be animated but change the apparance from normal icon to a syncing icon. Not sure why you don't see this happen it is implemented https://github.com/xor-gate/syncthing-macosx/blob/d23dbf5a7c463dc428afef0e40fccf0e0a900297/syncthing/STApplication.m#L112-L115. The icon is here https://github.com/xor-gate/syncthing-macosx/blob/master/syncthing/Assets.xcassets/StatusIconSync.imageset/sync-monochrome.pdf.

I have done a test and it seems indeed be broken when one or more folders are syncing.

I have changed the title, probably we should read some more information of syncthing to know the status of synchronisation.

It displays the sync status for me. Is there a repro case for this?
Here is the code that determines sync state:
https://github.com/syncthing/syncthing-macos/blob/develop/syncthing/STStatusMonitor.m#L80

@tricksystudio could you check if you still have this problem with the latest release (v0.14.49-1)?

I'm sorry I haven't got it installed any more.