[Feature Request] Custom asound.conf for 7.1 USB Sound Card with 8 individual channels

Hi.
Thanks for develop this great docker :)
I want to use your container for audio playback to a specific output of a USB 7.1 Soundcard.
I have found a asound.conf that works locally
asound.conf

#########################################################################
CUSTOM ASOUND.CONF by mfd https://knx-user-forum.de/member/7968-mfd
#########################################################################
Channel splitting your UA0099 USB sound card to get up to four
independent stereo outputs (...and more )
#########################################################################
stereofront        = the FRONT labeled green jack
stereosurround    = the SURROUND labeled black jack
stereocenter    = the CENTER / BASS labeled yellow jack
stereoback        = the BACK labeled black jack
stereoall        = all stereo outputs

for up to eight mono output channels use:

monofrontl        = the FRONT labeled green jack - tip
monofrontr        = the FRONT labeled green jack - ring
monosurroundl    = the SUROUND labeled black jack - tip
monosurroundr    = the SUROUND labeled black jack - ring
monocenterl     = the CENTER / BASS labeled yellow jack    - tip
monocenterr     = the CENTER / BASS labeled yellow jack    - ring
monobackl        = the BACK labeled black jack - tip
monobackr        = the BACK labeled black jack - ring
#########################################################################
#########################################################################
labels and colors could be different for differnt vendors or
versions of the device (Logilink, Sewell, Sweex, ...)
#########################################################################
put this file into your /ect/asound.conf
#########################################################################
pcm.logilink {        # alias for our sound card
type hw            # must be hw in our case
card 1            # check your correct card number by typing
device 0        # 'aplay -l' in your terminal
}                    # the 1st usb card on the Raspi shows as
# card 1: Device [USB Sound Device], device 0: USB Audio [USB Audio]
ctl.logilink {
type hw
card 1
device 0
}
pcm.dmixer {
type dmix
ipc_key 1024            # must be unique number on your system
ipc_perm 0666            # neccesary for normal user to have access
slave.pcm "logilink"
slave {
period_time 0
period_size 1024
buffer_size 4096
rate 44100
channels 8
}
bindings {
0 0
1 1
2 2
3 3
4 4
5 5
6 6
7 7
}
}
pcm.!default {
type plug
slave.pcm "dmixer"
}
pcm.stereofront {
type plug
slave {
pcm "dmixer"
channels 8
}
ttable.0.0 1
ttable.1.1 1
}
pcm.stereocenter {
type plug
slave {
pcm "dmixer"
channels 8
}
ttable.0.2 1
ttable.1.3 1
}
pcm.stereoback {
type plug
slave {
pcm "dmixer"
channels 8
}
ttable.0.4 1
ttable.1.5 1
}
pcm.stereosurround {
type plug
slave {
pcm "dmixer"
channels 8
}
ttable.0.6 1
ttable.1.7 1
}
pcm.stereoall {
type plug
slave {
pcm "dmixer"
channels 8
}
ttable.0.0 1    # input left and right to all 4 l+r outputs
ttable.1.1 1
ttable.0.2 1
ttable.1.3 1
ttable.0.4 1
ttable.1.5 1
ttable.0.6 1
ttable.1.7 1
}
pcm.monofrontl {
type route
slave {
pcm "dmixer"
channels 8
}
ttable.0.0 1 # in-channel 0, out-channel 0, 100% volume
ttable.1.0 1 # in-channel 1, out-channel 0, 100% volume
}
pcm.monofrontr {
type route
slave {
pcm "dmixer"
channels 8
}
ttable.0.1 1
ttable.1.1 1
}
pcm.monocenterl {
type route
slave {
pcm "dmixer"
channels 8
}
ttable.0.2 1
ttable.1.2 1
}
pcm.monocenterr {
type route
slave {
pcm "dmixer"
channels 8
}
ttable.0.3 1
ttable.1.3 1
}
pcm.monobackl {
type route
slave {
pcm "dmixer"
channels 8
}
ttable.0.4 1
ttable.1.4 1
}
pcm.monobackr {
type route
slave {
pcm "dmixer"
channels 8
}
ttable.0.5 1
ttable.1.5 1
}
pcm.monosurroundl {
type route
slave {
pcm "dmixer"
channels 8
}
ttable.0.6 1
ttable.1.6 1
}
pcm.monosurroundr {
type route
slave {
pcm "dmixer"
channels 8
}
ttable.0.7 1
ttable.1.7 1
}

The asound.conf splits the 7.1 USB-Soundcard into 8 individual channels.
With
aplay -D stereofront test.wav
the first two channels playing test.wav outside the container on the host.
If the Parameter SQUEEZELITE_AUDIO_DEVICE is set to "stereofront" the container log shows
[22:17:38.919198] test_open:281 playback open error: No such file or directory
[22:17:38.919593] output_init_common:401 unable to open output device: stereofront
Outside of the container on the host, the "virtual" devices are working but inside the container the "virtual" devices can not be used.
I think this could be solved if a custom asound.conf could be used inside the container.
Or is there another chance get this working?

Hello, so you already have a working asound.conf?
If so, maybe you can just mount that file to the container: under "- volumes", add a new line which should say:
- /etc/asound.conf:/etc/asound.conf:ro
Pay attention to the indentation in the compose file.
Let me know if this helps.

Nice. Thank you 👍
This works :)
But this leads to another Problem:
I have now setup 4 Docker. Every Docker uses the working asound.conf -> works
Every Docker can play musik to a different channel of the usb-soundcard -> works
BUT if I want to play sound at the same time to differnt channels of the same usb-soundcard, only the first channel can play music.
All Containers are using the same device "/dev/snd" and this leads to the problem. The first container that want to play music is blocking the device. The other containers cant use the device anymore.
Is it possible to start multiple instances of squeezelite in one container?

Hello, good to hear that. Unfortunately it's a partial success :-(
About your question: no, and it's generally a bad idea to run more applications in the same container. Unless maybe if one is an helper to the other. Kind of defeats the purpose of containerization.
Why not just keeping track of the individual command lines, installing the squeezelite binary on the host system, and create 4 systemd services? I can help you with that if it sounds right for you.

Ok, I tested this by myself and it could work.
I have started 3 more squeezelite processes manually in the container with the same commandline except playername and outputdevice.
All 4 Stereochannels can play music individual at the same time :)
BUT how should this be configured? How can I tell the container to open 4 instances of squeezelite with different parameters?
Would it be possible to parse the variables SQUEEZELITE_NAME and SQUEEZELITE_AUDIO_DEVICE like this:
SQUEEZELITE_NAME = "Player1|Player2|Player3|Player4"
SQUEEZELITE_AUDIO_DEVICE = "stereofront|stereoback|stereosurround|stereocenter"
And run the "Start-Script" multiple times with the variables above?
Found some infos to split a string to an array and do a loop:
IFS='|' read -r -a SQUEEZELITE_NAME_ARRAY <<< "$SQUEEZELITE_NAME"
IFS='|' read -r -a SQUEEZELITE_AUDIO_DEVICE_ARRAY <<< "$SQUEEZELITE_AUDIO_DEVICE"
for index in "${!SQUEEZELITE_NAME_ARRAY[@]}"
do
    echo "${SQUEEZELITE_NAME_ARRAY[index]}"
    echo "${SQUEEZELITE_AUDIO_DEVICE_ARRAY[index]}"
``
   START instance of sqeezelite with all parameters
done

Hello, good to hear that. Unfortunately it's a partial success :-(
About your question: no, and it's generally a bad idea to run more applications in the same container. Unless maybe if one is an helper to the other. Kind of defeats the purpose of containerization. Why not just keeping track of the individual command lines, installing the squeezelite binary on the host system, and create 4 systemd services? I can help you with that if it sounds right for you.

Thanks for your input.
Yes you are right. This is a very special case which not fit into the containerization.
The Host System is an unraid-server (NAS) and I dont want to run services on the host directly.
I think the better solution for me will be to setup a VM and passthrough the USB-Soundcard.
Thanks for your help :)

Hello, about this aspect:

Would it be possible to parse the variables SQUEEZELITE_NAME and SQUEEZELITE_AUDIO_DEVICE like this:
SQUEEZELITE_NAME = "Player1|Player2|Player3|Player4"
SQUEEZELITE_AUDIO_DEVICE = "stereofront|stereoback|stereosurround|stereocenter"

That should then be done for all the other variables as well. It's also quite impractical, the compose file could easily become hard to read. However, if you were determined to do this, you could just configure the first output, then, via a script, run the other docker exec commands, using bash -c "command &" maybe with some nohup if needed.
About your solution, would you run long speaker cables (meaning you have 4 amps close to your host) or long signal cables?
The former is ok, the latter is not IMO. Long unbalanced cables behave like antennas and this would result in lots of interferences, and generally poor quality.
Considering that squeelite and lms already support synchronized playback, can you install individual devices in each room? You can even repurpose any old single board computer, or maybe some old android tv box with coreelec or armbian. The container also has regular builds for armv5 so it should run almost everywhere.
Let me know if this helps.

Last thing about your card, wouldn't "stereoall" do for you? I understand it would send the same stereo signal to all the stereo outputs.

Last thing about your card, wouldn't "stereoall" do for you? I understand it would send the same stereo signal to all the stereo outputs.

Sadly no, because I need 4 individual channels for 4 individual rooms

Hello, about this aspect:

Would it be possible to parse the variables SQUEEZELITE_NAME and SQUEEZELITE_AUDIO_DEVICE like this:
SQUEEZELITE_NAME = "Player1|Player2|Player3|Player4"
SQUEEZELITE_AUDIO_DEVICE = "stereofront|stereoback|stereosurround|stereocenter"

That should then be done for all the other variables as well. It's also quite impractical, the compose file could easily become hard to read. However, if you were determined to do this, you could just configure the first output, then, via a script, run the other docker exec commands, using bash -c "command &" maybe with some nohup if needed.
About your solution, would you run long speaker cables (meaning you have 4 amps close to your host) or long signal cables? The former is ok, the latter is not IMO. Long unbalanced cables behave like antennas and this would result in lots of interferences, and generally poor quality.
Considering that squeezelite and lms already support synchronized playback, can you install individual devices in each room? You can even repurpose any old single board computer, or maybe some old android tv box with coreelec or armbian. We also regular builds for the armv5 platform so this image should run almost everywhere.
Let me know if this helps.

I think I will give it a try with a modified command line.
In my house are speaker-cables in every room which leads to the basement into one central place.
The longest cable is 7m. The cables are already into the walls. Hopefully this will work as aspected.
I have 12 Rooms so I need 12 individual outputs. For this I have 3x 7.1-Usb-Sound-cards

In my house are speaker-cables in every room which leads to the basement into one central place.
The longest cable is 7m. The cables are already into the walls. Hopefully this will work as aspected.
I have 12 Rooms so I need 12 individual outputs. For this I have 3x 7.1-Usb-Sound-cards

Ok, sorry if I mentioned the potential issue, of course you know it already :-)
Yours must be a nice house, congrats!

Last thing about your card, wouldn't "stereoall" do for you? I understand it would send the same stereo signal to all the stereo outputs.

Sadly no, because I need 4 individual channels for 4 individual rooms

I wrongly assumed that you wanted to play the same music to multiple locations sorry


Add support for hard deleting messages

fixes #132

Codecov Report

Merging #133 (2704651) into master (f28fd1f) will increase coverage by 2.52%.
The diff coverage is 16.66%.

@@            Coverage Diff             @@
##           master     #133      +/-   ##
==========================================
+ Coverage   55.32%   57.84%   +2.52%
==========================================
  Files          14       14
  Lines         949     1122     +173
==========================================
+ Hits          525      649     +124
- Misses        331      379      +48
- Partials       93       94       +1

Impacted Files
Coverage Δ

channel_type.go
65.95% <0.00%> (+3.79%)
:arrow_up:

message.go
41.66% <0.00%> (+1.83%)
:arrow_up:

version.go
77.77% <ø> (-5.56%)
:arrow_down:

event.go
58.13% <100.00%> (+1.72%)
:arrow_up:

rate_limits.go
62.16% <0.00%> (+0.09%)
:arrow_up:

query.go
86.66% <0.00%> (+1.37%)
:arrow_up:

channel.go
66.55% <0.00%> (+1.91%)
:arrow_up:

user.go
38.21% <0.00%> (+3.10%)
:arrow_up:

... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 493ac13...2704651. Read the comment docs.


[FIX] repeats access error message fix

Are you sure, that we don't need to send request_sync_call in case of access_error?

What access_error actually is?

@yelizariev, I think yes. When I run the synchronization if access_error then he causes sync is infinitely

What access_error  is?

@yelizariev exception type

You can check whether it was sync_all request and don't send new one and show another wanrning ("Cannot sync") instead

I mean when access_error happens?

what is the reason of the error?


Added ability to specify Elmah log levels

I added the ability to pass LogLevels into the constructor so that you can control which levels are output in your application.
The use case is that in production only error and fatal messages should be logged to Elmah. In other environments all messages should be turned on.
Moving access to the ElmahLogLevel object out allows applications to control which messages get logged.

Thanks!


[datasets] change-my-view-corpus

change-my-view-corpus
This corpus contains a metadata-rich subset of conversations made in the ChangeMyView subreddit between 2013/01/01-2015/05/07. Each convokit conversation in this dataset is the corresponding full comment thread of an original post made to changemyview. Within each full thread are comments made by redditors (with the objective of the subreddit to change the opinion of the original post). There are 34911 Users, 293297 Utterances, and 3051 Conversations.
Dataset Details

To see the original posts in each conversation, note that the 'root' metadata of an Utterance (i.e. Reddit comment) would be the same as the 'id' metadata (this only holds true for original posts).
The main indicator of interest in this data is whether an argument succeeded in changing the original poster's (OP's) view. To denote a successful argument: the "success" field of an utterance takes the value of 1 (these utterances were the comments in the original post that succeeded in changing OP's mind), or the "success" field takes the value of 0 (this collection of utterances were a comment thread that are similar in nature to a successful argument in the full thread(matched on pair_id), but this argument failed to change OP's mind -- See section 4 of the cited paper for selection criteria of successful/unsuccessful arguments. All other comments take a 'success' value of None.
Note for pair_ids: the successful-unsuccessful argument pairs originally compiled by the authors are not unique at the Conversation-level nor Utterance-level (i.e. the original posts to the ChangeMyView subreddit can have multiple successful-unsuccessful comment pairs in their full-comment threads and some comments can have multiple opposing pairs -- see the relevant metadata fields at Conversation-level and Utterance-level below).
Note on missing data: 530 'Reddit comments' were included from the original data which did not have text nor an author. These were included for completeness, each utterance has User(name='[missing]').

User-level information:
- contains information about each user (i.e. Redditor)
- fields:
	- meta: empty dict (note: the original 'all' data file may have more information on individual Redditors, but the authors' pair_data only had comment-level data, therefore, we included the User-level data because each Redditor has a unique identifier: their Reddit username)

Utterance-level information:
- contains information about every comment posted in the changemyview subreddit posts that were collected for successful/unsuccessful arguments in Section 4 of the cited paper above
- fields:
	- id, unique comment identification provided by Reddit
	- user, unique user id (i.e. Reddit username)
	- root, comment identifier of the original post in the thread that this comment was posted in
	- reply_to, unique identifier of the comment that this comment was in direct response to
	- timestamp, comment post time provided by Reddit API
	- text, the full text (in string) of the comment
	- meta:
		- success, an indicator taking the value of 1 if the comment was part of a successful argument thread (i.e. an argument thread that changed the OP's mind), 0 if unsuccessful, and None if not part of either a successful or unsuccessful thread.
		- pair_ids, every successful-unsuccessful argument pair originally compiled by the authors has a unique pair_id. However, it is important to note that not every argument is unique (i.e. a single negative argument within a conversation could have two opposing positive arguments, which necessitates two corresponding pair_ids. Therefore, pair_ids is a list).
		- replies, a list of comment ids that respond directly to the current comment. For the OP post in the thread, this was constructed by selecting all comment ids with a "reply_to" field equal to the original post id (this was necessary because the original data provided by the authors did not include all the children of the OP post in their data format). For all comments besides the original post, the "replies" field was originally provided by Reddit API.
		- originally provided by Reddit API: author_flair_text, author_flair_css_class, banned_by, controversiality, edited, distinguished, user_reports, ups, downs, subreddit_id, subreddit, score_hidden, score, saved, report_reasons, mod_reports,  num_reports, likes, gilded,approved_by (see the [Reddit API](https://www.reddit.com/dev/api/) for more information)

Conversation-level
- fields:
	- _owner: Corpus object: the 'Change My View Corpus'
	- _id: unique comment identifier of the original post, which started the conversation thread
	- _utterance_ids: unique comment identifiers of every Utterance (i.e. comment) in the Reddit thread
	- _meta:
		- op-userID, the Reddit username of the original poster (OP)
		- op-text-body, the text of OP's first post (starting the conversation)
		- op-title, the title of OP's first post
		- pair_ids, a list of all the successful-unsuccessful pairs in the conversation (to subset the successful from unsuccessful Utterances, refer to the "success" metadata indicator in Utterance-level meta (above))
		- train, an indicator taking the value of 1 if this conversation was included in the training data subset of the original paper and 0 if it was included in the holdout set.

Corpus-level information
- fields:
	- meta:
		- name, 'Change My View Corpus'

Usage
data available on Google Drive
Convert Jupyter Notebook -- takes original dataset and converts it to convokit format
Simple Statistics Jupyter Notebook  -- demonstrates how to subset the data into successful and unsuccessful arguments (as described in Section 4 of the original paper) and provides basic statistics about their respective comments.
Notes
License: see  data source for details.
The data available was originally used in the following publication (cite if using the data):
Title: Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions
Authors: Chenhao Tan and Vlad Niculae and Cristian Danescu-Niculescu-Mizil and Lillian Lee
Year: 2016
Proceedings of WWW
Contact
Corpus converted into ConvoKit format by Andrew Szmurlo and Meir Friedenberg

Hey Andrew,
Thanks for your work on the change-my-view corpus! I've currently reviewing it for integration into ConvoKit.
I have some feedback and questions that we should discuss before integration:

Could you revise either your dataset issue or your README to the style here or here? You're fine information-wise (good detail too), but it's split between your dataset issue and your README (which contains some other comments and writing in the first-person that would not fit on the dataset description page.) I think the first-person comments are good for explaining certain decisions, but this could be phrased more formally perhaps or placed elsewhere.

For example, the note regarding how to 'expand the arguments to include all conversant' might be a better fit for the conversion notebook instead.
You should include documentation for the metadata from the Reddit API as well. (For some of them, you can copy it off the reddit-corpus dataset documentation.

At the moment, the corpus includes only utterances from threads from the training period, but not the holdout period. What is the motivation for this and would it be possible to include the rest of the data from the original dataset in this corpus as well? I believe the ideal scenario would be that the entirety of the original dataset is converted to the corpus format - with metadata denoting the status of specific conversations / utterances as coming from the 'training' or 'holdout' periods perhaps.

You added a 'replies' field to the Utterance metadata for Utterances that correspond to the OP post. This is a good instinct, but I'm wondering if this is necessary because couldn't you do the same with corpus.get_conversation(utt_id).get_utterance_ids()? (since utt_id == convo_id for the OP post.)

In any case, 'replies' seems to be a Conversation-level feature rather than an Utterance-level feature.

During the conversion process, I see that 462 comments were excluded because they were missing one of the common attributes. This is a common issue with Reddit data but I think it would be more ideal to keep the data in and put a placeholder 'None' value for the missing attributes

MINOR: The utterance timestamps seem to be floats right now. Could you convert them to ints?

Having worked with this dataset extensively, you and Meir are more familiar with it than I am, so do let me know if these changes make sense. Thanks!

Thanks Caleb, I'll take a look this weekend.
Hope you have a good fall break,
Andrew
On Fri, Oct 11, 2019, 6:38 PM calebchiam<EMAIL_ADDRESS>wrote:

Hey Andrew,
Thanks for your work on the change-my-view corpus! I've currently
reviewing it for integration into ConvoKit.
I have some feedback and questions that we should discuss before
integration:

Could you revise either your dataset issue or your README to the
style here
https://zissou.infosci.cornell.edu/convokit/documentation/chromium.html or
here
https://github.com/emtseng/Cornell-Conversational-Analysis-Toolkit/blob/master/datasets/friends-corpus/README.md?
You're fine information-wise (good detail too), but it's split between your
dataset issue and your README (which contains some other comments and
writing in the first-person that would not fit on the dataset description
page.) I think the first-person comments are good for explaining certain
decisions, but this could be phrased more formally perhaps or placed
elsewhere.

For example, the note regarding how to 'expand the arguments to include
all conversant' might be a better fit for the conversion notebook instead.
You should include documentation for the metadata from the Reddit API as
well. (For some of them, you can copy it off the reddit-corpus dataset
documentation
https://zissou.infosci.cornell.edu/convokit/documentation/subreddit.html
.

At the moment, the corpus includes only utterances from threads from
the training period, but not the holdout period. What is the motivation for
this and would it be possible to include the rest of the data from the
original dataset in this corpus as well? I believe the ideal scenario would
be that the entirety of the original dataset is converted to the corpus
format - with metadata denoting the status of specific conversations /
utterances as coming from the 'training' or 'holdout' periods perhaps.
2.
You added a 'replies' field to the Utterance metadata for Utterances
that correspond to the OP post. This is a good instinct, but I'm wondering
if this is necessary because couldn't you do the same with
corpus.get_conversation(utt_id).get_utterance_ids()? (since utt_id ==
convo_id for the OP post.)
In any case, 'replies' seems to be a Conversation-level feature rather
than an Utterance-level feature.

During the conversion process, I see that 462 comments were excluded
because they were missing one of the common attributes. This is a common
issue with Reddit data but I think it would be more ideal to keep the data
in and put a placeholder 'None' value for the missing attributes
2.
MINOR: The utterance timestamps seem to be floats right now. Could you
convert them to ints?
Having worked with this dataset extensively, you and Meir are more
familiar with it than I am, so do let me know if these changes make sense.
Thanks!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CornellNLP/Cornell-Conversational-Analysis-Toolkit/issues/27?email_source=notifications&email_token=AHN7KFEZEE2LXDMUQN4OPLDQOD57LA5CNFSM4I4LYTA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBBMKKA#issuecomment-541246760,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHN7KFEZRAR6BEEWM7JNQFTQOD57LANCNFSM4I4LYTAQ
.

Hi Caleb,
Here's an update to your requests:

*Could you revise either your dataset issue or your README to the
style here
https://zissou.infosci.cornell.edu/convokit/documentation/chromium.html
or here
https://github.com/emtseng/Cornell-Conversational-Analysis-Toolkit/blob/master/datasets/friends-corpus/README.md?
You're fine information-wise (good detail too), but it's split between your
dataset issue and your README (which contains some other comments and
writing in the first-person that would not fit on the dataset description
page.) I think the first-person comments are good for explaining certain
decisions, but this could be phrased more formally perhaps or placed
elsewhere.     **For example, the note regarding how to 'expand the
arguments to include all conversant' might be a better fit for the
conversion notebook instead.     *You should include documentation for
the metadata from the Reddit API as well. (For some of them, you can copy
it off the reddit-corpus dataset documentation
https://zissou.infosci.cornell.edu/convokit/documentation/subreddit.html.

I've updated the issue to reflect Emily's README (Friends dataset) and I've
moved the first-person notes/process comments/data expansion methods to a
Notes section in the convert file as you suggested. I also provided a link
to the Reddit API. Note: the authors originally didn't include any
information on these 'Reddit Provided' variables, so I didn't have much to
work with. DNM suggested that I combine all of these into a single list
just for brevity in the readme.

At the moment, the corpus includes only utterances from threads from
the training period, but not the holdout period. What is the motivation for
this and would it be possible to include the rest of the data from the
original dataset in this corpus as well? I believe the ideal scenario would
be that the entirety of the original dataset is converted to the corpus
format - with metadata denoting the status of specific conversations /
utterances as coming from the 'training' or 'holdout' periods perhaps*.*
This was a good catch, thanks. I've added in the holdout data and included
a conversation-level metadata field 'train' which indicates whether the
conversation was in the training data (1) or holdout data (0). Note: we
only used and converted to convokit the "pair data" from section 4 of the
paper. Their full data is just the CMV subreddit
from 2013/01/01-2015/05/07, and I believe you've already included the
entire CMV subreddit in the Reddit Corpus.

*You added a 'replies' field to the Utterance metadata for Utterances
that correspond to the OP post. This is a good instinct, but I'm wondering
if this is necessary because couldn't you do the same
with corpus.get_conversation(utt_id).get_utterance_ids()? (since utt_id ==
convo_id for the OP post.) *In any case, 'replies' seems to be a
Conversation-level feature rather than an Utterance-level feature.
Note: the comments listed in the 'replies' metadata are only comments
made in direct response to a comment. This is separate from all the
comments in a conversation tree, which you can get from
"corpus.get_conversation(utt_id).get_utterance_ids()"   It is unique at the
comment-level, not the conversation-level. I apologize that the readme
wasn't clear, and I have updated it to reflect this.

*During the conversion process, I see that 462 comments were excluded
because they were missing one of the common attributes. This is a common
issue with Reddit data but I think it would be more ideal to keep the data
in and put a placeholder 'None' value for the missing attributes *
Even though they lack text, I had originally planned to add these, but
since they also are missing an author, the
Corpus(utterances=utterance_list) function breaks down because for these
comments, User=None.

*MINOR: The utterance timestamps seem to be floats right now. Could you
convert them to ints? *Fixed.
On Fri, Oct 11, 2019 at 6:42 PM Andrew Szmurlo<EMAIL_ADDRESS>wrote:

Thanks Caleb, I'll take a look this weekend.
Hope you have a good fall break,
Andrew
On Fri, Oct 11, 2019, 6:38 PM calebchiam<EMAIL_ADDRESS>wrote:

Hey Andrew,
Thanks for your work on the change-my-view corpus! I've currently
reviewing it for integration into ConvoKit.
I have some feedback and questions that we should discuss before
integration:

Could you revise either your dataset issue or your README to the
style here
https://zissou.infosci.cornell.edu/convokit/documentation/chromium.html or
here
https://github.com/emtseng/Cornell-Conversational-Analysis-Toolkit/blob/master/datasets/friends-corpus/README.md?
You're fine information-wise (good detail too), but it's split between your
dataset issue and your README (which contains some other comments and
writing in the first-person that would not fit on the dataset description
page.) I think the first-person comments are good for explaining certain
decisions, but this could be phrased more formally perhaps or placed
elsewhere.

For example, the note regarding how to 'expand the arguments to include
all conversant' might be a better fit for the conversion notebook instead.
You should include documentation for the metadata from the Reddit API as
well. (For some of them, you can copy it off the reddit-corpus dataset
documentation
https://zissou.infosci.cornell.edu/convokit/documentation/subreddit.html
.

At the moment, the corpus includes only utterances from threads from
the training period, but not the holdout period. What is the motivation for
this and would it be possible to include the rest of the data from the
original dataset in this corpus as well? I believe the ideal scenario would
be that the entirety of the original dataset is converted to the corpus
format - with metadata denoting the status of specific conversations /
utterances as coming from the 'training' or 'holdout' periods perhaps.
2.
You added a 'replies' field to the Utterance metadata for Utterances
that correspond to the OP post. This is a good instinct, but I'm wondering
if this is necessary because couldn't you do the same with
corpus.get_conversation(utt_id).get_utterance_ids()? (since utt_id ==
convo_id for the OP post.)
In any case, 'replies' seems to be a Conversation-level feature rather
than an Utterance-level feature.

During the conversion process, I see that 462 comments were excluded
because they were missing one of the common attributes. This is a common
issue with Reddit data but I think it would be more ideal to keep the data
in and put a placeholder 'None' value for the missing attributes
2.
MINOR: The utterance timestamps seem to be floats right now. Could
you convert them to ints?
Having worked with this dataset extensively, you and Meir are more
familiar with it than I am, so do let me know if these changes make sense.
Thanks!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CornellNLP/Cornell-Conversational-Analysis-Toolkit/issues/27?email_source=notifications&email_token=AHN7KFEZEE2LXDMUQN4OPLDQOD57LA5CNFSM4I4LYTA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBBMKKA#issuecomment-541246760,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHN7KFEZRAR6BEEWM7JNQFTQOD57LANCNFSM4I4LYTAQ
.

Thanks Andrew, these changes look good.
For #4, I would recommend instantiating a User(name='[missing]') and passing that as the argument. Just add a note about this in the conversion documentation and above in the description.

Fixed. The 530 comments are now included in the data -- I've updated the
convert nb to mention this and made a note in the issue above (under
Dataset Details).
On Mon, Oct 14, 2019, 12:18 PM calebchiam<EMAIL_ADDRESS>wrote:

Thanks Andrew, these changes look good.
For #4
https://github.com/CornellNLP/Cornell-Conversational-Analysis-Toolkit/pull/4,
I would recommend instantiating a User(name='[missing]') and passing that
as the argument. Just add a note about this in the conversion documentation
and above in the description.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CornellNLP/Cornell-Conversational-Analysis-Toolkit/issues/27?email_source=notifications&email_token=AHN7KFCB6PMRAPPILBF7W7DQOSLWRA5CNFSM4I4LYTA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBFNMHY#issuecomment-541775391,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHN7KFCIJYZA6C6RGRAU3TTQOSLWRANCNFSM4I4LYTAQ
.

Thanks for the prompt follow-up! Pleased to inform you that the corpus has been successfully integrated into ConvoKit (:

Awesome! Thanks Caleb.
On Wed, Oct 16, 2019 at 3:56 PM calebchiam<EMAIL_ADDRESS>wrote:

Thanks for the prompt follow-up! Pleased to inform you that the corpus has
been successfully integrated into ConvoKit
https://zissou.infosci.cornell.edu/convokit/documentation/winning.html
(:
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CornellNLP/Cornell-Conversational-Analysis-Toolkit/issues/27?email_source=notifications&email_token=AHN7KFBDFRLKC5JCILHIKNLQO5WQFA5CNFSM4I4LYTA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBNXN5Y#issuecomment-542865143,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHN7KFHDYUFI2C73AMZO6ETQO5WQFANCNFSM4I4LYTAQ
.

@all-contributors please add @szmurlo for data


Data format

Hi there, I'm recently processing the CCRs data and I noticed lines like these:
1	865645	865660	91.316325156	AL645608.1<PHONE_NUMBER>60,865665-865682	VARFALSE,VARFALSE	0.062	0.000	30.580	3.430	11.335235448	47070
1	865665	865682	91.316325156	AL645608.1<PHONE_NUMBER>60,865665-865682	VARFALSE,VARFALSE	0.062	0.000	30.580	3.430	11.335235448	47070

I'm wondering what is the rationale behind keeping the two ranges separate? The reason I ask is that the start, end and ranges seem to be a bit duplicated. And the VARFLAG has to be matched to each range accordingly. If these two ranges need to be separated for scientific reason, can we just omit the ranges field and keep the VARFLAG a single value instead of comma-separated entry?
Please let me know. Many thanks!

Hey,
So the starts and ends (columns 2 and 3) will only be duplicated if two different genes overlap the same linear space.  The reason for keeping them separate for multi-exon spanning CCRs like the one you've demonstrated above is that the unique key or (ranges + gene which is also unique always) will allow you to tell if multiple lines are part of the same CCR.  This facilitates the use of tools like BEDTools intersect, though you can also use a BED12 of those regions to accomplish a similar goal.  The main reason I did it though, is the average user would likely misuse it and if the intronic regions of the BED12 blocks intersected a variant or region of interest for them they'd assume it was in the CCR when it is not.
The VARFLAG is just FALSE if it is a true CCR (meaning it doesn't contain a variant) and TRUE if it is a non-constrained region, so if it is a large area with many overlapping INDELs, it could be a multi-exon region of non-constraint.  If your goal is to summarize VARFLAG into one value, that can be done with a pretty simple command; I don't really have any plans to change its representation.  The ranges field can be really helpful so I definitely don't have plans to remove it.

Hi @jimhavrilla,
Thanks very much for the detailed explanation. It's all clear to me now.
I'm looking forward to the new release.
Thanks!

Not a problem. Hopefully the release will be sooner than later
On Wed, Apr 10, 2019, 11:53 AM Li Yin<EMAIL_ADDRESS>wrote:

Closed #3 https://github.com/quinlan-lab/ccr/issues/3.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/quinlan-lab/ccr/issues/3#event-2266898923, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AGQ8BPdr_BEgdkvCr9xcL5nW4OAv6asRks5vfiUcgaJpZM4cn1BH
.


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-k8s-25-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 3297

Questions ?
Please refer to the Backport tool documentation


[WIP] Add 'refresh' script for (re)initializing/resetting Drupal environment

The Problem:
Cleaning up/initializing your Drupal environment to begin new work, or switching branches to test someone else's work, requires several steps. It's easy to forget some of them, and it takes cognitive load to remember. Lots of folks write a little custom bash alias/script to handle it for them on a given project. This is just a very complete version of that, and can be used on most Drupal projects.
The steps required to refresh a Drupal environment include:

fetch the latest code
download and import a database
compile front-end assets
composer install
execute Drupal database updates
rebuild caches
and other stuff

How this PR Solves The Problem:
Provides a single command to completely (re)initialize/refresh your ddev Drupal environment. Supports various flags to turn steps on and off. See README.md for the full list of options. This command is especially useful when collaborating with other developers or otherwise switching between branches for testing.
Manual Testing Instructions:

Create a ddev Drupal environment. You can follow the Drupal.org documentation, or use an existing site you have.

Copy the refresh script into place: .ddev/commands/host/refresh

Also install the drupal-login and gulp custom commands from ddev-contrib.

Run ddev refresh, and see that all the default steps run.

Run ddev refresh some-existing-branch and see that an existing branch is checked out + pulled before refreshing.

Run ddev refresh -h and see all the options available. Try out some them. E.g., --no-restart to skip restarting ddev, and --no-login to skip the login step at the end.

The only two steps you can't (easily) test are --import-db and --existing-sql to download and import a database dump from some upstream environment, as those are usually project specific and require some extra setup to use. Since they don't run by default, I chose to leave them in place for users to leverage if they wish.

Manual Testing Environments:

[ ] Windows
[ ] MacOS
[ ] Linux

Thank you @rfay!! And apologies for not following the contribution guidelines... I was trying out this newfangled GitHub Codespaces thing, and it auto-created the PR for me. Will update the PR description later.

All good, thanks!

I decided to take out the dependency on Drainpipe, so I need to do some more testing locally to make sure all the steps still work.

Did you get this to where you wanted it? If so, please mark ready for review.

No response from OP. Happy to reopen when you're ready to mark it ready for review.


latest calico, cni, dashboard, etcd

calico 3.4 (using kubernetes as datastore instead of an own etcd)
latest stable cni plugin
kubernetes dashboard link update
improved reliability for kirb config stage (waiting for usable admin.conf; allow label overwrite)
improved krib ha support (use proper node name for labeling)

Let me know when you are done and I'll review it for pulling in.

@galthaus I'm done, I will not add more in that PR


feat: add Solidity bounty

To-do list:

Identify version and exploit
Exploit.sol

Hi @guidanoli !

I did not see new commits so I am understanding that the bounty is not inside the machine yet, right?
Is any solidity code a valid input for the bounty?

Hi, @claudioantonio!

Correct. This will be done in a later PR, as, I believe, this will incur in changes on the back-end.
I've chosen the input to be a JSON that contains not only the Solidity contract(s) to be compiled but also compiler settings. You can learn more about the Solidity compiler input JSON description here.


Image info uploads fail

It's currently possible to add images without errors, but any attempt to add or edit the attribution info gets the Uh-Oh error:

This is what was printed in the console before the error info printed in the message above:
Failed to load resource: the server responded with a status of 401 (Unauthorized)


UNR-2189 - Bugfix cloud launch config default

Description
Changes the default cloud launch config to empty
Also, the user will now be warned if a cloud deployment is launch with the manual_worker_connection_only set to true, in the launch config
Release note
Features:

A warning is shown if a cloud deployment is launched with the manual_worker_connection_only flag set to true

Bug Fixes:

The default cloud launch configuration is now set to a configuration that works in the cloud

Tests
How did you test these changes prior to submitting this pull request?
Manual testing by loading the deploy window and checking the default launch config value
Launching cloud deployment with and without manual_worker_connection_only set to true
STRONGLY SUGGESTED: How can this be verified by QA?
Test that the default launch config value of the deploy window is empty by default.
Try launching cloud deployment (via editor) with and without the manual_worker_connection_only flag set in the launch config
Documentation
How is this documented (for example: release note, upgrade guide, feature page, in-code documentation)?
Documented in changelog
Primary reviewers

I checked the cloud launch tutorial docs and thankfully they do not assume that fields are pre-populated, so no docs changes are necessary.


Thread Safety Concerns with gilrs and audio_player

There is no mention of locking or managing concurrent access to shared resources like the gilrs gamepad instance and audio_player. If accessed from multiple threads, this could cause undefined behavior.
Ensure thread safety using appropriate synchronization mechanisms (e.g., Mutex or RwLock).

But there is no thread safety concerns because there is no threads here.
I don't see a point of concurrency if there is nothing running concurrently.
This tui binary is just a binary, i.e. others can't use it as a dependancy and introduce threading issues.

Yeah forgot


[release-0.59] Add 100Mi of overhead for guaranteed vmis

What this PR does / why we need it:
This is a manual cherrypick of https://github.com/kubevirt/kubevirt/pull/10566
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[ ] Design: A design document was considered and is present (link) or not required
[ ] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[ ] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[ ] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[ ] Community: Announcement to kubevirt-dev was considered

Release note:

NONE

/retest

/retest-required


EN Documentation: Checkbox component

TODO*

[ ] First version of EN documentation for component using documentation template
[ ] Review of documentation
[ ] Publishing documentation on site

Outcomes

First version of documentation for component for feedback

Deliverables

Published documentation page(s)

Estimated Effort

[ ] XS: 1 to 2 hours
[ ] S: half a day
[ ] M: 1-2 day
[ ] L: 1 week
[ ] XL: Move to an epic

Out of scope

Added my documentation to the working file

added my documentation to the working file


Use ParseResults to improve performance

Please describe your suggestion
Use parse results to improve performance.
Relevant links:
Parsing user input (Brigadier docs)
Pseudocode
Creating a wrapped version of the parse results
Basically, create parse results for a "default command sender"
let cmdDispatcher = CommandAPIHandler.brigadierDispatcher

String command = command[0] == "/" ? command.substring(1) : command
ParseResults = cmdDispatcher .parse(command, NMSServer.getServerCommandListener())

WrappedParseResults = wrap(ParseResults)

Executing the parse results
cmdDispatcher.execute(unwrap(WrappedParseResults))

Or:
Upon execution, change the command sender?
let pr = unwrap(WrappedParseResults)
let playerContext:CommandListenerWrapper = NMSPlayer.getCommandListener();

cmdDispatcher.execute(new ParseResults<CommandListenerWrapper>(pr.getContext().withSource(playerContext), pr.getStartIndex(), pr.getReader(), pr.getExceptions()))

Cache design
map<<CommandStr, BukkitCommandSender_UUID>, ParseResult>

Does MC not normally allow inputting parse results into their execute method in NMS CommandDispatcher?
The main issue is trying to prevent memory leaks.
Also I don't see any performance benefit unless you parse it (weather sync or async is a different question) while the user is typing the command. I checked protocol and the clients does not send tab Complete request unless it's an ask server request.
So one solution for real time live parsing could be to make the command register an ask server request at every single position within the command tree so the client sends current command line every single time the client makes a modification and also merge other preexisting suggestions into that fray without breaking anything else. This is odd and only partially plausible.
Now if this were the client this would be easy since the client does use parse results in real time.

It's currently just a suggestion, but for clarification, you can use parse results to reduce the overhead of parsing user input. You could look it up in some form of cache and it goes straight to execution instead. NMS does let you execute parse results once they've been generated.
The idea would be if say you have a command block on some form of loop (say you run some command 20 times a second, it could get very intense).
The major issue as you've said would be memory leaks and I honestly haven't thought of a workaround for it. For now, this idea is just speculation.
I don't plan to include this in the 3.0 release, but it's a potential idea for the far future and I thought it would be good to note down.


make more methods take mut self

I think most methods need to take &mut self to be implemented in sir.
Description
Description of the pull request changes and motivation.
Checklist

[ ] Linked to Github Issue
[ ] Unit tests added
[ ] Integration tests added.
[ ] This change requires new documentation.

[ ] Documentation has been added/updated.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@b2a2574). Click here to learn what that means.
The diff coverage is n/a.

:exclamation: Current head ac8eed6 differs from pull request most recent head 9e4e3d9. Consider uploading reports for the commit 9e4e3d9 to get more accurate results

@@           Coverage Diff           @@
##             main     #316   +/-   ##
=======================================
  Coverage        ?   71.62%
=======================================
  Files           ?      116
  Lines           ?    20617
  Branches        ?        0
=======================================
  Hits            ?    14767
  Misses          ?     5850
  Partials        ?        0

Files
Coverage Δ

src/starknet.rs
8.49% <ø> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


[Draft] Fix for JointsMatch

Purpose of this PR
Ticket/Jira #: PR #308
A user pointed out that JointsMatch() functionality regressed in PR #261, resulting in it returning true in nearly all cases.
I'm not sure whether the intended functionality for this method is to just check whether the two inputs are the same instance, or whether the full array of strings is the same, but as the behavior prior to the change was the latter I have reverted it to do that.
Testing
Functional Testing status:
Not sure how best to test this as no repro case was supplied. I'll run the normal test suite.
Performance Testing status:
Very small performance impact expected due to the amount of string comparisons that may be introduced.
Overall Product Risks

Complexity: minimal

Halo Effect: minimal

Additional information
Note to reviewers:

Reminder:

[ ] Add entry in CHANGELOG.md (if applicable)

(turns out this is a duplicate effort)


[INTERNAL] Move some verbose logging to silly level

This should help analysing issues.

bundle/Resolver: The list of resolved modules typically gets truncated
by the console.log anyways: "... xx more items". So we should think of
a better way to output something usefull there if needed (maybe the
count of modules?).
bundle/Builder: We typically find lots of source maps.
buildThemes: Not sure about this one, but I think it is typically OK
to not find theme libraries, since they might just not be requested.

Coverage remained the same at 93.843% when pulling 6059c161a49a79e0d190ba8737ef1ee7479348d3 on be-more-silly into c4632468b088cb9dfa64e058770d3786e689a183 on next.


Transform fillRule attribute to fill-rule

Forces the fillRule prop, generated by svg-react-loader, to be rendered as fill-rule. This is necessary for SVG icons in React 0.14, but can be removed in React 15.x.
In all its glory...

Reviewer: @aldipower


Allow using different OpenAI/Azure keys for embeddings vs LLM endpoints

See https://github.com/cpacker/MemGPT/discussions/999#discussioncomment-8461322

I could takle this issue but not sure where to start, either by extending the 'memgpt configure' menu to setup embeddings endpoint manually or .. ?
https://github.com/cpacker/MemGPT/discussions/999#discussioncomment-8584069


Regression performance

Performance isn't great at the moment. At low energies the prediction is quite poor. At higher energies we see some improvement, but it's still might not be good enough (see plots below).
Questions/ideas:

Search for more useful variables.
Would feature selection in each energy bin improve things (I doubt it)?
Why do we have a bias in our predictions? It seems like we generally predict better PSF than the true one. Such a "consistent" bias points to a problem in the logic or a bug, no?
Would more events for training or using diffuse gamma improve things?

Plots below are for gamma_onSource.S.3HB9-FD_ID0.eff-0.root.

Regarding the bias: weird! Although to tell you the truth, I'm not so worried about it, as what we want to do is rank the events... So biases will not really be a problem. Although lets keep an eye on it, as it could be the cause of some mistake somewhere...

I think we can clearly see this bias in the confusion matrix in #2 (pasted below for convenience). The events in the lower left cells are a direct result of this bias. It would be great to figure out why it's there and fix it.

Regarding the bias: weird! Although to tell you the truth, I'm not so worried about it, as what we want to do is rank the events... So biases will not really be a problem. Although lets keep an eye on it, as it could be the cause of some mistake somewhere...

I think we can clearly see this bias in the confusion matrix in #2 (pasted below for convenience). The events in the lower left cells are a direct result of this bias. It would be great to figure out why it's there and fix it.

Are you sure this is a result of the bias?
From a very naive point of view: If you increase that bias by a factor 10, but keep the shape of the "blob" in the confusion matrix identical, then we would get exactly the same predictions (when you rank the events, the only thing that matters is the shape of that blob, not where it is with respect to the dashed line you show).
Or perhaps I'm not properly understanding the effect of the bias? (you have thought more about this than me!)

Are you sure this is a result of the bias?
From a very naive point of view: If you increase that bias by a factor 10, but keep the shape of the "blob" in the confusion matrix identical, then we would get exactly the same predictions (when you rank the events, the only thing that matters is the shape of that blob, not where it is with respect to the dashed line you show).
Or perhaps I'm not properly understanding the effect of the bias? (you have thought more about this than me!)

Oh, now I realise the mistake. I defined the event_type bins based on the true angular error, but I should actually define it based on the reconstruction angular error. Defining it based on the true value results in the bias translating to that lower edge cell. If we define it from the reco value (which is what we should do!) then I think you are right and that the bias will not have an effect.

Oh, now I realise the mistake. I defined the event_type bins based on the true angular error, but I should actually define it based on the reconstruction angular error. Defining it based on the true value results in the bias translating to that lower edge cell. If we define it from the reco value (which is what we should do!) then I think you are right and that the bias will not have an effect.

Calculated the regression performance with the Prod5 sample, using the baseline south array, on source gamma, pointing north, with minimum multiplicity of 2 telescopes. Full file name is
'/lustre/fs22/group/cta/users/maierg/analysis/AnalysisData/prod5-Paranal-20deg-sq08-LL/EffectiveAreas/EffectiveArea-50h-ID0-NIM2LST2MST2SST2SCMST2-g20210921-V3/BDT.DL2.50h-V3.g20210921/gamma_onSource.S.BL-4LSTs25MSTs70SSTs-MSTF_ID0.eff-0.root'
Regression results are below. The scores plot compares the results for all variables and for cases where we exclude the two new "promising" variables. I had hoped that those variables will provide an improvement in performance, but they don't. However, we anyway get a significant boost in performance with this new sample. Possible reasons for this are:

The number of minimum telescope is now two, which means there is now a larger spread in the angular error.
The increase in the total number of events (almost a factor 10) could mean the training works better (extra motivation to study #11).

In particular I am happy to see we don't have a bias anymore. Not sure if an increase in statistics can be the reason for that, but maybe.
The confusion matrix plots will be uploaded to #2.

Hi Orel,
This really looks cool! Statistics definitely seem to play a role here indeed... I do have access to the path you linked, so I can try to copy these files and play around with them.
One question: When you say statistics are larger, is it because more MCs were produced? Or because when including multiplicity 2 the statistics went up?
Should we try to move to generating IRFs? I don't mind playing around with that... I guess Max will be happy to help!
Best,
Tarek

One question: When you say statistics are larger, is it because more MCs were produced? Or because when including multiplicity 2 the statistics went up?

Not entirely sure. Without digging into the previous file and testing the cuts, I can't answer whether that file already includes only >4 telescope multiplicity or not. It might be the case because the Prod3b file is significantly smaller (3.3GB vs. 14GB).
The new files are the same size, regardless of the cut on multiplicity, since they contain the same events, just with different cut values (as far as I can tell).

Should we try to move to generating IRFs? I don't mind playing around with that... I guess Max will be happy to help!

Yes, producing IRFs would be good. It would help answer also the question regarding the number of events. You can start with that if you wish and I will do the 2-3 things missing with the regression/classification.

Added a few more variables to improve performance. Namely, we have a few variables which are given per telescope, like size, distance, crossing, asymmetry, etc. So far we only used the average of those values. However the distribution of these variables is also important of course. So, in order to add some additional information, I added also the median and the std of each distribution of sizes, distance, crossing, etc. This resulted in a significant improvement in performance.
(I tried adding the entire set of variables as well, but this reduces performance, will explain if anyone is actually interested.)
Below I put plots for the comparison of scores of each model,

old_features: before adding these variables (essentially only the averages of the distributions).
dist_characteristics: all characteristics (average, median and std).
no_average: same as dist_characteristics without the average.
no_me: same as dist_characteristics without the median.
no_std: same as dist_characteristics without the std.

The model with all characteristics is clearly the best, so we will stay with it.
The distributions of the true angular error vs. the predicted one are pasted below.

The confusion matrix is also significantly better and I think now we can clearly have 3 event types.

Just for future reference: the plots shown from  Feb 11, 2021 were artificially good, because we were adding too much information to the training (if I remember correctly, we were using point-like gammas as training? Can't remember...).
We will update these plots and perhaps close the issue.

That is true, the plots here were made using point-like gammas. I am not sure I would consider this adding too much information, but indeed we most likely cannot reach this level of performance with diffuse-like gammas.


Use an alternate download mirror for the bootstrapper

Github's free Rest API hits a limit and breaks the entire installer application when the auto updater can't reach the latest release.
Perhaps an alternate service could be used instead for the updater to pull from.

Yeah I'm planning to publish the installer on my own BunnyCDN repo, that won't have any ratelimits (and should be faster)


feat(linter): Support potentially dangerous fixes behind a --fix-dangerously flag

Some problems are potentially auto fixable, but doing so may be undesireable. In some cases, it could produce incorrect code. In other cases, it could change the behavior of applications.
ESLint handles this with Rule Suggestions, which are made available to language servers but do not get applied via --fix.
Since the linter is mostly used as a CLI (oxlint), we need to support applying suggestions there too. I think we should do this with a --fix-dangerously flag.
Tasks

NOTE: only the first two tasks are prerequisites for #4179

[ ] refactor(linter): update FIx to support suggestions
[ ] feat(cli): add --fix-dangerously flag
[ ] feat(lsp): add command to apply suggestions

Interesting, can you hold onto adding the flag and the command? I want to research and understand these first.

Too late unfortunately, I've got a draft out already

We may also want to support multiple suggestions for each diagnostic.


Fix detection CustomEvent

We use polyfill with 'require' and we have problems with detection CustomEvent in Chrome.

I'm assuming this is related to browserify/webpack type of scenarios?  Looks to be a good request.  I will merge tomorrow evening if there are no objections.


Fixed pixi reference at phaser.comments.d.ts by rerunning "grunt tsdoc"

TypeScript Defs

Fixed typescript definitions by rerunning grunt tsdoc for proper use of phaser with tsd link.
The reference to pixi.d.ts within file phaser.comments.d.ts and the reference within package.json to pixi.comments.d.ts lead to a lot of duplicate errors, when compiling a typscript project after executing tsd link.
After grunt tsdoc the reference is set back to pixi.comments.d.ts at phaser.comments.d.ts

fixes #2445

Thank you very much for your pull request, and for helping Phaser out.
Sadly your PR is against the master branch of Phaser. As per our Contribution Guidelines all development needs to take place on the dev branch.
Feel free to re-submit your PR against the correct branch.


您好，运行example/re/standard/run.py报错

Describe the question
程序会卡特别久，提醒信息如下
This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/rnn.py:47: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.3 and num_layers=1
warnings.warn("dropout option adds dropout after all but last "
卡很久之后 报错信息如下
)
[2022-04-19 11:26:20,159][main][INFO] - ========== Start training ==========
Traceback (most recent call last):
File "run.py", line 162, in
main()
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/utils.py", line 201, in run_and_report
raise ex
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/utils.py", line 347, in
lambda: hydra.run(
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 107, in run
return run_job(
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/core/utils.py", line 127, in run_job
ret.return_value = task_function(task_cfg)
File "run.py", line 96, in main
train_loss = train(epoch, model, train_dataloader, optimizer, criterion, device, writer, cfg)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/deepke/relation_extraction/standard/tools/trainer.py", line 36, in train
y_pred = model(x)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/deepke/relation_extraction/standard/models/LM.py", line 22, in forward
a = self.bert(word, attention_mask=mask)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 833, in forward
encoder_outputs = self.encoder(
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 476, in forward
layer_outputs = layer_module(
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 398, in forward
self_attention_outputs = self.attention(
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 333, in forward
self_outputs = self.self(
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 240, in forward
mixed_query_layer = self.query(hidden_states)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/functional.py", line 1612, in linear
output = input.matmul(weight.t())
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)
wandb: Waiting for W&B process to finish, PID 15885... (failed 1).
wandb: You can sync this run to the cloud by running:
wandb: wandb sync /data/sen/DeepKE/example/re/standard/logs/2022-04-19_11-17-16/wandb/offline-run-20220419_111718-2zdxarri
wandb: Find logs at: ./logs/2022-04-19_11-17-16/wandb/offline-run-20220419_111718-2zdxarri/logs/debug.log
wandb:
Environment (please complete the following information):
使用显卡为 NVIDIA GeForce RTX 3090
Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

您好，引起上述错误的原因有很多种，建议您着重检查一下是否存在输入的矩阵维度与函数需求的维度不匹配的问题，详细请参考https://stackoverflow.com/questions/66600362/runtimeerror-cuda-error-cublas-status-execution-failed-when-calling-cublassge
以上意见仅供参考😊

代码未作任何修改 应该不是输入矩阵的问题  就是想跑通example

Describe the question 程序会卡特别久，提醒信息如下 This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model). This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). /root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/rnn.py:47: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.3 and num_layers=1 warnings.warn("dropout option adds dropout after all but last " 卡很久之后 报错信息如下 ) [2022-04-19 11:26:20,159][main][INFO] - ========== Start training ========== Traceback (most recent call last): File "run.py", line 162, in main() File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/main.py", line 32, in decorated_main _run_hydra( File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra run_and_report( File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/utils.py", line 201, in run_and_report raise ex File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/utils.py", line 198, in run_and_report return func() File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/utils.py", line 347, in lambda: hydra.run( File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 107, in run return run_job( File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/hydra/core/utils.py", line 127, in run_job ret.return_value = task_function(task_cfg) File "run.py", line 96, in main train_loss = train(epoch, model, train_dataloader, optimizer, criterion, device, writer, cfg) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/deepke/relation_extraction/standard/tools/trainer.py", line 36, in train y_pred = model(x) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/deepke/relation_extraction/standard/models/LM.py", line 22, in forward a = self.bert(word, attention_mask=mask) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 833, in forward encoder_outputs = self.encoder( File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 476, in forward layer_outputs = layer_module( File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 398, in forward self_attention_outputs = self.attention( File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 333, in forward self_outputs = self.self( File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/transformers/modeling_bert.py", line 240, in forward mixed_query_layer = self.query(hidden_states) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 87, in forward return F.linear(input, self.weight, self.bias) File "/root/anaconda3/envs/lian/lib/python3.8/site-packages/torch/nn/functional.py", line 1612, in linear output = input.matmul(weight.t()) RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)
wandb: Waiting for W&B process to finish, PID 15885... (failed 1). wandb: You can sync this run to the cloud by running: wandb: wandb sync /data/sen/DeepKE/example/re/standard/logs/2022-04-19_11-17-16/wandb/offline-run-20220419_111718-2zdxarri wandb: Find logs at: ./logs/2022-04-19_11-17-16/wandb/offline-run-20220419_111718-2zdxarri/logs/debug.log wandb:
Environment (please complete the following information): 使用显卡为 NVIDIA GeForce RTX 3090
Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

根据提醒信息和第一张图的报错推测您可能用错模型了，请检查 https://github.com/zjunlp/DeepKE/blob/main/example/re/standard/conf/config.yaml 里model是否为可选参数，transformers版本是否正确。

torch和transformers版本如下

import torch
torch.version
'1.5.0+cu101'
torch.cuda.is_available()
True
import transformers
transformers.version
'3.4.0'

config.yaml中model参数设置是lm

3090显卡上torch1.5的版本过低了，可以试试1.10

3090显卡上torch1.5的版本过低了，可以试试1.10哈

太感谢了 问题解决了 torch版本改为1.8可以了


Update julia from 1.10.0 to 1.10.3

The other patch releases had some issue affecting our build, specifically loading libribasim with Python / ribasim_api (https://github.com/JuliaLang/julia/pull/53643). These should all be resolved now, but I'll trigger a build to be sure.

TeamCity also likes 1.10.3 :)


Horizontal fusions of gemms and convolutions

Here are the times for this PR vs develop:

inceptionv4: 12.9815ms vs 13.9111ms
bert: 24.9702ms vs 28.804ms

The bert module can be improved more by removing the extra contiguoug operators, which I believe @scxiao is working on.
The inception model can also be improved by removing the extra multiply operators. I plan to address that in a future PR.

@kahmed10 @scxiao can you review this?

I am wondering if it'd be easier to keep track of the name of the input in the slice operator and the slice order whenever we parse split. In tensorflow, multiple outputs usually look something like "node:0," "node:1," "node:2..." This way, you can tell both the order of the output as well as the parent node.

@scxiao anymore feedback?


Make the @yamada-ui/notice component headless

script: make-component-headless
package: @yamada-ui/notice
Description
Currently, the @yamada-ui/notice component has minimal styles set within the component's source code.
We want to make this style transferable to the theme and make the component headless, enabling more flexible UI customization.
Problem Statement/Justification
By making the @yamada-ui/notice component headless, we can achieve more flexible styling tailored to the project.
Proposed Solution or API
The styles within the component are mostly defined by css: CSSUIObject and set to a prop named __css. We will transfer these styles to the packages/theme/src/components.
The code below is just an example and not the actual code.
packages/components/input/src/input-group.tsx
const css: CSSUIObject = {
- width: "100%",
- display: "flex",
- position: "relative",
  ...styles.container,
}

packages/theme/src/components/input.ts
baseStyle: {
  container: {
+   width: "100%",
+   display: "flex",
+   position: "relative",
  },
  field: {
    width: "100%",
    minWidth: 0,
    outline: 0,
    position: "relative",
    appearance: "none",
    transitionProperty: "common",
    transitionDuration: "normal",
    _disabled: {
      opacity: 0.4,
      cursor: "not-allowed",
    },
  },
  element: {
    color: ["blackAlpha.600", "whiteAlpha.700"],
  },
},

[!TIP]
In cases where the styles within the component are dynamically changed, there may be instances where it is not possible to transfer the styles to the theme.
In such cases, it needs to be discussed in this issue. Please feel free to send comments to this issue.

Additional Information
The @yamada-ui/notice component exists in packages/components/notice/src. You need to check all the components in this folder and if they contain styles internally, you need to modify them.
This change could have a significant impact on projects using Yamada UI, so we will work on it within the scope of v2.0.
Please create the branch for working on this issue from the remote branch v2.0, and direct your pull requests to the v2.0 branch as well.
Please select major for the changeset and create a pull request.

@108yen  Can I try this issue?

@108yen
Thanks!


Add "contact" field to New organization form

The current new organization form at /organizations/new does not currently include a contact address field. However, through #515 and related discussions we decided to change "organization email contacts" to be an optional,  single-valued field.
As a single-valued field representing a basic attribute of organization, I think it make sense to move this field from the current organization contacts location to a field editable through the "new organization" and "organization details" forms. This ticket focuses just on the new organization form accessed at /organizations/new

[ ] Add a new "contact" attribute to organization and include it in the new organization form, just above the icon input, as shown above. Details:

Field label: POD contact for this organization
Hint: How should other POD users contact this organization? Provide an email address or URL.

Do we really need to support a url pointing to a form or contact info or can we just use an email address?

We would like an email field. A separate URL field might be nice - to direct users the "right" part of the library website.
We plan to use a single email that goes to a support team, and not to an individual person.

I've sent out email to the pod-tech list asking whether anyone needs to use a contact URL for this, or whether a single email address is sufficient.  I'll see if I get any replies between now and Thursday's meeting, but at least for now I have not, and at least for Penn (per Bob) and for Harvard (per Corinna's comment above) a single email address would be fine.  I'll wait until after that meeting before making a definite decision, though.

After a suitable interval, I have not heard from anyone needing a URL contact field.  So I think a single email address should be fine for now.

I've answered the question above, but the implementation of the form addition needs to be done by someone else, so this issue should probably be reassigned to that person.

@JohnMarkOckerbloom I've unassigned you and queued this up for sprint 4

Relates to #545


use lengthLong() where possible

Version: 0.7.2
I am using TSNE on a Word2Vec model with 50000 x 100 shape. And this results into the exception:
java.lang.IllegalArgumentException: Length is >= Integer.MAX_VALUE: lengthLong() must be called instead
with shape 40000 x 100 it is working because the resulting Array length < Integer.Max
Params:
    val tsne = new BarnesHutTsne.Builder()
      .theta(0.0)
      .normalize(false)
      .numDimension(3)
      .perplexity(40)
      .useAdaGrad(false)
      .build()

i explicitley set theta to 0.0 to use the decomposed version.
see Gist for logs and stacktraces

So in other words, we're getting back to known question: what we're going to do with Long Indexing :)
@agibsonccc @saudet ?

Wait after NVIDIA? Does this use BLAS?

Yes, TSNE relies on gemm as everything in ml... :(

Note for the future, @crockpotveggies has had to use conversions HDF5 -> int indexes that should one day be removed : https://github.com/crockpotveggies/deeplearning4j/commit/cc2f63d116c5f02a970f383c86e70ec2b79b7f42

We should just get rid of int indexes :(
Closing this issue now, new one is opened: https://github.com/deeplearning4j/nd4j/issues/1662


Adding EC2ContainerCredentialsProviderWrapper to the credential providers

413

 Quality gate: Failed
6 Issues

 1 Vulnerabilities
 5 Code issues

Details

 Quality gate: Failed
6 Issues

 1 Vulnerabilities
 5 Code issues

Details


Custom prefix for subscription name in gcp pubsub

Since pubsub is a beast on its own, queues are in fact mapped to subscriptions with no ttl.
Currently, the subscription name is derived by concatenating fs2-queues- to the topic name.
It'd be good to make this customisable.

Done in #89


coveralls: Enable fips as it is disabled by default

The change of fips option to disabled by default is most probably the cause of the recent coverage drop:
https://coveralls.io/builds/39278631

I've also changed the coveralls to be run daily as the weekly runs are too coarse to identify the possible culprits.

This pull request is ready to merge

Merged to master. Thank you for the review.


Improve CSS Declarations.

[x] I have followed (at least) the PR section of the contributing guide.

I have removed all the unnecessary IDs with a single ID.

Related to #8


⚠ POST to /fact_check has degraded performance

In f95be91, POST to /fact_check (https://dev.factiverse.no/v1/fact_check) experienced degraded performance:

HTTP code: 200
Response time: 21021 ms

Resolved: POST to /fact_check performance has improved in 13ca3c5.


[Feature Request] To change in the configurations the default order as the movies/series are shown

When I access Streama either anonymously or with a user it always shows the movies/series in alphabetical order.
Can you make in the configuration an option to select how is the default sorting?
I would like the order to be first the most recent movies.

This can be done by editing the source and rebuilding it. Everything is in place to add some sort of "anchor" to have configuration settings.
I hope this can make it to a future update.
@archtron if you need help with this just PM me on discord.

Thank you @Ircas
Can you make a Pull Request with the changes you indicate?
I prefer that the Streama has it by default rather than having to be patching the program every time a new version is released.


Add an "offset-store" persistence subsystem.

This will be used to store the current "progress" of one application through another application's event stream. So it will be have a bit like a key/value store where the key is an application identity key and the value is an offset.
It will need OCC semantics, but rather than adding a "revision" field we can just use the existing offset value, so it will behave like a "compare-and-swap" operation, only setting the "new" value if you provide the correct "current" value.
There are a lot of moving parts to adding a new subsystem, but it's fairly well established at this point.

Add test suite to provider/internal/providertest
Design public interface in provider/subsystems/offsetstore
Implement the offset store for each of:

BoltDB
Memory
SQL (mysql, postgres and sqlite)

I strongly recommend a TDD approach. I start by defining the interfaces in the subsystem package, then start with one simple test (see provider/internal/providertest/datastore.go for example). I try to only move onto the next test once all of the provider implementations are working.

This may also be needed for #126.

Fixed in #148.


numbered it

Thank you for your pull request. Before we can look at your contribution, we need to ensure all contributors are covered by a Contributor License Agreement.
After the following items are addressed, please respond with a new comment here, and the automated system will re-verify.

User @rutvijraut isn't covered by a CLA. They will need to complete the form at https://identity.linuxfoundation.org/projects/cncf

Regards,
CLA GitHub bot

There is no need to use numbers instead of bullets here, and the numbered list is formatted incorrectly anyway.


Add mdman for generating man pages.

This introduces a new utility called mdman that converts a markdown-formatted document to a man page. This replaces asciidoctor, with the intent to make it easier to contribute, easier to have consistent formatting across platforms, and easier to generate plain-text documents for use on Windows (for #8456). This also includes a number of formatting fixes.
There is some documentation in the mdman/doc directory explaining how to use it, and the docs in src/doc/README.md have been updated (this explains the structure of the files). The Makefile has been replaced with a simple shell script.
CI has been updated to verify the checked-in docs are up-to-date. Perhaps in the future, these can be generated automatically (perhaps by build.rs?), but since that requires a bit of build system work (like upstream rust), this is deferred till later.

r? @Eh2406
(rust_highfive has picked a reviewer for you, use r? to override)

Thanks for taking a look!  And sorry it's such a big diff.  I did split it up into multiple commits to try to organize things a little. If any issues come up, I'll try to take care of them. I don't have anything else here, so:
@bors r=alexcrichton

:pushpin: Commit 566706e8632b7a3c3e93701283e3125df3c9f447 has been approved by alexcrichton

:hourglass: Testing commit 566706e8632b7a3c3e93701283e3125df3c9f447 with merge e9281208cdcf933ff24f44104aa7df06aa89d1a8...

:sunny: Test successful - checks-actions
Approved by: alexcrichton
Pushing e9281208cdcf933ff24f44104aa7df06aa89d1a8 to master...


Don't call runtime.GC() after compaction

Since we now memory-map the Head chunks and block loading has minimal overhead, there is no more requirement of calling runtime.GC() immediately after compaction. Hence remove it and let the normal GC cycle take care of it.

I will run prombench on this on Monday since I won't be able to check the results until then.

Sure, what's the harm of GC though? stopping world?

In the case of Cortex, when you have 100s of tenants (hence a TSDB per tenant), it leads to 100s of GC in short span making the entire compaction process very slow overall.

/prombench master

/prombench master

/prombench cancel

I see some continuous increase in load after a while Prometheus has been running and I doubt if it is related to this change

Looks to be largely in the noise, however both RSS and compaction duration seem a smidge higher with this change.

compaction duration seem a smidge higher with this change.

This one I dont understand why

Yeah, it's odd but it's also consistent.

And the RSS increase seems to sustain sometime in between 2 compaction and come to same level after the next compaction

The RSS increase doesn't make sense, we should reclaim that memory at the next natural GC. That's not consistent though, so could be an oddity of that particular cycle.


When upgrading RKE1 v1.24/v1.23 cluster with PSP objects to v1.25, clarify the error messages in the Advanced Options section

Is your feature request related to a problem? Please describe.

When upgrading RKE1 v1.24 with PSPs enabled, I note the following error messages when attempting to upgrade to v1.25:

The first error message is great; it is clear and explicitly tells me as a user what I need to do to get rid of this error message. The second message, however, is a bit unclear. Specifically, the wording of unreachable. This leads me to begin questioning what this could mean when I shouldn't have to do so.
We need to make the second error message just as clear as the first error message.
Describe the solution you'd like

The second error message in this scenario should say There are PSP resources existing in this cluster. Prior to upgrading, you must manually remove them.

/backport v2.7.2

Validated that this is addressed in v2.7-head https://github.com/rancher/rancher/commit/76c5c80bd1069bc14c8961014bd8b559b97bf22a.
ENVIRONMENT DETAILS

Rancher install: Docker
Rancher version: v2.7-head https://github.com/rancher/rancher/commit/76c5c80bd1069bc14c8961014bd8b559b97bf22a
Browser: Chrome

TEST RESULT

Scenario
Test Case
Result

1
As a standard user, provision a downstream RKE1 v1.24 custom cluster,  attempt to upgrade to v1.25 and validate that the warning message has been updated
:white_check_mark:

VALIDATION STEPS
Scenario 1

As a standard user, navigated to Cluster Management > Create.
Provisioned a downstream RKE1 v1.24 custom cluster with PSP objects enabled.
Validated that the cluster and nodes came up as Active in Cluster Management and Cluster Explorer.
Validated that the pods in the system project page are up and Active.
Navigated back to Cluster Management.
Edited the cluster configuration, updated the K8s version to 1.25 and noted the following

After toggling off PSP but not manually removing any PSP objects, I see the following:

Navigated to top-left menu > cluster > More Resources > Policy > PodSecurityPolicies.
Removed all existing PSP objects and navigated back to editing the cluster.
Validated that I see the following:

Disabling PSP and enabling PSA, I see the following as expected:

Reopening issue. After discussing offline further with QA, we agreed to do the following:

Remove the second banner altogether
Keep the first banner as it is very clear

The reason we came to this is because when you upgrade RKE1 v1.24/v1.23 to v1.25 with PSPs enabled, you are not prevented from upgrading. In fact, the upgrade will remove any PSP resources in the cluster. As such, the second banner is misleading and we should remove it.
The first banner message is good as it states PSPs are not removed and when you switch to v1.25, the error message goes away as illustrated in the previous test results.


Enable adaptive text

Experiments with adaptive text. This is currently very costly to render, so resizing the panel can get choppy for a large number of values.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@marcusolsson

This is currently very costly to render, so resizing the panel can get choppy for a large number of values.

We can consider setting a limit of how many values we are willing to show adaptive text for.
Perhaps, a parameter on for the panel.


Adding a parameter for the Widget event

Hey! Very cool library
There is a proposal in Widget to add the name of the called object to the callbacks. Thus, it will be possible to dynamically create fields and manage all with one callback)
https://github.com/peterbrittain/asciimatics/blob/770a097cb532cc1fdbcf5f7889bfcdeeddd92b3c/asciimatics/widgets/text.py#L184-L185
adding only one parameter
if old_value != self._value and self._on_change:
            self._on_change(self)

Thank you!

Thanks.
Just a quick question: why not use a method on the object for the callback instead of passing in a separate function (that then needs the reference to the object to interact with it)?

Probably because I'm just learning Python)
Are you talking about such a decision?
def __init__(self, screen, config: ConfigModel):
...
        for i in range(0, 10):
                layout.add_widget(Text("Text " + i+":", "txtBox_" + i, on_change=self._change_txtBox))
...
  def _change_txtBox(self):
        self.save()
        print(self)

then it will return a Frame object

Thanks.
Just a quick question: why not use a method on the object for the callback instead of passing in a separate function (that then needs the reference to the object to interact with it)?

Ok... I see now.  I suspect that this comes down to the wider state you want to enforce in the callback.  Typically, I'd expect you to want to handle state across multiple objects and so passing the changed widget is of little benefit.
Looking at your example, if you just need to enforce limits on the entered text, why not use a validator?  See forms.py   for an example of how to use them.
However, there are some other use cases you might need.  LMK if you have something else in mind and I'll see how we can tweak something...

Did the validator give you what you need?  I'll leave this open a few more days for you to experiment and then close if I hear nothing...

I'm assuming the validator did the trick.  Please feel free to re-open if not...


(client) certificate chain verification - rooted in CA?

In the client configuration, we may have a certficate. This is not checked to form a certificate chain rooted in a known CA.
It is still unclear (TODO: check openvpn behaviour) what the intended chain is (and whether it must be complete):

the --ca may point to a set of CA certificates
the --capath may point to a set of CA certificates
the --extra-certs may point to another set of CA certificates

What is unclear, whether the client must know the CA its certificate has been issued by. We should figure that out from reading code, but it has a rather low priority (the server will reject our connection attempt if the cert is not known by it).
This has been discussed and worked on in #32 #34 #79

I'm closing this -- it may re-appear if there's need for extra-certs in a concrete configuration, but working this out right now has no priority.


Exception when opening DrawPolygons process in vvvv 2022.5 Preview

Kindly reported by chk via vvvv elementa chat originally.
Exception here
https://pastebin.com/UHKXX0YE
Looks like something to do with method declarations that changed in the upgrade to .net 6.

Note to self: I should also check if the changes that I make for 2022.5 will break VL.Polytools for 2022.4 and put an appropriate note in the changelog

Seems resolved with the preview 2022.5 260 but I will wait until a 2022.5 Release Candidate to close this issue


Support common Quarkus errors

In REST, there are common errors such as 404/NOT_FOUND, 401/UNAUTHORIZED etc.
Quarkus itself is handling these errors. This can be seem in the package io.quarkus.resteasy.runtime, where some javax.ws.rs.ext.ExceptionMapper are implemented (ForbiddenExceptionMapper for example).
When using this quarkus-problem-extension, there is still not a "problem" returned in case one of these common errors occurs:
$ curl -v http://localhost:8080/notpresent
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to localhost (<IP_ADDRESS>) port 8080 (#0)
> GET /notpresent HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Length: 0
< Content-Type: application/json

Feature request: In case of a common error, return a "problem", too.

fixed with 268cb33057246601fcf4ec529f51c85e0d387a26


Labels and messages are displayed to late

puilabels and puimessages aren't displayed immediately, but one key stroke to late. In the case of labels, they aren't shown at all until the user starts typing.
This may be an AngularJS 1.3 problem.

Fixed.


About local editors can not automatically restart？

The project cannot be automatically refreshed after editing the local file. I wonder if it is a mounting problem.

The local environment is Windows 10, docker installed in the Linux subsystem. Is there a better solution?