Document backbone output stride and minimum input size for DeepLabV3

📚 The doc issue
Currently, the documentation for DeepLabV3 (segmentation) doesn't state both the minimum input size[^1] and the output stride of the backbone network, as configured by the original contributor in #820. Both these pieces of information are interlinked with the choice of atrous rates (see background) and vital for end-users in configuring and effectively using this class of models for their specific needs.
Many use off-the-shelf models, more generally so without concerning themselves with the details of the underlying architecture. For some, this may work well, but there are cases where such details (incl. implementation caveats) can help end-users make well-informed decisions that impact their results; one such case related to medical imaging is detailed later in the issue. This issue covers a fair few technical details in explaining why the aforementioned pieces of information are vital. Perhaps these can make their way into the documentation where deemed relevant.

Background
Output Stride
The term output stride originates from the DeepLabV3 paper (Link). The authors note that traditional deep convolutional networks aggressively decimate valuable, detailed information in feature maps through repeated striding or pooling operations. To quantify this decimation, they introduce the notion of output stride, defined as the ratio of the input spatial resolution and the final backbone-encoded feature map resolution; in Fig. 1, the output stride is 1024/64 = 16. Models with a low output stride tend to produce higher quality, finely detailed segmentation masks as there's more information available for reconstruction (upsampling stage). However, such models are also more resource-intensive due to the comparatively larger intermediary activations.
Atrous (Dilated) Convolutions

Instead of using strided transposed convolutions for reconstruction (like UNets), DeepLabV3 uses an atrous spatial pyramidal pooling module, at the core of which lies three atrous convolutions configured with different atrous rates; the convolutional layers in question form an inverted triangle in the architecture diagram visualised in Fig. 1. Atrous convolutions facilitate a larger receptive field without aggressively downsampling the feature map (unlike striding) by poking holes through convolutional kernels to inflate their size, as visualised in the figure on the right; the receptive field is demarcated by dotted vertical markers. Varying the atrous or dilation rate allows the capture of information from a larger spatial context. The ASPP module, thus, is architected to learn multi-scale contextual information. The choice of atrous rates impacts the module's effectiveness in capturing said information.

DeepLabV3, as initially proposed, uses atrous rates 6, 12, and 18 when the output stride is 16; these rates are doubled for an output stride of 8[^2]. The authors mention that its crucial to choose atrous rates according to the output stride and feature map resolution. There's a good reason for this: when the dilation/atrous rate grows, kernel parameters, except the central one(s), spend a growing fraction of their time "observing" the zero-padded region rather than the feature region. In the extreme cases that this rate grows to or beyond the feature map size, the convolution degenerates to a pointwise one, as visualised in the figure on the left (6-padded, 6-dilated 3×3 conv. on a 5×5 map).
Figure 1

Now, DeepLabV3 (Google Research) was originally written in TensorFlow (Link), and the Torchvision port (Link) configures the network with hardcoded atrous rates 12, 24, and 36. Following the guideline noted in the paper (Link), the backbone networks should be configured with an output stride of 8[^3]. So, the minimum input size to avoid the atrous convolutions from degenerating to pointwise ones is 36 × 8 = 288[^2].
In echonet/dynamic and echonet/lvh, the authors use DeepLabV3 (ResNet-50 backbone) via Torchvision with input images having a spatial resolution of 112×112; that's more than 2x lower than the minimum. With an output stride of 8, the backbone-encoded feature map resolution is 14×14. Considering the chosen rates, all three atrous convolutions in the ASPP module, each with 4.71 million parameters and together accounting for 35.7% of the model's total parameter count (39.6M), are completely ineffective! This is backed up via an experiment of low-magnitude weight pruning (see sidetrack).
Furthermore, DeepLabV3 was designed for segmenting objects at multiple scales (e.g., cars in traffic). As such, it's inherently ineffective on echocardiograms (fairly standardised), where the segmentation target is a single, continuous blob in roughly the same position for every sample, not differing much in scale. Any researcher who neither has the time to dive deep into architectural details nor the experience to navigate and understand source code in intricate codebases may have easily missed these critical requirements.

Sidetrack
As nearly all of the kernel parameters of the three atrous convolutions only observe zeros, their weights are bound to be tiny and much less likely to influence the output. It's easy to verify this using globally scoped L1-norm unstructured weight pruning. Figure 2 showcases parameter sparsity diagrams of the entire network after pruning at different sparsity targets; these diagrams were obtained by taking all the network parameters via the following and reshaping the resulting vector into a square-ish matrix:
torch.nn.utils.parameters_to_vector(model.parameters(recurse=True)).detach()

Figure 2

The 50% sparse case is the most interesting. The distinctive characteristic of this subfigure is the missing band of parameters, which, upon further investigation, was identified to belong to the three atrous convolutions in question. Manually changing the atrous rates to 2, 4, and 6 in an effort to adapt to the low backbone-encoded feature map resolution[^4] results in a much denser band for the same sparsity target, as seen in the figure on the right. However, presumably due to the aforementioned lack of variety in scale, there wasn't a notable performance difference.
Unrelated yet interesting is the compression rate. The 98.44% sparse case is within 1% of the baseline's DICE score while only having 633K NNZ parameters. Applying channel pruning via Microsoft's NNI results in a model with 34.5K parameters (1148x smaller) that's within 4% of the baseline. Simply choosing an off-the-shelf model without considering task complexity is analogous to using a truck to transport a single grape.

[^1]: The Torch hub model card (Link) makes a remark on this. However, not only is it easy to miss, there isn't any reasoning to explain why the height and width of the input are expected to be at least 224.
[^2]: The DeepLabV3 authors assume that with a low output stride of 8, the backbone encoded feature map resolution is high enough to use larger atrous rates 12, 24, and 36. This expectation sets a minimum constraint on the resolution of the input image: 36 * 8 = 288.
[^3]: Torchvision supports three backbone networks: ResNet-50, ResNet-101 and MobileNet-V3 (Large). While the ResNets are correctly configured with an output stride of 8, MobileNet-V3 (Large) is configured with an output stride of 16. This looks to be an implementation error and will be raised shortly via another issue.
[^4]: Perhaps it might be useful to allow end-users to configure the atrous rates for added flexibility in use cases, such as the one presented, where the input resolution is much smaller than the minimum (determined by the default hard-coded rates). The same applies to the converse where the input is quite large; gigapixel images are common with certain imaging modalities in healthcare settings.
Suggest a potential alternative/fix
The proposal is to update the documentation with a callout card that makes the requirements for effective use clear. It'd also be great if the documentation highlighted relevant characteristics of the underlying architecture (e.g., "works best for inputs where the segmentation target(s) differ in position and scale"), which would serve as a concise guideline for end-users.

Thanks for the report @lambda-shuttle .
Could you please submit a PR for the changes you're proposing? We can either add a short sentence to the models docstrings, or perhaps we can also document the weight enums (although the doc rendering is broken right now :/ https://github.com/pytorch/vision/issues/7992)
BTW, we currently document the min size of the model (not the backbone) to be (1, 1). Are you able to confirm that this is still correct, or does this need to be fixed as well?


[$500] Room-After 2nd WS created, from WS selection moves up to public in visibility page

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.35
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: applause-Internal Team
Slack conversation:
Action Performed:

Go to https://staging.new.expensify.com/

Login into new account

Tap profile icon --- Workspaces

Tap new workspace

Navigate back to LHN

Tap fab--- start chat--- room

Tap visibility

Note, with one workspace created, opening create room visibility section shows, public selected and WS option with tick mark

Navigate to LHN

Tap profile icon---Workspaces

Tap new Workspace and create second workspace

Navigate to LHN

Tap fab--- start chat--- room

Tap visibility

Note after creating second Workspace, from WS option selection moves up to public

Expected Result:
With any number of workspaces created, the visibility page behaviour must be same.
Actual Result:
Login as new user, with 1WS created, opening create room visibility section shows, public selected and WS option with tick mark. Navigating back and create 2nd WS, in this case, from WS option selection moves up to public is shown.
Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/Expensify/App/assets/115492554/664e9ce8-6f37-40fd-98d6-1fcf8a881925

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01fb13ef2922bb85d7
Upwork Job ID:<PHONE_NUMBER>996166144
Last Price Increase: 2024-02-02
Automatic offers:

Ollyws | Reviewer | 28143074
bernhardoj | Contributor | 28143075

@izarutskaya The video is not вуыскшиуе Actual Result: . I don't see the selected option change after adding the second WS in the video

Proposal
Please re-state the problem that we are trying to solve in this issue.
Opening the visibility selection page of the new room shows the first option (Public) as highlighted even though the selected is the last (Workspace) option.
What is the root cause of that problem?
The highlight moves to the top because whenever the section object is changed, it will scroll and highlight the first option.
https://github.com/Expensify/App/blob/cf8279f28ca69e84bf0001c7d8206e1037ef014a/src/components/SelectionList/BaseSelectionList.tsx#L356-L361
In our case, the section data itself never changed, but we pass an inline array value to the props, so it's recreated every time the ValueSelectorModal is re-rendered.
https://github.com/Expensify/App/blob/cf8279f28ca69e84bf0001c7d8206e1037ef014a/src/components/ValuePicker/ValueSelectorModal.js#L71-L72
And one of the cases where it re-renders is when we open it, the modal visible state becomes true.
What changes do you think we should make in order to solve the problem?

Memoize the section so it only changes when it really changes.
https://github.com/Expensify/App/blob/cf8279f28ca69e84bf0001c7d8206e1037ef014a/src/components/ValuePicker/ValueSelectorModal.js#L45-L50
So, instead of using the useEffect, we can do it like this:
const sections = useMemo(() => {
    const itemsData = _.map(items, (item) => ({value: item.value, alternateText: item.description, keyForList: item.value, text: item.label, isSelected: item === selectedItem}));
    return [{data: itemsData}];
}, [items, selectedItem]);

sections={sections}

This is a pretty minor bug, but the fix looks straightforward so we may as well get it taken care of. Assigning @bernhardoj

PR is ready
cc: @Ollyws

All paid up


[VS Code] [Audio] HTTPS request failed for SkypeToken: Parse Error: JS Exception

Error:
HTTPS request failed for SkypeToken: Parse Error: JS Exception
Steps to Reproduce:
1.
2.

Version Data

extensionName
VSLS-Audio

extensionVersion
0.1.84

applicationName
VSCode

applicationVersion
1.47.3

platformName
Windows

platformVersion
10.0.19041

Thanks for opening this issue, @osamashadwo. I'm going to close it as a duplicate of  #3514. Can you upload your logs there and join the conversation? You can access them via the Live Share: Export Logs command from the command palette (ctrl + shift + p) and attach them to #3514.


fix: some errors in fillhandler

fix: When there is no content in the cell, the fill handle selection is missing.
fix: Incorrect orientation of the selection area of the filling handle in the lower part of the cell
fix: When filling the handle to select areas left or up, the selection area is lost
fix: Error copying content after filling handle selection when containing empty cells

LGTM

LGTM


Add intent for modern Android browsers

For modern Android browsers, we need to tag them with intent:// instead of the usual app URI path, else they do not work and just proceed randomly in the Google Play Store.

Superb. Thanks!


Fix indexer tests on macOS [phab:D1314]

Created by shahms at 2017-02-07 20:18:23:
Linking is hard and OS X is "different"

salguarnieri commented inline (context lost during migration) at 2017-02-07 20:36:42:
If this file comes from somewhere, can we record that in a comment at the top or in a README.md?

salguarnieri wrote at 2017-02-07 20:36:42:
Nice work!! 🙌

salguarnieri accepted this revision at 2017-02-07 20:36:42

shahms wrote at 2017-02-07 20:40:16:

Add comment on original source of Bazel files.

shahms commented inline (context lost during migration) at 2017-02-07 20:40:24:
Done.

shahms closed via commit 3b748f0fa1474d2f0df8366c5c9876bb0b95ca21 at 2017-02-07 20:41:07


[BACKLOG-14636]  Updates to support engine adapters

config stored with Operations now has the string xml from the stepmeta,
TransEngineAdapter was still expecting the StepMeta object.
short circuit .putRow in the adapter when no conversionManager is
available.

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -f engine/build.xml checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/ael/adapters/StepInterfaceEngineAdapter.java
  engine/src/org/pentaho/di/trans/ael/adapters/TransEngineAdapter.java
  engine/src/org/pentaho/di/trans/step/StepMeta.java

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory

Branch Change: -7.14%:small_red_triangle_down:
Complexity Change: -7.69%:small_red_triangle_down:

org.pentaho.di.core.injection.bean.BeanInjector

Branch Change:  + .84%
Complexity Change:  + 1.39%

org.pentaho.di.job.entries.ftpsget.JobEntryFTPSGet

Branch Change: -.53%:small_red_triangle_down:
Complexity Change: -.56%:small_red_triangle_down:
Instruction Change: -.07%:small_red_triangle_down:

org.pentaho.di.job.entries.getpop.MailConnectionMeta

Branch Change: -1.00%:small_red_triangle_down:
Complexity Change: -1.41%:small_red_triangle_down:
Instruction Change: -.30%:small_red_triangle_down:

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change:  + 3.28%
Line Change:  + 3.87%

org.pentaho.di.trans.ael.adapters.StepInterfaceEngineAdapter

Complexity Change: -1.96%:small_red_triangle_down:
Instruction Change: -.94%:small_red_triangle_down:
Line Change: -1.73%:small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Complexity Change: -.22%:small_red_triangle_down:
Instruction Change: -.87%:small_red_triangle_down:
Line Change: -.96%:small_red_triangle_down:
Method Change: -.54%:small_red_triangle_down:

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change:  + .10%

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change: -.05%:small_red_triangle_down:

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Instruction Change:  + .09%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change:  + 1.04%
Complexity Change:  + 1.05%

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta

Instruction Change: -.05%:small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change:  + 1.18%
Complexity Change:  + 1.35%

org.pentaho.di.trans.steps.pgbulkloader.PGBulkLoaderMeta

Branch Change: -.88%:small_red_triangle_down:
Complexity Change: -.93%:small_red_triangle_down:

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Instruction Change:  + .08%

License header violations

:heavy_exclamation_mark: Copyright year is not the current year on engine/src/org/pentaho/di/trans/step/StepMeta.java. Found 2016. Was expecting 2017

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -f engine/build.xml checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/ael/adapters/StepInterfaceEngineAdapter.java
  engine/src/org/pentaho/di/trans/ael/adapters/TransEngineAdapter.java
  engine/src/org/pentaho/di/trans/step/StepMeta.java

Newly Fixed Tests:
org.pentaho.di.trans.steps.syslog.SyslogMessageConcurrentTest.concurrentSyslogMessageTest:
:large_blue_circle: org.junit.runners.model.TestTimedOutException
org.junit.runners.model.TestTimedOutException: test timed out after 2000 milliseconds
	at sun.misc.Unsafe.park(Native Method)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1465)
	at org.pentaho.di.trans.steps.syslog.SyslogMessageConcurrentTest.concurrentSyslogMes

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory

Branch Change: -7.14%:small_red_triangle_down:
Complexity Change: -7.69%:small_red_triangle_down:

org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics

Branch Change:  + .39%
Complexity Change:  + .51%
Instruction Change:  + .08%

org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent

Branch Change: -1.01%:small_red_triangle_down:
Complexity Change: -1.06%:small_red_triangle_down:
Instruction Change: -.17%:small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change:  + .66%
Complexity Change:  + .93%
Instruction Change:  + .13%

org.pentaho.di.trans.ael.adapters.StepInterfaceEngineAdapter

Complexity Change: -1.96%:small_red_triangle_down:
Instruction Change: -.94%:small_red_triangle_down:
Line Change: -1.73%:small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Complexity Change: -.22%:small_red_triangle_down:
Instruction Change: -.87%:small_red_triangle_down:
Line Change: -.96%:small_red_triangle_down:
Method Change: -.54%:small_red_triangle_down:

org.pentaho.di.trans.steps.combinationlookup.CombinationLookupMeta

Instruction Change: -.04%:small_red_triangle_down:

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change: -.10%:small_red_triangle_down:

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change:  + .05%

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Instruction Change:  + .09%

org.pentaho.di.trans.steps.dbproc.DBProcMeta

Instruction Change:  + .11%

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Branch Change: -.31%:small_red_triangle_down:
Instruction Change: -.06%:small_red_triangle_down:
Line Change:  + .23%
Method Change:  + 1.02%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change:  + 1.04%
Complexity Change:  + 1.05%
Instruction Change:  + .14%
Line Change:  + .37%

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -2.35%:small_red_triangle_down:
Complexity Change: -2.70%:small_red_triangle_down:
Instruction Change: -.18%:small_red_triangle_down:
Line Change: -.48%:small_red_triangle_down:

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change:  + .07%

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change:  + .10%

org.pentaho.di.trans.steps.synchronizeaftermerge.SynchronizeAfterMergeMeta

Instruction Change:  + .04%

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -.91%:small_red_triangle_down:
Complexity Change: -.82%:small_red_triangle_down:

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.webservices.WebServiceMeta

Branch Change: -1.79%:small_red_triangle_down:
Complexity Change: -1.19%:small_red_triangle_down:
Instruction Change: -.07%:small_red_triangle_down:


Add an optional -l, --lang, and/or \l flag feature

For example, --lang fr would mean that the HTML documents are in French, and would include , while -l pt-BR would mean the text is using Brazilian Portuguese: . By default, use en-CA for Canadian English.

closed by (e59061b)


Gallery Submission Feedback - wait_for_update should be at least 100ms

When reviewing your template, we've noticed that wait_for_update was set to a default less than 500 milliseconds. When specifying this it's a best practice to give some buffer time to attempt to allow for the updateConsent command to be fired. This is very important when loading the consent management script dynamically.

Hi @yeeha567 i have updated default value to 200ms kindly review now.
Thank you

@yeeha567 how long it will take to get approved and listed in Google tag manager template gallery :)


Web-console: Add actions column to segments view

Add actions column to segments view. the wrench now gives the user the ability to drop a segment using the /druid/coordinator/v1/datasources/${terminatetDatasourceId}/segments/${terminateSegmentId} endpoint. The magnifying glass now shows both metrics and  dimensions in the table action dialogs component.
Fixes #7897

Actions should appear here:

The 'action' on segment is Drop segment (disable) not drop data source


Looping ssh connection Timeout

Hey, i was wondoring if it could possible to do that, because i'm encounring difficuties.
The first connection connects successfully but all subsequent connections return an error on session.connect()
if i do session.connect().unwrap() :
thread 'main' panicked at 'called "Result::unwrap()" on an "Err" value: Error: { Kind(Timeout), Message(time out.) }
{ Kind(Timeout) }
loop {
  let mut session: Session = ssh::create_session();
  session.set_timeout(3);
  session.set_user_and_password("user", "password");
  match session.connect("<IP_ADDRESS>:22") {
         Ok(_res) => {
              //Do something
         },
         Err(_err) => {
              //handle error
        }
  }
session.close().unwrap();
}

Thank you

Same issue here with ssh-rs = "0.2.0".
Not using it in a loop but I get these timeout messages as well.
Bad packets received in my SSH server logs.

Sorry, I have a problem with my code and I need to fix it, thank you very much for your feedback.

No problem, thank you for your work ! Keep us in touch :)

Yeah no problemo amigo, nice work anyway :)

Did you plan to release an 0.2.1 version ?
Thank you for your reactivitie

Yes, it is expected to be released in the next few days.

v0.2.1 released

It seems to work for me, thank you !

Updated, thank you !


Bug: parameters handled correctly when using quotedStringSupport with custom usageDelim

When quotedStringSupport is set to true and a command uses a usageDelim that is not a space, commands with the repeator tag do not work correctly. (Tested with user type).
Expected Behaviour
Klasa would get all users and send an array of users.
Actual Behaviour
When I input one user, it works perfectly, but it fails with two or more.
Command input
> $test<PHONE_NUMBER>75937034,<PHONE_NUMBER>11808769
Error: person must be a mention or valid user id.
> $test "383383927575937034" "242043489611808769"
Error: person must be a mention or valid user id.
> $test "383383927575937034", "242043489611808769"
Error: person must be a mention or valid user id.

Code to reproduce
const { Command } = require('klasa');

module.exports = class extends Command {

	constructor(...args) {
		super(...args, { usage: '<person:user> [...]', usageDelim: ', ' });
	}

	run(msg, [...user]) {
		return msg.send(`Mentioned: ${user.map(us => us.tag).join(', ')}`);
	}

};

Extra details
When quotedStringSupport is set to false, it does parse the ids correctly.

When using usageDelim: ', ', this behaviour happens:
> %args<PHONE_NUMBER>75937034,<PHONE_NUMBER>11808769
[ '383383927575937034', '<PHONE_NUMBER>11808769' ]
> %args "383383927575937034", "242043489611808769"
[ '383383927575937034', ' ', '242043489611808769' ]

As noticed above, the first is adding a space before the second parameter instead of being removed (since usage delimiter is comma + space, and both parameters are separated by that); and the second one is adding an extra parameter with is a single space. Expected input for both should be:
[ '383383927575937034', '242043489611808769' ]


idea: respond with json provided by a file

It would be great if a json file could be provided to the docker image which will be used to send as the response body for all requests. With that, we can control which response data is returned without having to send the json payload in the body of the request.

I've pushed a test tag to Docker Hub, issue-67-68, if you can test it that would be great but I'll merge it later.
Set the environment variable to the file path, the file needs to be mounted in the /app directory.
docker run -d --rm -v ${PWD}/test.html:/app/test.html -p 8080:8080 -e OVERRIDE_RESPONSE_BODY_FILE_PATH=/test.html -t mendhak/http-https-echo:issue-67-68

Tag :34 is now pushed to Docker Hub and Github


Use local vars and execute command for complex watch/repl expressions

The debug protocol does not currently support complex expressions in the REPL or watch panel. To work around this, we should do the following:

Whenever a hover/repl/watch request comes in, identify whether the expression is a simple variable expression (alpha, alpha.beta, alpha[0], alpha[0].beta, etc...), or a more complex expression such as function calls, string concatenation, etc.
For all complex expressions, assign to a single local variable (perhaps call it __rokuDebug__evalVars) which will contain a collection of evaluated expressions. It'll end up looking something like this:

__rokuDebug__evalVars = {}
__rokuDebug__evalVars.expression1 = doSomething()
__rokuDebug__evalVars.expression2 = 1 + 2 + 3
__rokuDebug__evalVars.expression3 = "alpha" + str(3)

We need to figure out of these variables should be purged on every debug step, or if they should live indefinitely for the lifetime of the function scope/application.

@TwitchBronBron shouldn't that variable be an array so that there is zero type conversion from the expression based on the discovery we did?

No, based on our discovery a few weeks ago both arrays and AAs properly support the zero type conversion issue, as long as you do the assignment separately from the AA/Array declaration. Also, if we use an array, then we have to maintain array indexing in the debugger, whereas arbitrary AA keys can be auto-generated in the debugger without needing to know what index we're at.

So, you're right. I mis-remembered that AAs lose the proper type. Yeah, then we need to use arrays instead. Ugh, that complicates this a bit because now we need to keep the array length in sync in the roku-debug side of things.


Introduce vpn connect and disconnect commands

Please add two furyctl subcommands to make it simpler to connect and disconnect from the configured vpn:
furyctl connect openvpn furyctl.yaml
furyctl disconnect openvpn furyctl.yaml

The PR https://github.com/sighupio/furyctl/pull/329 introduces furyctl connect openvpn with foreground process


Remove last activity limitation for unassigning inactive people from bugs

To avoid adding hundreds of needinfos to triage owners, run the script weekly and add a limit on the number of needinfos.
Fixes #266

I did a small refactoring to minimize the diff, but it's the same behavior as the first iteration of the patch.


PHP-Side limitations exists?, permanent optimizations might needed on high-load environments?

Im fastinated by the Workerman benchmarks, im wondering how to achieve them, respectively how they was setup?
As we know for example, Apache Worker/MPM same as PHP-FPM in NGINX have always some sorts of Limits regarding how much requests it can handle.
I never really used PHP as "direct" Web Server daemon that much.
As Workerman uses PHP (CLI), are there any limitations or tweaks necessary other then "solid" Hardware to reach the probably maximum possible requests/performance?

If you only use the http protocol, the PHP Cli side does not need to do any configuration.
If you use the websocket protocol, you need to install the event extension for PHP Cli so that the workerman can hold more connections(10000 connections or more).

Thank you walkor,
I was thinking the same but wasn´t sure, this make me think about why to even using webservers for PHP code those days, they just give many people out there headaches all the time, especially on high load envs, where you have to endless tweak settings... Adding more CPU or RAM is a way easier and works longer then anything else...
I will slowly port as much of my code code as possible to workerman :-)


Implement unit tests for the network utility code

This should provide 100% coverage for the file.

/botio unittest

/botio unittest


Don't work

I don't know the origine of the cause, but it don't work with my custom theme.
Screenshoot1 without the [gallery] :

Screenshoot2 with the [gallery] :

HTML code that is generated looks fine.
What version of the plugin are you using?
Is Javascript enabled in your browser?
What happens when you add more images to the gallery?
What happens when you decrease the row height like so: [gallery rowHeight=100]?

No response from @softyoda . So closing this.


ERROR in Error encountered resolving symbol values statically. Only initialized variables and constants can be referenced

I get below error when I try to use latest angular version with this api.
ERROR in Error encountered resolving symbol values statically. Only initialized variables and constants can be refere
nced because the value of this variable is needed by the template compiler (position 8:22 in the original .ts file),
resolving symbol SEED_DATA in /Users/jaichandra/Documents/Work/repositories/angular2-polymer-quickstart/node_modules/
angular2-in-memory-web-api/in-memory-backend.service.d.ts, resolving symbol SEED_DATA in /Users/jaichandra/Documents/
Work/repositories/angular2-polymer-quickstart/node_modules/angular2-in-memory-web-api/index.d.ts, resolving symbol Ap
pModule in /Users/jaichandra/Documents/Work/repositories/angular2-polymer-quickstart/src/app/app.module.ts, resolving
 symbol AppModule in /Users/jaichandra/Documents/Work/repositories/angular2-polymer-quickstart/src/app/app.module.ts

I was using an old version of the api. This error is gone with latest version.


chore(blockifier): add flag run_native

This change is

#2092  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

crates/native_blockifier/src/py_block_executor.rs line 127 at r4 (raw file):

    pub block_compilation: bool,
    pub global_contract_cache_size: usize,
}

What does block_compilation mean?
Why add the global_contract_cache_size here?
We are going to have 4 caches and a queue. All with size limits. Shouldn't we wait for the struct to be finalized before deciding on the config for it?
Code quote:
pub struct MeshiAviConfig {
    pub run_cairo_native: bool,
    pub block_compilation: bool,
    pub global_contract_cache_size: usize,
}


Azure DNS currently doesn't support purchasing of domain names

is this part in the document correct ?
"Azure DNS currently doesn't support purchasing of domain names. If you want to purchase a domain name, you need to use a third-party domain name registrar. The registrar typically charges a small annual fee. The domains can then be hosted in Azure DNS for management of DNS records. See Delegate a Domain to Azure DNS for details."
Azure already provide the purchase option for domains using app service domains , so not really sure if the above is accurate or it means something else ?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 38109b47-d859-83f2-3a46-e207cfcc5c3f
Version Independent ID: fb0c04d2-4f39-1a5e-7666-d20a16a582bd
Content: DNS Zones and Records overview - Azure DNS
Content Source: articles/dns/dns-zones-records.md
Service: dns
GitHub Login: @greg-lindsay
Microsoft Alias: greglin

@abdohanafy , thank you for your query. We'll review this and get back to you soon!

@abdohanafy , thank you for your feedback.
The statement "Azure DNS currently doesn't support purchasing of domain names" is correct, as Azure DNS and Azure App service domains are 2 different services and Azure DNS service doesn't support purchasing domain names as of today but only supports hosting your domains in Azure with record management.
However, the part you highlighted seems to be outdated as it doesn't refer to the Azure app service domain purchase option, so I've updated the doc to include the same as below:
Azure DNS currently doesn't support purchasing of domain names. For an annual fee, you can buy a domain name by using App Service domains or a third-party domain name registrar. Your domains then can be hosted in Azure DNS for record management. For more information, see Delegate a domain to Azure DNS.
This update will go live soon.


Mac build

this allow to build.on Mac OS
it also an autogenerated version.hpp file for the version numbers.

when compiling on Mac OS with non-standalone mode I got:
Undefined symbols for architecture x86_64:
"_Gopt", referenced from: [...]

which is solved by defining struct program_option_struct Gopt; which is defined original in option.hpp as extern.

when compiling on Mac OS with non-standalone mode I got:
Undefined symbols for architecture x86_64:
"_Gopt", referenced from: [...]

which is solved by defining struct program_option_struct Gopt; which is defined original in option.hpp as extern.

@gonuke @pshriwise @ljacobson64 this should be ready to be reviewed.

@gonuke I think I addressed all your comments.
(@pshriwise @gonuke this is the next PR to get reviewed/merged in the Trellis-plugin GitHub Action chain.)

In the absence of CI, I'll merge this for testing downstream


docs(style-props): Make Filter examples live and editable

Subject
Style Props
Description
Make the Filter examples in the Style Props page live and editable. This would include providing a bit more rich examples in which you can see the effect of the filter props.

Hey @nikolovlazar!
May I attempt to update this section?

Sounds good, @TylerAPfledderer!


Improved progress bar

Task/Issue URL: https://app.asana.com/0/856498667320406/1117573875936431
Tech Design URL:
CC:
Description:
Improvements to progress bar:

It is now moved above Omni Bar.
Smooth animation on update.
Gradient animation for 'filled' part of the progress bar.

Steps to test this PR:
Regular scenario:

Open new tab.
Start loading webpage.
Look out for any glitches/issues around progress bar.

Redirect:

Open new tab.
Navigate to page that redirects to another domain, e.g. bbc.co.uk
Look out for any glitches/issues around progress bar.

Navigating across tabs:

Open new tab.
Navigate to page that loads slowly.
Open another tab, load another web page there.
Progress bar should reflect actual progres when switching between tabs.

Clearing data:

Open new tab.
Navigate to page that loads slowly.
Use fire button to clear data.
Progress bar should disappear.

Gradient animation checks:

Open new tab.
Navigate to page that loads slowly.
Open privacy protection dashboard, then close it OR put app to the background, then move back to it.
If visible, progress bar gradient should be animating.

Internal references:
Software Engineering Expectations
Technical Design Template

@brindy Thank you!


Remove MBEDTLS_ENABLE_WEAK_CIPHERSUITES configuration option

Fix #4416

Yes I was thinking just the same.

I've added the change log and migration guide.

@chris-jones-arm @TRodziewicz @gilles-peskine-arm Please have a look to the last version addressing Gilles' last comment.


tf.keras.layers.PReLU outputs NaN on positive input

Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
2.15.0
Custom code
Yes
OS platform and distribution
No response
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
Following the documentation, https://www.tensorflow.org/api_docs/python/tf/keras/layers/PReLU, tf.keras.layers.PReLU should  output the input if the input > 0 regardless of the weight. However, it outputs NaN when weight is infinity although input > 0.
I think the issue is located here:
https://github.com/tensorflow/tensorflow/blob/90d97b99797892f3d571f27076e4fcdfb1bde059/tensorflow/python/keras/layers/advanced_activations.py#L164
, where the calculation inf*0 happens.
According to the documentation, I think it would be better to directly return input if input > 0.
Also, I quickly checked the PyTorch's implementation and I find that pytorch use the following code to do prelu:
(input > scalar_t(0)) ? input : weight * input;

https://github.com/pytorch/pytorch/blob/3bc2bb678150e02bfa2552b82572d771a892d0c3/aten/src/ATen/native/cpu/Activation.cpp#L1380
Standalone code to reproduce the issue
import numpy as np
import tensorflow as tf
input = tf.constant(1.1234, dtype='float32')
weight = tf.constant(np.random.randn(1)*np.inf, dtype='float32')
out = tf.keras.layers.PReLU(weights=weight)(input)
print(out)

### Relevant log output

_No response_

Hi @maybeLee, I tried to reproduce the issue, from the given constraint/dummy code, and i am able to get the error output as

and after spending some time over your issue, got to know, this issue may be arised due to mismatch in the package version.

HI @AkashKumarSingh11032001
My previous code is working for tensorflow 2.15.0 and it seems that this API does not support the weights parameter in 2.16.0.
Here is the adjusted code for 2.16.0:
import numpy as np
import tensorflow as tf
input = tf.constant(1.1234, dtype='float32')
out = tf.keras.layers.PReLU(alpha_initializer=tf.keras.initializers.Constant(np.inf))(input)
print(out)

Hi @maybeLee ,
I tried to run your code on colab using tf 2.15, 2.16 and tf-nightly versions faced same issue. Please find the gist here for reference. We need to work on it, In few days i will provide a solution for it.
Thank you!

Hi @maybeLee ,

Could you please take a look at the comment from the developer in the raised PR? Also the pr got closed and try to contribute for the same.

Thank you!


misspell: Add locale setting to misspell

This adds a --misspell-locale option to gometalinter. It passes the
desired locale setting to misspell so that you can check for common
typos like "color" which are obviously supposed to read "colour". ;)
Unfortunately there is no Canadian linter at this time, so UK will have
to do for now.
Example:
gometalinter --disable-all --enable=misspell --misspell-locale=UK

Please note: I did not test this patch yet, so please give it a once over before you merge.
Thanks!

/cc @client9

The code does not compile. See the CI.

@alecthomas My bad, I had a one char typo, since I didn't want to setup a custom GOPATH to test this :P (just use git master instead of vendoring please!!)
In any case, patch pushed (StringVar not StringsVar)
No idea why travis is failing now (logs not visible)-- this builds on my machine.

The regression tests are failing. I suspect because the default empty locale results in misspell -j 1 --locale  (note the lack of argument to --locale).
--- FAIL: TestMisSpell (1.21s)
	Error Trace:	support.go:51
			misspell_test.go:15
	Error:      	Not equal:
	            	expected: regressiontests.Issues{regressiontests.Issue{Linter:"misspell", Severity:"warning", Path:"test.go", Line:2, Col:7, Message:"\"langauge\" is a misspelling of \"language\""}, regressiontests.Issue{Linter:"misspell", Severity:"warning", Path:"test.go", Line:3, Col:9, Message:"\"langauge\" is a misspelling of \"language\""}}
	            	actual: regressiontests.Issues{}

	            	Diff:
	            	--- Expected
	            	+++ Actual
	            	@@ -1,4 +1,2 @@
	            	-(regressiontests.Issues) (len=2) {
	            	- (regressiontests.Issue) test.go:2:7:warning: "langauge" is a misspelling of "language" (misspell),
	            	- (regressiontests.Issue) test.go:3:9:warning: "langauge" is a misspelling of "language" (misspell)
	            	+(regressiontests.Issues) {
	            	 }

@alecthomas I've digged a little bit, but your test infra is sufficiently complex that it's hard to follow.
I don't understand how this is the case since passing an empty locale to misspell should simply not add US or UK dictionaries, and instead use only the main dictionary. I've checked that the language misspelling is included in that dictionary.
I've added quotes around the option in case it needs this to interpret an empty string. If this doesn't work, I'm afraid I'll have to abandon the patch if you can't suggest a proper solution.
Thanks!

I've added quotes around the option in case it needs this to interpret an empty string.

That is exactly the case, and what I was referring to.

\o/

Hey so I wrote patch 9af9a4364096fb170d968120f019fd9ad7234cd2 yet you merged 94ffa0f81f02e7bd6a9b33c2549d4a4de73d0ee6 ... what gives?

It's a squash merge.

Just fast-forward when you can :) It's more pro!

I'll take it under advisement.


fix: RefSeq GFF file now uses the key GenBank rather than Genbank (#60)

Closes: #60

looks good, thanks, will do a new release soon


Need more info to create ACL Tensor

Hi,
Could you elaborate more on second argument for the TensorInfo constructor?
When i go through documentation , it was saying as below

num_channels
It indicates the number of channels for each tensor element

But we we are already specifying the num_channels in the Tenshape(Width, Height, Channels). Then why again num_channels required(second argument). Its little bit confusing for me and if i use 1 inplace of second argument (num_channels) then it works fine with out any issue.
Thanks,
Hari
Originally posted by @nhkrishna in https://github.com/ARM-software/ComputeLibrary/issues/702#issuecomment-542608639

Hi @nhkrishna
You're right, it's redundant and unsafe to have both a tensor_shape and num_channels in the constructor. All we need to specify the number of channels is a tensor_shape and a data_layout.
What function does not work if you do not use 1u for the 2nd argument?

Hi @morgolock ,
By default i'm using 1 in place of second argument. I haven't tried using other than one.

Hi @nhkrishna
Using 1u is fine, we'll be addressing this api problem.
Closing the issue, if you require help please reopen it.


Prevent potential injection attack

There is a potential for an injection attack in the kafka.py script. Using a list of commands instead of cmd.split() will fix it.
The issue was raised here: https://github.com/openshift/managed-scripts/issues/84
JIRA: CSSRE-2200
I tested the changes by creating strimzi and kafka instances locally on CRC and running the script - no issues encountered.

/assign @reetika-vyas

/lgtm
/approve


PostgreSQL transaction error with multiple consumers on SQL transport

Contact Details
No response
Version
8.x (develop, pre-release)
On which operating system(s) are you experiencing the issue?
MacOS
Using which broker(s) did you encounter the issue?
No response
What are the steps required to reproduce the issue?
1. Configure a consumer with PostgreSQL transport and `SqlReceiveMode.PartitionedOrdered`.
2. Start several instances of the same consumer.
3. Publish messages to the queue.

I've made a simple project to reproduce the issue: https://github.com/olegtarasov/MassTransitSqlError. Substitute correct Postgres credentials, build the project and launch 2-3 instances of `Consumer`. Then launch `Producer` and observe the warnings in consumer logs.

Setting `IsolationLevel.ReadCommitted` seems to solve this issue:

x.SetBusFactory(new SqlRegistrationBusFactory((ctx, config) =>
                        {
                        config.UsePostgres(ctx,
                        hostConfig => { hostConfig.IsolationLevel = IsolationLevel.ReadCommitted; });

What is the expected behavior?
No warnings or errors are produced.
What actually happened?
Consumers generate the following warnings:
Retrying null: 40001: could not serialize access due to concurrent update
Npgsql.PostgresException (0x80004005): 40001: could not serialize access due to concurrent update

Related log output, including any exceptions
[22:08:33 WRN] Retrying null: 40001: could not serialize access due to concurrent update
Npgsql.PostgresException (0x80004005): 40001: could not serialize access due to concurrent update
   at Npgsql.Internal.NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at Npgsql.NpgsqlDataReader.<ReadMessage>g__ReadMessageSequential|49_0(NpgsqlConnector connector, Boolean async)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at Dapper.SqlMapper.QueryAsync[T](IDbConnection cnn, Type effectiveType, CommandDefinition command) in /_/Dapper/SqlMapper.Async.cs:line 434
   at MassTransit.SqlTransport.PostgreSql.PostgresDbConnectionContext.<>c__DisplayClass17_0`1.<<Query>b__0>d.MoveNext() in /_/src/Transports/MassTransit.SqlTransport.PostgreSql/SqlTransport/PostgreSql/PostgresDbConnectionContext.cs:line 83
--- End of stack trace from previous location ---
   at MassTransit.SqlTransport.PostgreSql.PostgresDbConnectionContext.<>c__DisplayClass17_0`1.<<Query>b__0>d.MoveNext() in /_/src/Transports/MassTransit.SqlTransport.PostgreSql/SqlTransport/PostgreSql/PostgresDbConnectionContext.cs:line 87
--- End of stack trace from previous location ---
   at MassTransit.RetryPolicies.PipeRetryExtensions.Retry[T](IRetryPolicy retryPolicy, Func`1 retryMethod, CancellationToken cancellationToken) in /_/src/MassTransit/RetryPolicies/PipeRetryExtensions.cs:line 55
  Exception data:
    Severity: ERROR
    SqlState: 40001
    MessageText: could not serialize access due to concurrent update
    Where: SQL statement "WITH msgs AS (
        SELECT md.*
        FROM "transport".message_delivery md
        WHERE md.message_delivery_id IN (
            WITH ready AS (
                SELECT mdx.message_delivery_id, mdx.enqueue_time, mdx.lock_id, mdx.priority,
                       row_number() over pw as row_number
                FROM "transport".message_delivery mdx
                WHERE mdx.queue_id = v_queue_id
                    AND mdx.delivery_count < mdx.max_delivery_count
                WINDOW pw as (partition by mdx.partition_key order by mdx.priority, mdx.enqueue_time, mdx.message_delivery_id)
            )
            SELECT ready.message_delivery_id
                FROM ready
                WHERE row_number = 1 AND ready.lock_id IS NULL
                AND ready.enqueue_time < v_now
                ORDER BY ready.priority, ready.enqueue_time, ready.message_delivery_id
            LIMIT fetch_count)
        FOR UPDATE OF md SKIP LOCKED)
        UPDATE "transport".message_delivery dm
        SET delivery_count = dm.delivery_count + 1,
            last_delivered = v_now,
            consumer_id = fetch_consumer_id,
            lock_id = fetch_lock_id,
            enqueue_time = v_enqueue_time
            FROM msgs
            INNER JOIN "transport".message m on msgs.transport_message_id = m.transport_message_id
        WHERE dm.message_delivery_id = msgs.message_delivery_id
        RETURNING
            dm.transport_message_id,
            dm.queue_id,
            dm.priority,
            dm.message_delivery_id,
            dm.consumer_id,
            dm.lock_id,
            dm.enqueue_time,
            dm.expiration_time,
            dm.delivery_count,
            dm.partition_key,
            dm.routing_key,
            dm.transport_headers,
            m.content_type,
            m.message_type,
            m.body,
            m.binary_body,
            m.message_id,
            m.correlation_id,
            m.conversation_id,
            m.request_id,
            m.initiator_id,
            m.source_address,
            m.destination_address,
            m.response_address,
            m.fault_address,
            m.sent_time,
            m.headers,
            m.host"
PL/pgSQL function transport.fetch_messages_partitioned(text,uuid,uuid,interval,integer) line 15 at RETURN QUERY
    File: nodeLockRows.c
    Line: 226
    Routine: ExecLockRows

Link to repository that demonstrates/reproduces the issue
https://github.com/olegtarasov/MassTransitSqlError

Update: I've tested the workaround more thoroughly and found that setting IsolationLevel.ReadCommitted breaks exactly-once delivery, and one message can be delivered to several consumers.

Update: I've tested the workaround more thoroughly and found that setting IsolationLevel.ReadCommitted breaks exactly-once delivery, and there are messages that are delivered to several consumers.

Of course it would, that's why repeatable read is required.

So is running multiple consumers for a single queue a supported scenario for SQL transport? Right now it seems that only one consumer can actively process messages and others are constantly getting transaction exceptions and eventually get restarted with errors like these:
[08:44:42 DBG] Consumer Stopping: db://localhost/message-queue (Consume Loop Exited)
[08:44:42 DBG] Endpoint Completed: db://localhost/message-queue
[08:44:42 DBG] Consumer Completed: db://localhost/message-queue: 0 received, 0 concurrent

Scaling out consumer instances is supported and meant to be used. If the transaction synchronization errors are causing the endpoint to fail, that's something that would need to be fixed. As more people use the SQL transport, updates will be prioritized with other updates for support customers (customers have priority).

Unfortunately, the error doesn't seem to be fixed. I've updated my test repo https://github.com/olegtarasov/MassTransitSqlError to use Nuget package version 8.2.0-develop.1702 and I get the same errors when using 3 concurent consumers and 1 producer:
[09:54:01 WRN] Retrying null: 40001: could not serialize access due to concurrent update
Npgsql.PostgresException (0x80004005): 40001: could not serialize access due to concurrent update
   at Npgsql.Internal.NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at Npgsql.NpgsqlDataReader.<ReadMessage>g__ReadMessageSequential|49_0(NpgsqlConnector connector, Boolean async)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at Dapper.SqlMapper.QueryAsync[T](IDbConnection cnn, Type effectiveType, CommandDefinition command) in /_/Dapper/SqlMapper.Async.cs:line 434
   at MassTransit.SqlTransport.PostgreSql.PostgresDbConnectionContext.<>c__DisplayClass17_0`1.<<Query>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at MassTransit.SqlTransport.PostgreSql.PostgresDbConnectionContext.<>c__DisplayClass17_0`1.<<Query>b__0>d.MoveNext() in /_/src/Transports/MassTransit.SqlTransport.PostgreSql/SqlTransport/PostgreSql/PostgresDbConnectionContext.cs:line 87
--- End of stack trace from previous location ---
   at MassTransit.RetryPolicies.PipeRetryExtensions.Retry[T](IRetryPolicy retryPolicy, Func`1 retryMethod, CancellationToken cancellationToken)
  Exception data:
    Severity: ERROR
    SqlState: 40001
    MessageText: could not serialize access due to concurrent update
    Where: SQL statement "WITH msgs AS (
        SELECT md.*
        FROM "transport".message_delivery md
        WHERE md.message_delivery_id IN (
            WITH ready AS (
                SELECT mdx.message_delivery_id, mdx.enqueue_time, mdx.lock_id, mdx.priority,
                       row_number() over pw as row_number
                FROM "transport".message_delivery mdx
                WHERE mdx.queue_id = v_queue_id
                    AND mdx.delivery_count < mdx.max_delivery_count
                WINDOW pw as (partition by mdx.partition_key order by mdx.priority, mdx.enqueue_time, mdx.message_delivery_id)
            )
            SELECT ready.message_delivery_id
                FROM ready
                WHERE row_number = 1 AND ready.lock_id IS NULL
                AND ready.enqueue_time < v_now
                ORDER BY ready.priority, ready.enqueue_time, ready.message_delivery_id
            LIMIT fetch_count FOR UPDATE SKIP LOCKED)
        FOR UPDATE OF md SKIP LOCKED)
        UPDATE "transport".message_delivery dm
        SET delivery_count = dm.delivery_count + 1,
            last_delivered = v_now,
            consumer_id = fetch_consumer_id,
            lock_id = fetch_lock_id,
            enqueue_time = v_enqueue_time
            FROM msgs
            INNER JOIN "transport".message m on msgs.transport_message_id = m.transport_message_id
        WHERE dm.message_delivery_id = msgs.message_delivery_id
        RETURNING
            dm.transport_message_id,
            dm.queue_id,
            dm.priority,
            dm.message_delivery_id,
            dm.consumer_id,
            dm.lock_id,
            dm.enqueue_time,
            dm.expiration_time,
            dm.delivery_count,
            dm.partition_key,
            dm.routing_key,
            dm.transport_headers,
            m.content_type,
            m.message_type,
            m.body,
            m.binary_body,
            m.message_id,
            m.correlation_id,
            m.conversation_id,
            m.request_id,
            m.initiator_id,
            m.source_address,
            m.destination_address,
            m.response_address,
            m.fault_address,
            m.sent_time,
            m.headers,
            m.host"
PL/pgSQL function transport.fetch_messages_partitioned(text,uuid,uuid,interval,integer) line 15 at RETURN QUERY
    File: nodeLockRows.c
    Line: 226
    Routine: ExecLockRows
[09:54:01 WRN] Consume Loop faulted
Npgsql.PostgresException (0x80004005): 40001: could not serialize access due to concurrent update
   at Npgsql.Internal.NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at Npgsql.NpgsqlDataReader.<ReadMessage>g__ReadMessageSequential|49_0(NpgsqlConnector connector, Boolean async)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at Dapper.SqlMapper.QueryAsync[T](IDbConnection cnn, Type effectiveType, CommandDefinition command) in /_/Dapper/SqlMapper.Async.cs:line 434
   at MassTransit.SqlTransport.PostgreSql.PostgresDbConnectionContext.<>c__DisplayClass17_0`1.<<Query>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at MassTransit.SqlTransport.PostgreSql.PostgresDbConnectionContext.<>c__DisplayClass17_0`1.<<Query>b__0>d.MoveNext() in /_/src/Transports/MassTransit.SqlTransport.PostgreSql/SqlTransport/PostgreSql/PostgresDbConnectionContext.cs:line 87
--- End of stack trace from previous location ---
   at MassTransit.RetryPolicies.PipeRetryExtensions.Retry[T](IRetryPolicy retryPolicy, Func`1 retryMethod, CancellationToken cancellationToken)
   at MassTransit.RetryPolicies.PipeRetryExtensions.Retry[T](IRetryPolicy retryPolicy, Func`1 retryMethod, CancellationToken cancellationToken) in /_/src/MassTransit/RetryPolicies/PipeRetryExtensions.cs:line 74
   at MassTransit.SqlTransport.PostgreSql.PostgresDbConnectionContext.Query[T](Func`3 callback, CancellationToken cancellationToken)
   at MassTransit.SqlTransport.PostgreSql.PostgresDbConnectionContext.Query[T](Func`3 callback, CancellationToken cancellationToken) in /_/src/Transports/MassTransit.SqlTransport.PostgreSql/SqlTransport/PostgreSql/PostgresDbConnectionContext.cs:line 74
   at MassTransit.SqlTransport.Middleware.SqlMessageReceiver.ReceiveMessages(Int32 messageLimit, CancellationToken cancellationToken) in /_/src/MassTransit/SqlTransport/SqlTransport/Middleware/SqlMessageReceiver.cs:line 108
   at MassTransit.Util.RequestRateAlgorithm.RunRequest[T](RequestCallback`1 requestCallback, ResultCallback`1 resultCallback, CancellationToken cancellationToken)
   at MassTransit.Util.RequestRateAlgorithm.Run[T](RequestCallback`1 requestCallback, ResultCallback`1 resultCallback, CancellationToken cancellationToken) in /_/src/MassTransit.Abstractions/Util/RequestRateAlgorithm.cs:line 186
   at MassTransit.SqlTransport.Middleware.SqlMessageReceiver.Consume() in /_/src/MassTransit/SqlTransport/SqlTransport/Middleware/SqlMessageReceiver.cs:line 73
  Exception data:
    Severity: ERROR
    SqlState: 40001
    MessageText: could not serialize access due to concurrent update
    Where: SQL statement "WITH msgs AS (
        SELECT md.*
         The packages should already be updated and available.

Okay, I've tested it with a custom test rig based on your sample, and I no longer get the warnings when concurrency exceptions are retried.


EOS-21138: Support '--pre-factory' option in Hare factory reset

Solution:
By default, cleanup will take you to post-factory and Pre-factory will take you to before-factory (i.e. undo everything that was done in post-install).
So we need to add this option in cleanup state.
Signed-off-by: Supriya Yadav<EMAIL_ADDRESS>

Can one of the admins verify this patch?

@supriyachavan4398 --pre-factory option is now supported by the CLI but it is effectively no-op. Is it expected?
This makes me think that the problem is not stated properly in the PR description. What problem do we solve?

@supriyachavan4398 , kindly restrict the commit message content to 80 columns.

@supriyachavan4398 --pre-factory option is now supported by the CLI but it is effectively no-op. Is it expected?
This makes me think that the problem is not stated properly in the PR description. What problem do we solve?

In Hare code, post-install supports to no-op. So for this task, we can't revert changes to before post-install. That's why --pre-factory option is now only supported by the CLI.

Deployment job: http://eos-jenkins.colo.seagate.com/job/Cortx-Deployment/job/VM-Deployment-R2-1Node/1796/


rails 6 + (including 6.0.3 and 6.1.1) having issue with switchery-rails ( railties < 6.0, >= 3.2)

When am trying to upgrade rails <IP_ADDRESS> to 6.0.3 am having issue with gentelella-rails dependent gem (switchery-rails).
Bundler could not find compatible versions for gem "railties":
In Gemfile:
rails (~> 6.0.3) was resolved to 6.0.3, which depends on
railties (= 6.0.3)
**gentelella-rails was resolved to 0.1.11, which depends on
  switchery-rails was resolved to 0.8.1, which depends on
    railties (~> 4)**

when i comment gentelella-rails from Gemfile, am able to upgrade rails 6.0.3. but with switchery-rails am not able to upgrade.
Problem is the dependent gem switchery-rails can only support till <IP_ADDRESS> rails version, then how there people using gentelella-rails in to Rails 6.
Can anybody help to fix this issue.

Any updates on this issue?
I am facing same dependency issue while upgrading to rails 6.0.0
 gentelella-rails was resolved to 0.1.11, which depends on
      switchery-rails was resolved to <IP_ADDRESS>, which depends on
        railties (>= 3.2, < 6.0)

This issue is fixed in switchery-rails gem in version <IP_ADDRESS>


Introduce segstats.py script for fast partial volume computation

The FreeSurfer mri_segstats program computes volume statistics from a combination of a segmentation and bias-field corrected image. Fundamentally, it assumes local regional intensity consistency.
In the PR, we introduce three alternatives to this script:

python port of the mri_segstats algorithm (called legacy)
a. python implementation using python and numpy code throughout including multiple processes for parallel execution (this is extremely inefficient and should be avoided, but avoids the python global interpreter lock)
b. python+numba (python code can be properly compiled by llvm) implementation including multiple threads for parallel execution and numba-based parallelization (this code is about as efficient as the FreeSurfer mri_segstats code, and avoids the global interpreter lock)
updated partial volume algorithm implemented in python and numpy including multiple threads for parallel execution (this code is significantly more efficient, but does not avoid the global interpreter lock as it does not require numba and compilation)
updated partial volume algorithm implemented in python and torch with almost no multi-threading but calculation on the GPU (this code is also significantly faster than other version, but optimized using sparse tensors and for SIMD processors)

I am currently running some benchmarks to document the runtime of the respective versions.
@m-reuter there is also a question of whether we want to even include and distribute the "legacy algorithms". I suspect, including them in one commit but removing them again might be an option, but I do not expect we want to add numba as a dependency (which is really required for acceptable run times).

benchmarking on landau
6-core, 12 thread
Intel(R) Xeon(R) CPU E5-2643 v3 @ 3.40GHz
FreeSurfer 7.3.2 mri_seg_stats
python -m timeit "import subprocess; subprocess.run('mri_segstats --seed 1234 --seg $TSUB/mri/wmparc.DKTatlas.mapped.mgz --sum $TSUB/stats/wmparc.DKTatlas.mapped.fsst
ats --pv $TSUB/mri/norm.mgz --excludeid 0 --brainmask $TSUB/mri/brainmask.mgz --in $TSUB/mri/norm.mgz --in-intensity-name norm --in-intensity-units MR --subject OAS1_0061_MR1_modified --surf-wm-vol --ctab $FREESURFER/WMParcStatsLUT.txt'.split(' '))"

FastSurfer 2.0.1* segstats.py
on CPU
python -m timeit "from FastSurferCNN.segstats import main, make_arguments; main(make_arguments().parse_args('-norm $TSUB/mri/norm.mgz -i $TSUB/mri/wmparc.DKTatlas.mapped.mgz -o $TSUB/stats/wmparc.DKTatlas.mapped.pyvstats --lut $FREESURFER/WMParcStatsLUT.txt --device cpu'.split(' ')))"

1 loop, best of 5: 4.06 sec per loop
Theoretically, there might be a very small advantage here for the python code, because I am not firing up a new process.
on GPU, I am still facing memory constraints, so it remains unclear, if that is possible. Also, it seems to be much smaller because of the sparsity of PV-affected voxels. The solution probably has to be to "patchify" the code similar to the cpu code, but then even more of the potential speed advantage of gpu goes away because of synchronizations and overhead.

I removed all non-core code so we do not have to maintain multiple versions of the code. Specifically, even the numba accelerated legacy algorithm was too slow and torch requires very high memory allocation. Furthermore, even the overhead for torch was already slower than the cpu implementation. Therefore, for now there will not be a dedicated torch implementation.
I also added a "--threads" flag to specify the number of threads. This seems to somewhat reliably load one cpu thread. However, numpy has inherent parallelism through MKL and openmp.
https://stackoverflow.com/questions/30791550/limit-number-of-threads-in-numpy
Performance
Even on one thread, the script is still pretty fast:
~/projects/FastSurfer$ python -m timeit "from FastSurferCNN.segstats import main, make_arguments; main(make_arguments().parse_args('-norm $TSUB/mri/norm.mgz -i $TSUB/mri/wmparc.DKTatlas.mapped.mgz -o $TSUB/stats/wmparc.DKTatlas.mapped.pyvstats --lut $FREESURFER/WMParcStatsLUT.txt'.split(' ')))"
1 loop, best of 5: 4.03 sec per loop

This is effectively on 12 threads.
~/projects/FastSurfer$ python -m timeit "from FastSurferCNN.segstats import main, make_arguments; main(make_arguments().parse_args('-norm $TSUB/mri/norm.mgz -i $TSUB/mri/wmparc.DKTatlas.mapped.mgz -o $TSUB/stats/wmparc.DKTatlas.mapped.pyvstats --lut $FREESURFER/WMParcStatsLUT.txt --threads 1'.split(' ')))"
1 loop, best of 5: 11.6 sec per loop


APM-316012 aurora RDS

without severity profiling:

2.603 seconds
2.472 seconds
2.481 seconds

with that profiling:

3.397 seconds
3.198 seconds
3.234 seconds

@mmajcher can you review?

@mmajcher or @dt-be can you look at this PR?


carbonplan: update k8s from 1.19 to 1.24 is made, now update eksctl cluster config template

This PR reflects the upgrade made, where the carbonplan.jsonnet file isn't actively used by our automation, but just a template we use to generate a eksctl config file that we manually reference when running eksctl CLI commands to make changes such as upgrading the k8s version.
For reference, these are my notes for this upgrade, based on notes from having done this for the JMTE hub before.
  # For reference, this is the steps I took when upgrading from k8s 1.19 to k8s
  # 1.24, Jan 24th 2023.
  #
  # 1. Updated the version field in this config from 1.19 to 1.20
  #
  #    - It is not allowed to upgrade the control plane more than one minor at the time
  #
  # 2. Upgraded the control plane (takes ~10 minutes)
  #
  #    - I ran into permission errors, so I visited the AWS cloud console to
  #      create an access key for my user and set it up temporary environment
  #      variables.
  #
  #      export AWS_ACCESS_KEY_ID="..."
  #      export AWS_SECRET_ACCESS_KEY="..."
  #
  #    eksctl upgrade cluster --config-file eksctl-cluster-config.yaml --approve
  #
  # 3. Deleted all non-core nodegroups
  #
  #    - I had to add a --drain=false flag due to an error likely related to a
  #      very old EKS cluster.
  #
  #    - I used --include="nb-*,dask-*" because I saw that the core node pool
  #      was named "core-a", and the other nodes started with "nb-" or "dask-".
  #
  #    eksctl delete nodegroup --config-file=eksctl-cluster-config.yaml --include "nb-*,dask-*" --approve --drain=false
  #
  # 4. Updated the version field in this config from 1.20 to 1.22
  #
  #    - It is allowed to have a nodegroup +-2 minors away from the control plan version
  #
  # 5. Created a new core nodepool (core-b)
  #
  #    - I ran into "Unauthorized" errors and resolved them by first using the
  #      deployer to acquire credentials to modify a ConfigMap named "aws-auth"
  #      in the k8s namespace kube-system.
  #
  #      deployer use-cluster-credentials carbonplan
  #
  #      kubectl edit cm -n kube-system aws-auth
  #
  #    eksctl create nodegroup --config-file=eksctl-cluster-config.yaml --include "core-b" --install-nvidia-plugin=false
  #
  # 6. Deleted the old core nodepool (core-a)
  #
  #    - I first updated the eksctl config file to include a "core-a" entry,
  #      because I didn't really add a "core-b" previously, I just renamed the
  #      "core-a" to "core-b".
  #
  #    eksctl delete nodegroup --config-file=eksctl-cluster-config.yaml --include "core-a" --approve
  #
  # 7. Upgraded add-ons (takes ~3*5s)
  #
  #    eksctl utils update-kube-proxy --cluster=carbonplanhub --approve
  #    eksctl utils update-aws-node --cluster=carbonplanhub --approve
  #    kubectl patch daemonset -n kube-system aws-node --patch='{"spec":{"template":{"spec":{"$setElementOrder/containers":[{"name":"aws-node"}],"containers":[{"name":"aws-node","securityContext":{"allowPrivilegeEscalation":null,"runAsNonRoot":null}}]}}}}'
  #    eksctl utils update-coredns --cluster=carbonplanhub --approve
  #
  #    - I diagnosed two separate errors following this:
  #
  #      kubectl get pod -n kube-system
  #      kubectl describe pod -n kube-system aws-node-7rcsw
  #
  #      Warning  Failed     9s (x7 over 69s)  kubelet            Error: container has runAsNonRoot and image will run as root
  #
  #      - the aws-node daemonset's pods failed to start because of a too
  #        restrictive container securityContext not running as root.
  #
  #        aws-node issue: https://github.com/weaveworks/eksctl/issues/6048.
  #
  #        Resolved by removing `runAsNonRoot: true` and
  #        `allowPrivilegeEscalation: false`. Using --output-patch=true led me
  #        to a `kubectl patch` command to use.
  #
  #        kubectl edit ds -n kube-system aws-node --output-patch=true
  #
  #      - the kube-proxy deamonset's pods failed to pull the image, it was not
  #        found.
  #
  #        This didn't need to be resolved mid way through upgrades, and was an
  #        issue that went away in k8s 1.23.
  #
  # 8. Update the version field in this config from 1.22 to 1.21
  #
  # 9. Upgraded the control plane, as in step 2.
  #
  # A. Upgraded add-ons, as in step 7.
  #
  # B. Update the version field in this config from 1.21 to 1.22
  #
  # C. Upgraded the control plane, as in step 2.
  #
  # D. Upgraded add-ons, as in step 7.
  #
  # E. I refreshed the ekscluster config's .jsonnet file based on
  #    template.jsonnet which has been updated to declare a addon related to ebs
  #    storage. In practice, this was probably not used later by subsequent
  #    commands I realize. It feels good to have it in the ekscluster config
  #    though to reflect adding it manually.
  #
  #    addons: [
  #        {
  #            // aws-ebs-csi-driver ensures that our PVCs are bound to PVs that
  #            // couple to AWS EBS based storage, without it expect to see pods
  #            // mounting a PVC failing to schedule and PVC resources that are
  #            // unbound.
  #            //
  #            // Related docs: https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html
  #            //
  #            name: 'aws-ebs-csi-driver',
  #            wellKnownPolicies: {
  #                ebsCSIController: true,
  #            },
  #        },
  #    ],
  #
  #    eksctl create iamserviceaccount \
  #             --name=ebs-csi-controller-sa \
  #             --namespace=kube-system \
  #             --cluster=carbonplanhub \
  #             --attach-policy-arn=arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
  #             --approve \
  #             --role-only \
  #             --role-name=AmazonEKS_EBS_CSI_DriverRole
  #
  #    eksctl create addon --name=aws-ebs-csi-driver --cluster=carbonplanhub --service-account-role-arn=arn:aws:iam::631969445205:role/AmazonEKS_EBS_CSI_DriverRole --force
  #
  # F. Update the version field in this config from 1.22 to 1.23
  #
  # G. Upgraded the control plane, as in step 2.
  #
  # H. Upgraded add-ons, as in step 7.
  #
  # I. Update the version field in this config from 1.23 to 1.24
  #
  # J. Upgraded the control plane, as in step 2.
  #
  # K. Upgraded add-ons, as in step 7.
  #
  # L. I created a new core node pool and deleted the old, as in step 5-6.
  #
  #    eksctl create nodegroup --config-file=eksctl-cluster-config.yaml --include "core-a" --install-nvidia-plugin=false
  #    eksctl delete nodegroup --config-file=eksctl-cluster-config.yaml --include "core-b" --approve
  #
  # M. I recreated all other nodegroups.
  #
  #    eksctl create nodegroup --config-file=eksctl-cluster-config.yaml --include "nb-*,dask-*" --install-nvidia-plugin=false
  #

Do you think we can turn your notes into docs of some sort? maybe create a sre guide on how to update the addons for an aws cluster for example? would that make sense and would it be useful for the future?

For now I'll link this thorougly so we don't loose these notes, but I'd like to not document in sphinx yet as there are so many open questsions I'm thinking about still and technical parts that may be one-off or may be important to have lying around etc.
There is a lot for us to do with regards to practices of managing k8s upgrades, as well as documenting how we technically perform them. I'm tracking a lot of AWS EKS upgrades to make in https://github.com/2i2c-org/infrastructure/issues/2057 still, and will likeley gain more experience soon as I want to upgrade openscapes from 1.21 to 1.22 at least as well.
In short, I'm on it and will make things happen in the future.

Thank you soo much for reviewing this @GeorgianaElena!!!!!!


'Upload Image' button on Image page does not work

Image (for example .tif) cannot be uploaded, the image wasn't uploaded when clicking 'upload and continue'.

Page was tested and does work properly.


Setup Instrumentation in VulnLoc for Angelix

this issue is now monitored in benchmark repo (https://github.com/nus-apr/vulnloc-benchmark/issues/1)


Sema: Fix order dependency in @objc inference from witnessed protocol requirement [5.1]

If we haven't validated the declaration yet, the 'witnesses @objc
requirement' check would immediately fail. Move the validateDecl()
call to matchWitness() to fix this.
Fixes rdar://problem/49482328, https://bugs.swift.org/browse/SR-10257.

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 4710da5fb35d19115cf542f3f43e8219dd8246f4

Build failed
Swift Test Linux Platform
Git Sha - 4710da5fb35d19115cf542f3f43e8219dd8246f4

@shahmishal Looks like there's an issue with update-checkout on the swift-5.1-branch?

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 4710da5fb35d19115cf542f3f43e8219dd8246f4

@swift-ci Please test macOS

@shahmishal Looks like there's an issue with update-checkout on the swift-5.1-branch?

It should be resolved now.

When will swift 5.1 been release？It is really a big problem.

I agree, It was breaking my CollectionViews and who knows what else, which I believe is the main problem, it could be breaking anything silently
Shouldn't this be release as a hotfix to the compiler/Xcode?

@LiuDeng @Robuske seems like Xcode 10.3 fixes the problem! At least that's what we got from our initial tests.. 😄


[GLUTEN-4763][VL] Add RewriteTypedImperativeAggregate rule for collect_list

What changes were proposed in this pull request?
For some typed imperative aggregate functions, like collect_list/collect_set, Spark will make agg buffer stored as BinaryType in partial/ partial merge phase, even though these two functions' output is ArrayType. Gluten has no such special handling. For Velox backend, agg function's raw data type is used for agg buffer in partial/partial merge. Thus, there will be a type mismatch issue in the latter shuffle operator, as Gluten plan inherits attributes from its corresponding Spark plan. For collect_list/collect_set, shuffle expects BinaryType, but gets ArrayType.
So, we need a rule RewriteTypedImperativeAggregate to rewrite collect_list and collect_set for Velox backend. Referenced the implementation in https://github.com/oap-project/gluten/pull/2669.
Velox does not register the companion functions for collect_set, to enable collect_set, modifications need to be made in Velox.
Currently, there are still some issues, such as the semantics of null, where Spark and Velox have many differences. At present, collect_list supports ignoring null values, but when all inputs are null, Spark returns an empty array, while Velox returns null. collect_set does not yet support ignoring null values, and there is the same problem when all inputs are null. Adaptations on the Velox side are still needed.
How was this patch tested?

Exists CI.
Add collect_list in VeloxAggregateFunctionsSuite.distinct functions
Add a new test case VeloxAggregateFunctionsSuite.collect_list null inputs

cc @PHILO-HE, thanks.


net/icmp: icmp bug fix and enhancement

Summary
net/icmp: fix race condition in icmp recvmsg
net/icmp: add nonblocking support
net/icmp: consume the data length to avoid duplicate packet
net/icmp: fix invalid condition comparison
Impact
ping/ping6
Testing
ping/ping6 works as expected

@yamt @antmerlino could you take a look again?

@xiaoxiang781216 give me a day or so to look at this again

@xiaoxiang781216 give me a day or so to look at this again

Sure.


Add functions: Get and Set-SlackReminder

Add functions: Get and Set-SlackReminder

https://api.slack.com/methods/reminders.add (note: example for user was slack user ID form, not friendly @ name.  Need to test if we can use the friendly name, otherwise will need to translate)
https://api.slack.com/methods/reminders.list
https://api.slack.com/methods/reminders.info
Not sure if anyone would need it, but consider Remove - https://api.slack.com/methods/reminders.delete

Thanks to Ramprabhu for the idea!

I have the code for this written and tested but want to wait until https://github.com/RamblingCookieMonster/PSSlack/pull/71 has been reviewed and merged.  Once that's done I'll open a new pull request


Example data fits Sphinx Gallery's image dimensions

Title:
Adapt example data in demo to fit Sphinx Gallery's image dimensions
Summary:
Sphinx Gallery renders single images at 640px by 480px. Adjusting the layout of the example data to better fit these dimensions and reduce white space.
Relevant references:
https://app.shortcut.com/xanaduai/story/76532
Before:

After:

LGTM! Just a note, this new figure layout introduces a 6th figure. Not sure if that's an issue or not given the context of this demo. cc @ikurecic

These figures are a representative sample of the data. Afaik the initial choice of 5 was arbitrary, so changing it to 6 shouldn't material affect the demo.


Credentials for multiple hosts.

Now cargo login stores a token per host.
If the host parameter is omitted cargo stores a token as default, i.e. as a token for crates.io.
Resolves #3365.
Implemented based on the conversation in #3978.
This RP could be postponed until the cargo team decides which format for registry we should use. Currently, it's a URL.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

This seems related to https://github.com/rust-lang/cargo/issues/4208 where the choice of url here is not the best today (e.g. the index everywhere) but rather we should find a better identifier for this. In that sense this may not be able to land for awhile unfortunately :(

Let me know when I can update this PR accordingly to a new identifier.

:umbrella: The latest upstream changes (presumably #4433) made this pull request unmergeable. Please resolve the merge conflicts.

It would also occasionally be useful to read .cargo/credentials from current directory or it's parents the same way as .cargo/config is searched. For example if I had projects for two different customers (or work and private), I might use two different GitHub accounts to keep them separate. And then I might have /home/me/customer1/.cargo/config and /home/me/customer2/.cargo/config. But it appears not to be currently possible to have corresponding .cargo/credentials.
Also, note that (at least unless I am misunderstanding the code), the set of places where config and credentials are searched is actually disjoint. Say I have the project (on Windows this is rather common) in d:\projects\name. The cargo will look for config in d:\projects\name\.cargo\config, d:\projects\.cargo\config and d:\.cargo\config, but as far as I can tell, not c:\Users\me\.cargo\config. But it will only look for credentials in c:\Users\me\.cargo\credentials. Is this split really desired?

This split is really desired. $CARGO_HOME/credentials file is global and should handle all cargo related credentials.

Normally it should. That still does not explain why it must.

This PR is kind of stale at this point and needs further changes as discussed, so I'm going to close it. Please open a new PR for updated changes! Thank you!


JOSS review - API documentation

Issue raised as part of the JOSS review process to address the "Functionality Documentation" check https://github.com/openjournals/joss-reviews/issues/6869
The link to the API documentation in the README.md leads to a page which I think has missing information - https://clinical-genomics-umea.github.io/fraggler/fraggler.html - here I just see the below:

The "fraggler" submodule link leads to the detailed code documentation describing the functions - but there is no overview of how to use the API version of the tool.

Hi @KatyBrown!
The API of Fraggler is best demonstrated in the tutorial.ipynb. This notebook documents everything needed to use Fraggler in a Python environment.
The link to the documentation provides a comprehensive overview of the source code. I have tried to document and describe the code thoroughly, making it easy to understand the logic and flow.
Hope this helps!
William

I see,  I was expecting that the link above was the main documentation of the API.


Hackage version still requires text <1.3

https://hackage.haskell.org/package/text-display-<IP_ADDRESS>/dependencies
It should get some <3 :smile:

Good catch!

@dpwiz Do you need a release to be made post-haste or can you depend on a Github tag?

The most recent Hackage version still wants text-1.2.

@dpwiz Hi, it's done. :) Could you please try <IP_ADDRESS> and tell me if this is okay for you?

I haven't actually tried it, but the Hackage says it requires text >= 2 which is unexpected. I suspect that wasn't intentional as the previous version worked fine with text < 2.


add build container dockerfile to rebuild VICAR with new code as new image

think this would be a new dockerfile, but let's write up the steps that recompile vicar so that it's easier to build a new image that includes updated vicar programs given new source code

doing instead as a multi stage build


Move get_deployments query to the DAO

Which issue(s) this PR fixes

Fixes None
Additional comments

Split the get_deployment query into several utility queries in the DAO.
This PR is linked to this comment: https://github.com/TOSIT-IO/tdp-lib/pull/592#issuecomment-2152184234
Agreements

[X] I hereby declare this contribution to be licensed under the Apache License Version 2.0, January 2004.
[X] I hereby agree to grant TOSIT a copyright license to use my contributions.

I also moved the get_planned_deployment. Maybe I should move this commit to another PR 👀

get_planned_deployment has been moved to #595

Force pushed after solving merge conflicts.


feat: Implement pipeline to build the helm chart

This makes a few changes to support building the helm chart in CI:

Tag each image with a version string, so that the helm chart can be explicit about what it's pulling in.

Due to concourse limits, add a S3 file to save the tag name in the file name (‽)

Also build the -setup docker images in CI.
Add a script to package the helm chart.

Include a second script to be able to patch in the image versions so that they by default are pinned.  This, of course, can be overridden by the user when deploying.

This is comment-preserving.

Modify the helm chart to ensure that each image can be overridden individually (because the tag for each image will be different).

Fixes https://github.com/cloudfoundry-incubator/kubecf/issues/653

A test deployment of this pipeline is available at https://concourse.suse.dev/teams/marky/pipelines/marky-eirinix (requires suse-platform-team membership to view, if I have set things up correctly).


Add support of Interpolated strings.

Add support of Interpolated strings.
see https://github.com/zzzprojects/System.Linq.Dynamic.Core/issues/527
I have added two test cases: QueryableTests.FormattableString.cs and EntitiesTests.FormattableString.cs. All the tests are passed:

Hello @yangzhongke, thank you for your contribution.
However, I thought that this functionality would be integrated in the current code, and not creating new extension methods.
Can you change your PR?

If there are two overloaded methods as below:

Where(string predicate,params object[] args)
2)Where(FormattableString predicate)

Due to the deliberate decision by the Microsoft , The code Where($"Id={id}") will invoke  Where(string predicate,params object[] args) instead of Where(FormattableString predicate),  which lead to my decision to create new method names, like WhereInterpolated instead of using Where.

Can't you just add a new method like?
Existing...
public static IQueryable Where([NotNull] this IQueryable source, [NotNull] ParsingConfig config, [NotNull] string predicate, params object[] args)

New...
public static IQueryable Where([NotNull] this IQueryable source, [NotNull] ParsingConfig config, [NotNull] FormattableString predicate)

I can. But where($" id={id}")  will invoke the "string predicate" version instead of "FormattableString" version. I have mentioned the reason  before.

Even EF Core created two methods to execute sql statements, ExecuteSqlRaw(string version) and ExecuteSqlInterpolated(FormatableString version). So I reckon that the two different methods solution, Where and WhereInterpolated , is a tradeoff.

Agree with the renaming.
Should I do something to do the changes?

@yangzhongke
Yes, if possible, change your PR code.

I have renamed those two files and committed the changes.
There are two files: DynamicQueryableWithFormattableStringExtensionsand EFDynamicQueryableWithFormattableStringExtensions.
DynamicQueryableWithFormattableStringExtensions.cs contains: AllInterpolated and other XXXInterpolated methods, and EFDynamicQueryableWithFormattableStringExtensions contains: AllInterpolatedAsync and other XXXInterpolatedAsync methods.
DynamicQueryableWithFormattableStringExtensionsand .cs is already on the src/System.Linq.Dynamic.Core folder; because EFDynamicQueryableWithFormattableStringExtensions uses Microsoft.EntityFrameworkCore, just like EntityFrameworkDynamicQueryableExtensions.cs, so EFDynamicQueryableWithFormattableStringExtensions  should be on the folder Microsoft.EntityFrameworkCore.DynamicLinq.EFCore3.


[FEATURE REQUEST] Select All within an Optgroup

I would love to see the ability to click on a header of an optgroup and have it select or deselect all it's members.
It could be added the same way search box's are:
<select class="selectpicker"
            data-live-search="true"
            data-select-header="true"
            multiple
            title="Select a Country">

Which would then add the functionality to click on the header to select all it's members like so:

Duplicate of #737.


arm: support arm build

Generated the vmlinux.h from an ARM server via
bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

Tested on Ampere setup:
[root@perf-arm-12 kepler]# uname -r
5.14.0-362.21.1.el9_3.aarch64
[root@perf-arm-12 kepler]# curl localhost:8888/metrics 2>&1 |grep kepler_exporter_build_info
# HELP kepler_exporter_build_info A metric with a constant '1' value labeled by version, revision, branch, goversion from which kepler_exporter was built, and the goos and goarch for the build.
# TYPE kepler_exporter_build_info gauge
kepler_exporter_build_info{branch="",goarch="arm64",goos="linux",goversion="go1.20.14",revision="unknown",tags="include_gcs,include_oss,containers_image_openpgp,gssapi,providerless,netgo,osusergo,libbpf,linux,gpu",version=""} 1
[root@perf-arm-12 kepler]# curl localhost:8888/metrics |grep kepler_process_ |grep joules |sort -k 2 -g |tail
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  132k    0  132k    0     0  32.4M      0 --:--:-- --:--:-- --:--:-- 32.4M
kepler_process_core_joules_total{command="node",container_id="system_processes",mode="dynamic",pid="86862",source="intel_rapl",vm_id=""} 0.195
kepler_process_package_joules_total{command="node",container_id="system_processes",mode="dynamic",pid="86862",source="intel_rapl",vm_id=""} 0.195
kepler_process_core_joules_total{command="sh",container_id="system_processes",mode="dynamic",pid="86735",source="intel_rapl",vm_id=""} 0.549
kepler_process_package_joules_total{command="sh",container_id="system_processes",mode="dynamic",pid="86735",source="intel_rapl",vm_id=""} 0.549
kepler_process_core_joules_total{command="irqbalance",container_id="system_processes",mode="dynamic",pid="2403",source="intel_rapl",vm_id=""} 0.681
kepler_process_package_joules_total{command="irqbalance",container_id="system_processes",mode="dynamic",pid="2403",source="intel_rapl",vm_id=""} 0.681
kepler_process_core_joules_total{command="node",container_id="system_processes",mode="dynamic",pid="87516",source="intel_rapl",vm_id=""} 1.581
kepler_process_package_joules_total{command="node",container_id="system_processes",mode="dynamic",pid="87516",source="intel_rapl",vm_id=""} 1.581
kepler_process_core_joules_total{command="kworker/73:0",container_id="kernel_processes",mode="dynamic",pid="383",source="intel_rapl",vm_id=""} 229.143
kepler_process_package_joules_total{command="kworker/73:0",container_id="kernel_processes",mode="dynamic",pid="383",source="intel_rapl",vm_id=""} 229.143

@vimalk78 @SamYuan1990 how did you build the arm64 kepler_builder image? I can find one for s390 but not arm64

checking

@vimalk78 @SamYuan1990 how did you build the arm64 kepler_builder image? I can find one for s390 but not arm64

I built it on internal RH systems, not in upstram CI.

let see what will happen in #1261

Tested it on ARM
# curl localhost:8888/metrics |grep joules |sort -k 2 -g |grep kepler_process_core |tail -5
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  131k    0  131k    0     0  21.3M      0 --:--:-- --:--:-- --:--:-- 21.3M
kepler_process_core_joules_total{command="node",container_id="system_processes",mode="dynamic",pid="268421",source="intel_rapl",vm_id=""} 0.033
kepler_process_core_joules_total{command="node",container_id="system_processes",mode="dynamic",pid="256051",source="intel_rapl",vm_id=""} 0.06
kepler_process_core_joules_total{command="node",container_id="system_processes",mode="dynamic",pid="256093",source="intel_rapl",vm_id=""} 0.075
kepler_process_core_joules_total{command="irqbalance",container_id="system_processes",mode="dynamic",pid="2403",source="intel_rapl",vm_id=""} 0.3
kepler_process_core_joules_total{command="migration/71",container_id="kernel_processes",mode="dynamic",pid="371",source="intel_rapl",vm_id=""} 104.277
# rpm -qa |grep kepler
kepler-v<IP_ADDRESS>.gf4b2922d.dirty-20240226.aarch64
# uname -r
5.14.0-362.21.1.el9_3.aarch64

kepler amd64 and arm64 images are now available

Awesome! Works as expected 👍

@beneiltis this sounds great! would you please write a PR and share your story? https://github.com/sustainable-computing-io/kepler-doc

Sure I am happy to help. But can you elaborate on what I should create a PR for? 😇
I only checked that the image works on arm64 clusters.

Sure I am happy to help. But can you elaborate on what I should create a PR for? 😇 I only checked that the image works on arm64 clusters.

@beneiltis The installation process on ARM is not there yet. It would be great if you can walk through your steps and share how to get started. https://github.com/sustainable-computing-io/kepler-doc/blob/main/docs/installation/kepler.md

I had a clarification regarding this implementation. The implementation has "source=intel_rapl". Is that a display artifact and how does it work with ARM64 when you are using the intel_rapl? Thanks!


Not sure if this exactly the same issue that @skorphil is referring to, but when I have tables in preview mode using the dark theme there is not a visible outer border.

Not sure if this exactly the same issue that @skorphil is referring to, but when I have tables in preview mode using the dark theme there is not a visible outer border.

Yeap, that's what I mean

I see. There is also no outer in border in light mode. This is a feature not a bug 😄
As of Minimal 4.1.7 there are many new helper classes available to help you style tables
https://github.com/kepano/obsidian-minimal/releases/tag/4.1.7