YARN-11330: use secure XML parsers

Description of PR
How was this patch tested?
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 48s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 24s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
28m 42s

trunk passed

+1 :green_heart:
compile
10m 34s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
9m 11s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
2m  4s

trunk passed

+1 :green_heart:
mvnsite
3m 27s

trunk passed

+1 :green_heart:
javadoc
3m  3s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
2m 47s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 30s

trunk passed

+1 :green_heart:
shadedclient
24m 22s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  4s

the patch passed

+1 :green_heart:
compile
9m 52s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
9m 52s

the patch passed

+1 :green_heart:
compile
8m 59s

the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
8m 59s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 53s

the patch passed

+1 :green_heart:
mvnsite
3m  8s

the patch passed

+1 :green_heart:
javadoc
2m 37s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
2m 30s

the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 36s

the patch passed

+1 :green_heart:
shadedclient
24m 27s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
103m 17s

hadoop-yarn-server-resourcemanager in the patch passed.

-1 :x:
unit
24m  5s
/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
hadoop-yarn-server-nodemanager in the patch passed.

+1 :green_heart:
unit
28m 25s

hadoop-yarn-client in the patch passed.

+1 :green_heart:
asflicense
1m  7s

The patch does not generate ASF License warnings.

327m 56s

Reason
Tests

Failed junit tests
hadoop.yarn.server.nodemanager.webapp.dao.gpu.TestGpuDeviceInformationParser

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4981/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4981

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 1983159e8c5f 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 976eb867c808e5b11c304c57f74d02d88e91e316

Default Java
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4981/1/testReport/

Max. process+thread count
883 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4981/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 43s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 16 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m 13s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 56s

trunk passed

+1 :green_heart:
compile
23m 21s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
20m 59s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 14s

trunk passed

+1 :green_heart:
mvnsite
5m 48s

trunk passed

+1 :green_heart:
javadoc
5m 14s

trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
4m 33s

trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
8m 43s

trunk passed

+1 :green_heart:
shadedclient
21m 58s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 15s

the patch passed

+1 :green_heart:
compile
22m 41s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
22m 41s

the patch passed

+1 :green_heart:
compile
21m  6s

the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
21m  6s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 10s

the patch passed

+1 :green_heart:
mvnsite
6m  4s

the patch passed

+1 :green_heart:
javadoc
4m 39s

the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
4m 44s

the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
9m 34s

the patch passed

+1 :green_heart:
shadedclient
22m  6s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 42s

hadoop-common in the patch passed.

+1 :green_heart:
unit
98m 40s

hadoop-yarn-server-resourcemanager in the patch passed.

+1 :green_heart:
unit
24m 14s

hadoop-yarn-server-nodemanager in the patch passed.

+1 :green_heart:
unit
28m 22s

hadoop-yarn-client in the patch passed.

+1 :green_heart:
asflicense
1m 19s

The patch does not generate ASF License warnings.

412m 50s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4981/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4981

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 362bd7fcfbb2 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 83781e208509fffbb2cd01379b63349103a03723

Default Java
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4981/2/testReport/

Max. process+thread count
2788 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4981/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


missing character in list example

Hi, the example to list VM extensions has a missing character for the resource group.
it shows -r but should be --r

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2ebbeb5f-6ab6-3fb3-c969-5d935e60f251
Version Independent ID: 9d4169c9-f5b5-512a-8061-2234d6dec1eb
Content: az vm extension
Content Source: src/command_modules/azure-cli-vm/azure/cli/command_modules/vm/_help.py
Service: virtual-machines
GitHub Login: @rloutlaw
Microsoft Alias: routlaw

The example to list VM extensions is:
az vm extension list -g MyResourceGroup --vm-name MyVm
This is correct. Can you copy the exact problematic example?

hi, my bad. seems indeed correct.
topic can be closed


[SUPPORT] Spark planner choose broadcast hash join for large HUDI data source

After apply HUDI-6941 in the internal HUDI version(based on 0.14.0 version), there is a frequent occurrence of the execution plan selecting "broadcast hash join" to broadcast a large HUDI data source.

I tried to investigate the cause of this issue.
In those cases,  usingHadoopFsRelation to read HUDI source, and Spark JoinSelection would call HadoopFsRelation#sizeInBytes to estimate the relation size to decide whether use broadcast join or not. And HadoopFsRelation#sizeInBytes would call HoodieFileIndex#sizeInBytes. But at the moment, no partitions are loaded because using default lazy Hudi's file-index implementation's file listing mode. So FileIndex#cachedAllInputFileSlices is an empty map, then HadoopFsRelation#sizeInBytes returns 0, it caused the suboptimal join plan.
After apply HUDI-6941, more cases could enabled lazy list mode by default, so the issue has become more frequent.

@xuzifu666 @codope Please help me confirm whether my analysis of this issue is correct.  Is the FileIndex#sizeInBytes better to return the Long.MAX instead of 0 if FileIndex has not done partition pruning yet?

Already find the root cause.


chore: release 0.10.1

:robot: I have created a release *beep* *boop*
0.10.1 (2021-06-03)
Reverts

empty test data (#339) (ffecd9f)

This PR was generated with Release Please. See documentation.

From #342 breaking changes:

Bee.pin(), Bee.unpin(), Bee.pssSend() methods now does not return anything

From #341 breaking changes:

Methods Bee.setJsonFeed(), SocWriter.upload(), FeedWriter.upload() no returns directly the reference hash (string) instead of it being wrapped in object


Question: Is IFormFile SaveAsAsync function not available in AspNetCore Http ?

Hi Team,
is IFormFile in Microsoft.AspNetCore.Http namespace doesn't have SaveAsAsync function?
it available when use Microsoft.AspNet.Http namespace

It was simplified to CopyToAysnc(Stream)
https://github.com/aspnet/HttpAbstractions/blob/dev/src/Microsoft.AspNetCore.Http.Features/IFormFile.cs#L61

@Tratcher  thanks for the information. I found old source and use the extension


Fix SQLPLanMetric constructor for DB Platforms

Signed-off-by: Ahmed Hussein (amahussein<EMAIL_ADDRESS>Fixes #762
handles the construction of SQLPlanMetric during runtime. Databricks has a different constructor which causes the tools to crash during the profiler.

Sorry, Reviwed the code but didn't check the pre-merge CI status.

Sorry, Reviwed the code but didn't check the pre-merge CI status.

no worries. it is line length issue


Bad value in variable $lb_current_script

based on $BASH_SOURCE instead of $0.
Affects:

$lb_current_script_name
$lb_current_script_directory
Versions:
1.6.0
1.6.1
1.6.2
1.6.3
1.7.0

Fixed in v1.7.1


issue trying to get start-dbamigration to run

I am not a big powershell user.  But, I am working with dbatools to test it for some upcoming SQL server migrations.  I am starting with the line below on a test server:
start-dbamigration -source sql2012 -destination sql2019 -backuprestore -sharedpath Z:\ -whatif

I do have open connections to both database servers, the local one, and the on to be migrated, in SSMS.
Where Z:\ is a mapped drive to a backup location on the server to be migrated.  With a sub-directory for each database's backups.  ie Z:\db1, Z:\db2, etc.  Could that be causing an issue?
I have also tried the using the FQDN of the server: \\servername\d$\mssqlbu\backup\

I am receiving the message:
What if: Performing the operation "Set variable" on target "Name: __dbatools_interrupt_function_78Q9VPrM6999g6zo24Qn83m09XF56InEn4hFrA8Fwhu5xJrs6r Value: True".
The network path was not found
At C:\Program Files\WindowsPowerShell\Modules\dbatools\1.0.22\allcommands.ps1:80163 char:9
+         throw $records[0]
+         ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], Exception
    + FullyQualifiedErrorId : dbatools_Connect-DbaInstance

I am not clear if the cmdlet is not finding the location set for Z:, or if it is having an issue with the code in the module.

A shared path needs to be a UNC share. A mapped drive cannot be seen across two different SQL Server instances.
As well, use of the admin share \\servername\D$ will not work if the service account for SQL Server (both instances) has access to it as well. You will need to utilize a network share that both service accounts for SQL Server can physically access and read/write to them.

Shawn is correct. If Z is mapped, it's only mapped for the account that created it. Once you've got permissions set properly for the network share, then you'd need to run a Test-DbaPath using the credentials of the SQL Server service accounts to ensure they've got access.


Spelling correction

small spelling correction on section - Confirm data is flowing,  from "If you can't run the application or you aren't getting data as expected, wee the dedicated [troubleshooting article]" to "If you can't run the application or you aren't getting data as expected, see the dedicated [troubleshooting article]".

@arjun-ga - Thanks for the fix!


Cpd4 storage

Pull update from before didn't seem to work right on the first PR.

@annumberhocker portworx_is_ready carried over from when Eric had cp4d 3.5, it's always set to 1 in the cp4d module:
portworx_is_ready   = 1 // Assuming portworx is installed if using VPC infrastructure
I don't think odf_is_ready is needed.


Remove occurrences of Unknonwn ICECredentialType

In 3.1.37, we removed the possibility for ICECredentialType
to be Unknown.  Remove remaining tests for this case from
the WASM code.

Thanks for cleaning this up @jech !


Listen for development server on different port for tests

Required some refactoring of how we listen since the tests were not actually,
adding a new listener.
Also fixes issues with tests not closing properly by allowing tests to turn off
nunjucks watch mode.
Used https://github.com/mafintosh/why-is-node-running to identify and fix this issue.
Use https://github.com/alphagov/govuk-frontend/pull/622/files?w=1 to see changes without whitespace updates, since I've indented a lot of code.
Fixes https://github.com/alphagov/govuk-frontend/issues/612

do you also need to change the heroku script in package.json?

could we have  http://localhost:8888/ some where central, as we're repeating it in multiple places?

I don't like using globals in Node.js projects, so tried that but it felt over-the-top.
Happy to put it back in though, I don't expect us to want to change this port programatically from now on though.

don't forget the changelog as well


add fractionutil and update forge-std to add vm.breakpoint

will have to reinit submodules
# Re-initialize Git submodules when changing between branches with different dependency versions
# h/t @PaulRBerg
reinit() {
    git submodule deinit -f .
    git submodule update --init
}

Codecov Report
Patch and project coverage have no change.

Comparison is base (f99aaa0) 100.00% compared to head (d93d33d) 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##               1.5     #1180    +/-   ##
==========================================
  Coverage   100.00%   100.00%
==========================================
  Files           46        46
  Lines         2652      2414   -238
  Branches       396       354    -42
==========================================
- Hits          2652      2414   -238

Flag
Coverage Δ

foundry
?

production
100.00% <ø> (ø)

reference
100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 29 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Closing this for now as it's causing a wave of miscellaneous tests to break; can reopen once we get that sorted


Added KnockoutValidationGroupingOptions: live

knockout.validation/knockout.validation.d.ts
to author(@danludwig). could you review this PR?
:+1: or :-1:?
check list

[ ] pass the Travic-CI test?

Hello and thank you for your contribution!
Due to an excessively long queue of pull requests that have become stale, we are "declaring bankruptcy" and closing all PRs opened before May 1, 2016. If you'd still like to merge this code in, please open a new PR that has been merged and rebased with the master branch.
Going forward, we are committing to review or merge all PRs on a regular basis so this bankruptcy will not occur again. We apologize for the incovenience and hope you will continue to contribute to DefinitelyTyped in the future.
Thanks
The TypeScript and DefinitelyTyped teams


Don't set widget values in the widgets (as done from file)

Fixes #377

Codecov Report
Patch coverage: 100.00% and project coverage change: -0.02% :warning:

Comparison is base (009622b) 85.91% compared to head (616287d) 85.89%.
Report is 2 commits behind head on main.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #379      +/-   ##
==========================================
- Coverage   85.91%   85.89%   -0.02%
==========================================
  Files          30       30
  Lines        2172     2148      -24
  Branches      396      393       -3
==========================================
- Hits         1866     1845      -21
+ Misses        225      223       -2
+ Partials       81       80       -1

Files Changed
Coverage Δ

btrack/napari/widgets/_general.py
100.00% <ø> (ø)

btrack/napari/widgets/_motion.py
100.00% <ø> (ø)

btrack/napari/widgets/_hypothesis.py
100.00% <100.00%> (ø)

... and 5 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Unable to get projection from EPSG

Hello, I was unable to get projection from EPSG code. I looked up the error and found that http://spatialreference.org has moved to https://spatialreference.org, so I already made a PR: #50. It's just a quick fix, since it would be great to migrate to epsg.io (I could work on that)

Merged #50


UNIX Sockets are supported in the Win 10 version 1809, Oct. 2018 Update

Based on attempts to bring applications like Snapmail and Acorn out into the wild bundled as Electron apps, we have run in to the requirement that Windows users install wsl2 which is a big technical hurdle for ordinary users. We have been wanting to make these apps available at lower friction. When @ddd-mtl @guillemcordoba and I discussed recently with @zippy it was brought up that this "Unix Domain Socket for Lair" thing might be the main blocker for native Windows support.
We are thinking about investigating it and seeing what we can do to move this forward. I started a bit of digging and immediately found something interesting, which appears to be that an implementation (if not 100% feature complete) of Unix sockets was brought into Windows 10, with a release in 2017 first to Windows Insider, then it became generally available in version 1809 (aka the October 2018 Update).
Microsoft open source seems to have published a library that may suit:
https://github.com/Azure/tokio-uds-windows
There's also the Tokio UDS on Windows thread to add:
https://github.com/tokio-rs/tokio/issues/2201
The new Deno language (like nodejs) is also pulling for this clearly.
Were we collectively unaware of this? I am assuming so, given these lines
https://github.com/holochain/lair/blob/3bd7105108ab241d6719e200dd15905cd3e74da1/crates/lair_keystore_api/src/internal/ipc.rs#L11-L19
and the comments in
https://github.com/holochain/lair/blob/3bd7105108ab241d6719e200dd15905cd3e74da1/crates/lair_keystore_api/src/internal/ipc/win_ipc.rs#L1
The original post is here, from 2017
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/
But there's more relevant comments here
https://github.com/microsoft/WSL/issues/4240
I found looking at this commit which referred to that issue gave a bit of insight
https://github.com/MisterDA/ocaml/commit/5855ce5ffd931a2802d5b9a5b2987ab0b276fd0a
https://github.com/holochain/lair/blob/3bd7105108ab241d6719e200dd15905cd3e74da1/.github/workflows/ops.test.yml#L14-L15
@thedavidmeister @neonphog

I was planning on looking into it this month.
The problem is with tokio since UnixStreams dont compile on windows. The
fix is to add support for TcpStreams which can we used on windows,
regardless of version.
On Thu., Jul. 1, 2021, 20:51 Connor Turland, @.***>
wrote:

Based on attempts to bring applications like Snapmail and Acorn out into
the wild bundled as Electron apps, we have run in to the requirement that
Windows users install wsl2 which is a big technical hurdle for ordinary
users. We have been wanting to make these apps available at lower friction.
When @ddd-mtl https://github.com/ddd-mtl @guillemcordoba
https://github.com/guillemcordoba and I discussed recently with @zippy
https://github.com/zippy it was brought up that this "Unix Domain
Socket for Lair" thing might be the main blocker for native Windows support.
We are thinking about investigating it and seeing what we can do to move
this forward. I started a bit of digging and immediately found something
interesting, which appears to be that an implementation (if not 100%
feature complete) of Unix sockets was brought into Windows 10, with a
release in 2017. Were we collectively unaware of this? I am assuming so,
given these lines
https://github.com/holochain/lair/blob/3bd7105108ab241d6719e200dd15905cd3e74da1/crates/lair_keystore_api/src/internal/ipc.rs#L11-L19
and the comments in
https://github.com/holochain/lair/blob/3bd7105108ab241d6719e200dd15905cd3e74da1/crates/lair_keystore_api/src/internal/ipc/win_ipc.rs#L1
The original post is here, from 2017
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/
But there's more relevant comments here
microsoft/WSL#4240 https://github.com/microsoft/WSL/issues/4240
I found looking at this commit which referred to that issue gave a bit of
insight
@.***
https://github.com/MisterDA/ocaml/commit/5855ce5ffd931a2802d5b9a5b2987ab0b276fd0a
https://github.com/holochain/lair/blob/3bd7105108ab241d6719e200dd15905cd3e74da1/.github/workflows/ops.test.yml#L14-L15
@thedavidmeister https://github.com/thedavidmeister @neonphog
https://github.com/neonphog
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/holochain/lair/issues/34, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIEPDHPZZXVYIW32FJDQJELTVUERXANCNFSM47VXJV2A
.

I see, so you think it's a better solution to add it (do you estimate it to be more work?) because it adds broader compatibility?

Also because Lair is supposed to work over TCP anyway. So it's a 2 birds
one stone solution.
On Thu., Jul. 1, 2021, 21:23 Connor Turland, @.***>
wrote:

I see, so you think it's a better solution to add it (do you estimate it
to be more work?) because it adds broader compatibility?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/holochain/lair/issues/34#issuecomment-872646698, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AIEPDHPF5GKCQCOZTPZHX23TVUII3ANCNFSM47VXJV2A
.

Cool cool. That's good and there might be one additional prospect here which is that UDP works on both mac/linux/windows AND you can select for via config to use TCP.
UnixStream for windows for tokio was implemented here by Microsoft devs themselves, so could be used:
https://github.com/Azure/tokio-uds-windows
3 birds with 2 stones?

Oh that's cool. Should be easier to use that to start with. A recent win10
version was already required for WSL anyway.
On Thu., Jul. 1, 2021, 21:33 Connor Turland, @.***>
wrote:

Cool cool. That's good and there might be one additional prospect here
which is that UDP works on both mac/linux/windows AND you can select for
via config to use TCP.
UnixStream for windows for tokio was implemented here by Microsoft devs
themselves, so could be used:
https://github.com/Azure/tokio-uds-windows
3 birds with 2 stones?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/holochain/lair/issues/34#issuecomment-872649648, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AIEPDHNDH3H3PAUHLRQZASDTVUJM7ANCNFSM47VXJV2A
.

my thoughts exactly

@Connoropolous @ddd-mtl - I hadn't been intending to introduce TCP to lair because of security concerns, although we could discuss it.
The original intention was to use tokio-named-pipes on windows.

Oh, I see. Different stories going around here then.
Sounds like the named pipes solution would be the number 1 preference for you @neonphog ?
It seems to be folded right into tokio nowadays:
https://docs.rs/tokio/1.8.0/tokio/net/windows/named_pipe/index.html

@Connoropolous

It seems to be folded right into tokio nowadays:
https://docs.rs/tokio/1.8.0/tokio/net/windows/named_pipe/index.html

Oh nice, I hadn't realized that, awesome!

Sounds like the named pipes solution would be the number 1 preference for you @neonphog ?

Yeah, if this is part of core tokio, let's do that to start, and see if it covers our use-cases. It eliminates a whole set of concerns if we can guarantee only local processes have access, and can apply os security.

@neonphog
I'm glad it's a good surprise. Can you confirm a rough outline of the work @neonphog ?
My guess is that it involves writing in the win_rpc.rs file equivalent implementations of:
IpcRead, IpcWrite, ipc_connect, and IpcServer from unix_ipc.rs
Is UnixStream completely abstracted out?

@Connoropolous

My guess is that it involves writing in the win_rpc.rs file equivalent implementations of:
IpcRead, IpcWrite, ipc_connect, and IpcServer from unix_ipc.rs
Is UnixStream completely abstracted out?

You got it. IpcRead must implement tokio::io::AsyncRead, IpcWrite tokio::io::AsyncWrite, and ipc_connect / IpcServer should have the same signatures. They're switched out at compile time with config directives here.

Actually TCP was my extrapolation for the Holo use-case and the lair-shim thing, but now I see I could be totally wrong so nevermind the TcpStream solution I mentionned.

@ddd-mtl I have access to Windows on a laptop here dual booted from a Mac, so I can co-code, or also run and compile code you write, etc. Any preferences? :)

this looks like a good thing that is happening here, but i don't have much to add so i'm taking myself off notifications :)

v0.0.1-alpha.13 keystore / api / client published with https://github.com/holochain/lair/pull/37 included. Closing this -- feel free to re-open if there's anything remaining to be addressed.


Safe to assume this package is dead?

Hasn't seen any updates over 2 years now. And the latest version hasn't even been uploaded to npm.

It clearly states in the README that this project is no longer maintained. Anyone, including you, are free to create new versions and upload them to npm.


Improve control over segmentation output filenames

Motivation for the feature
I'm trying to implement IVADOMED in AxonDeepSeg, and would like for consistency that our output masks filenames remain the same pre/post IVADOMED implementation.
See: https://github.com/neuropoly/axondeepseg/pull/547
Description of the feature
It would be nice to have more control over the segmentation output filenames.
Currently I see that I can control the "suffix" (note the quotation marks, as "_pred" always seems to be added) of the individual binary masks in the model file (at least, in the json).

I'd really like it if I could omit the "_pred" part that's added somewhere in the code.
Also, it looks like the combined mask of the binary images is set to always be called "mask" (dot) (file extension). It would be great to have more control over this filename, as 1) I don't see it set anywhere in the config files of the models and 2) we already have a file called "mask.png" in a lot of our datasets, as some of them were manually segmented and called "mask.png" to be used in training.

Oh wait, maybe I confused myself actually; does IVADOMED even output mask.png? Looks like no right? I think I confused myself as it was already in my folder; then only my first point would stand (remove the _pred suffix after the user-input suffix in the model file)

You are right, the file called mask.png is not an output of ivadomed but is the original mask from ADS.
The 2 files generated by ivadomed are image_seg-axon-manual_pred.png and image_seg-axon-manual_pred.png.
Currently in ivadomed, there are no direct options to control the output filenames.
The "_seg-axon-manual" and "_seg-myelin-manual" target_suffix refers to the names of the ground-truths used during training and testing and are not meant to be changed to control the output filenames.
The _pred part is added automatically to differenciate the ground-truth (<image_name>_seg-axon-manual.png) from the prediction (<image_name>_seg-axon-manual_pred.png).
In ADS, we will have to create the "combined" masks from these outputs with ADS suffix naming convention (_seg-axonmyelin). Similarly, I think it would be best to rename the 2 files outputted by ivadomed in ADS to follow our convention (_seg-axon, _seg-myelin).
Otherwise, I think it would be possible to remove the _pred part in ivadomed but the -manual part would remain and we would have to rename them anyway.

Hi @mathieuboudreau, I forgot to ask in the meeting this morning, is the suggestion I made above to rename ivadomed's output PNG files in ADS ok with you?

Hi @mathieuboudreau, I forgot to ask in the meeting this morning, is the suggestion I made above to rename ivadomed's output PNG files in ADS ok with you?

Sounds good to me! I did implement the _seg-axonmyelin merged mask on our side; we'll just have to change some of the default suffix keys I think also.


Enemies should kill mobile respawns

One thing thats pretty annoying in Liberation is that enemies ignore the mobile respawns. (as long as they are empty) You can drive that thing pretty much right into an occupied town and once nobody is any longer riding the MOB its safe. Thats terrible.
Enemies should take every chance to destroy a mobile respawn. It would be a better, more immersive game flow when players have to take care of their MOB, protect and defend it!

I even think the whole mobile respawn and mobile arsenal is not immersive, but yeah, I totally agree with this suggestion.
It's normal ArmA behaviour that empty vehicles are no targets for the enemy AI. I'll have a look, if there is a way to let the AI still target empty vehicles which are declared as mobile respawn classnames.

Me and my clan many times the enemy attacks our MOB (if positioned near the sector) obviously if we leave it to 300-400 meters, as we usually do ,they don't attack it

I've not found a "suitable" solution to let enemy AI attack an empty vehicle without things like "place a invisible drone or something inside the empty vehicle" etc.
If someone has a better solution, post it with explanation and source. Then I'll have a look into it again.


Let KotlinDslScriptsModel include all Kotlin scripts evaluated in the build

KotlinDslScriptsModel provides IDEs with the set of scripts of the build. Currently it only contains the settings script, projets build scripts and precompiled script plugins found in kotlin source sets. This issue is about adding all Kotlin scripts evaluated in the build
Expected Behavior
The IDE knows about all Kotlin scripts involved in a build.
Current Behavior
This IDE only knows about a simple subset of the scripts involved in a build.
Context
This is a follow up to https://github.com/gradle/gradle/pull/10776.
Also see https://github.com/gradle/gradle/issues/12639

PR #14807 continues the current naive approach with adding init scripts of the running build.
It is not a full fix of this issue though.

The new internal ScriptExecutionListener would be of good use for this.

Is there any progress on this issue that still exists on  Android Studio Giraffe | 2022.3.1 Patch 2


I can't identify the hand in the picture

I am very happy to see the report :How to implement real-time hand detection based on SSD neural network.when I in my ubuntu16.04LTS running program. I met such problem:
2018-02-08 14:34:22.983867: E tensorflow/core/common_runtime/executor.cc:643] Executor failed to create kernel. Invalid argument: NodeDef mentions attr 'T' not in Op<name=Where; signature=input:bool -> index:int64>; NodeDef: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where = WhereT=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:GPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where = WhereT=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: NodeDef mentions attr 'T' not in Op<name=Where; signature=input:bool -> index:int64>; NodeDef: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where = WhereT=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:GPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where = WhereT=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/sun/untitled/hhand_rrecognition.py", line 93, in
boxes,scores=detect_objects(image_np,detection_graph,sess)
File "/home/sun/untitled/hhand_rrecognition.py", line 81, in detect_objects
feed_dict={image_tensor: image_np_expanded})
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: NodeDef mentions attr 'T' not in Op<name=Where; signature=input:bool -> index:int64>; NodeDef: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where = WhereT=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:GPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where = WhereT=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]
Caused by op 'Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where', defined at:
File "/home/sun/untitled/hhand_rrecognition.py", line 92, in
detection_graph,sess=load_inference_graph()
File "/home/sun/untitled/hhand_rrecognition.py", line 48, in load_inference_graph
tf.import_graph_def(od_graph_def, name='')
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 313, in import_graph_def
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): NodeDef mentions attr 'T' not in Op<name=Where; signature=input:bool -> index:int64>; NodeDef: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where = WhereT=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:GPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan/Where = WhereT=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]
The tensorflow version 1.4.0rc0 GeForce GTX 1050 ,Can you give me some advice?

Hi,
This error is likely due to differences in tensorflow version.
My advice is that you use the checkpoint data I provide and export a frozen graph using your tensorflow version.
There is a discussion on this in issue #1
The exported model checkpoint is here.
A tutorial on how to export the frozen graph is here ..
-V.

Thank you for the reply.when I execute the following instructions:
python3 export_inference_graph.py \

--input_type image-tensor
--pipeline_config_path traning/ssd_mobilenet_v1_coco.config
--trained-checkpoint_prefix traning/model.ckpt-200002.data-00000-of-00001
--output-directory NEW
I met such problem:raise AttributeError('Flag --%s must be specified.' % flag_name)
AttributeError: Flag --trained_checkpoint_prefix must be specified.
Can you give me some advice？Looking forward to your reply！

Thank you for the reply.when I execute the following instructions:
python3 export_inference_graph.py \
--input_type image-tensor\
--pipeline_config_path traning/ssd_mobilenet_v1_coco.config\
--trained-checkpoint_prefix traning/model.ckpt-200002.data-00000-of-00001\
--output-directory NEW
I met such problem:raise AttributeError('Flag --%s must be specified.' % flag_name)
AttributeError: Flag --trained_checkpoint_prefix must be specified.

AttributeError: Flag --pipeline_config_path must be specified.
Can you give me some advice？Looking forward to your reply！

I am very happy for your reply.when I execute the following instructions:
python3 export_inference_graph.py \

--input_type image_tensor
--pipeline_config_path traning/ssd_mobilenet_v1_coco.config
--trained_checkpoint_prefix traning/model.ckpt-200002.data-00000-of-00001 \

--output_directory NEW
I met the problem is that:
Traceback (most recent call last):
File "export_inference_graph.py", line 119, in
tf.app.run()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 41, in run
flags_passthrough = f._parse_flags(args=args)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/flags.py", line 45, in _parse_flags
self._assert_all_required()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/flags.py", line 78, in _assert_all_required
self._assert_required(flag_name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/flags.py", line 74, in _assert_required
raise AttributeError('Flag --%s must be specified.' % flag_name)
AttributeError: Flag --pipeline_config_path must be specified.
Can you give me some advice? Looking forward to your reply!

I am very happy for your reply.when I execute the following instructions:
python3 export_inference_graph.py \
--input_type image_tensor\
--pipeline_config_path traning/ssd_mobilenet_v1_coco.config\
--trained_checkpoint_prefix traning/model.ckpt-200002.data-00000-of-00001 \

    --output_directory NEW
    I met the problem is that:
    Traceback (most recent call last):
    File "export_inference_graph.py", line 119, in
    tf.app.run()
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 41, in run
    flags_passthrough = f._parse_flags(args=args)
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/flags.py", line 45, in _parse_flags
    self._assert_all_required()
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/flags.py", line 78, in _assert_all_required
    self._assert_required(flag_name)
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/flags.py", line 74, in _assert_required
    raise AttributeError('Flag --%s must be specified.' % flag_name)
    AttributeError: Flag --pipeline_config_path must be specified.
    Can you give me some advice? Looking forward to your reply!

Thank you for your reply! Execute the following instructions:
python3 export_inference_graph.py
--input_type image_tensor
--pipeline_config_path traning/ssd_mobilenet_v1_coco.config
--trained_checkpoint_prefix traning/model.ckpt-200002.data-00000-of-00001
--output_directory NEW
I met the problems is that:
Traceback (most recent call last):
File "export_inference_graph.py", line 119, in
tf.app.run()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "export_inference_graph.py", line 115, in main
FLAGS.output_directory, input_shape)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/models/research/object_detection/exporter.py", line 422, in export_inference_graph
is_training=False)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/models/research/object_detection/builders/model_builder.py", line 83, in build
return _build_ssd_model(model_config.ssd, is_training)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/models/research/object_detection/builders/model_builder.py", line 139, in _build_ssd_model
is_training)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/models/research/object_detection/builders/model_builder.py", line 107, in _build_ssd_feature_extractor
pad_to_multiple = feature_extractor_config.pad_to_multiple
AttributeError: 'SsdFeatureExtractor' object has no attribute 'pad_to_multiple'
I have a question which of the three .ckpt files to choose .Thank you for you reply!

All of the ckpt files are for the same checkpoint model.ckpt-200002
I recommend you look at the tutorial video in the link I shared (its short), it does a good job of explaining the model export process!
Best!
-V.

Thank you very much for your help. I realized the function. I have the problems are  that  whether you test the model  in image set  and what is the accuracy rate?


restructuring: introduce bexpr

Still to do

Move relabel and subst from Behave to BExpr
use weeder

By moving file: one issue fixed and one introduced ;-(

I will fix the hlint issues (not the todo)!
@keremispirli What went wrong at SemaphoreCI: Why do'n't we get more info?

"4 parallel sequential-processes, with alternating actions" test has failed, which is from Benchmark sanity tests. I think benchmark sanity tests don't output logs for some reason. I see slight differences between how Benchmark, examples, model exploration and integration tests are running; Benchmark sanity tests go back into test/sqatt folder unlike Examples tests. I'll create a task to make them the same.


Change performance diagnostics to use Managed Identity

Add MSI support for performance diagnostics and remove SAS usage per Security Wave requirements.
Notes:

This will still not work until the MSI support for Azure Performance Diagnostics is released. The work for that is still on-going.
The VM will need to be preconfigured with these role assignments for Storage Account at Subscription level so it can inherit the permissions:
- Storage Account Contributor
- Storage Account Blob Contributor
- Storage Account Table Contributor

@lukeluong1212 let know which package version need to be updated, thanks!

@lukeluong1212 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]

Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree company="Microsoft"


No logs appearing in Papertrail

I'm trying to run logspout to put my logs in Papertrail without much luck. Here's the output of docker logs logspout:
# logspout v3.1 by gliderlabs
# adapters: udp tls raw syslog tcp
# options : persist:/mnt/routes
# jobs    : http[logs,routes]:80 pump routes
# routes  :
#   ADAPTER	ADDRESS				CONTAINERS				SOURCES	OPTIONS
#   syslog+tls	logs.papertrailapp.com:xxxx	myapp1,myapp2		map[]

Which looks right to me. Tailing the log locally over http (e.g. curl http://localhost:8000/logs) shows me the relevant log messages. However, with the 3.2.1 container, all I could see in Papertrail was a whole bunch of these:
May 24 13:39:03 my-hostname dockerd:  time="2017-05-24T11:39:03.246506810Z" level=error msg="Error streaming logs: invalid character '\\x00' looking for beginning of value"

Looks like the same thing that was reported https://github.com/gliderlabs/logspout/issues/246#issuecomment-286284186, so I tried the recommendation there, running v3.1 instead. With 3.1 the error goes away, but I still do not see a single log statement in Papertrail (the http endpoint is still logging messages as expected). Am I holding it wrong?

The logspout container doesn't have any more logs. Where can I find some information about logspout's attempts to send logs to papertrail?

I assumed that all the filters supported a comma-separated list of containers. Turns out this is not the case, so the above filter actually filters on container with name "myapp1,myapp2", instead of the two intended containers. Worked around this with labels instead. I would suggest making this limitation on filter.name explicit in the Readme (or supporting a comma-separated list of names for filter.name).

I agree it would be preferable to support a comma separated list. I haven't been in that part of the code yet but I'd be interested to see a PR for this change.

@cjohansen how did you started the logspout?
When I run it as:
docker run --restart=always --name logspout --hostname logspout -d -v=/var/run/docker.sock:/var/run/docker.sock gliderlabs/logspout:master syslog://logs6.papertrailapp.com:$PORT

works for me.


Need topic level summary metrics

Can we get the topic summary metrics as well in addition to the existing metrics.
Getting consumer group current offset, and low and high watermark at topic level would be  great feature to add on.
Thanks,
Kaushal

I am not sure if I understand your issue. There's no such thing like a group offset on a topic level?
All offsets/watermarks and lags are calculated on a per partition basis. How would it help to sum all topic partitions' low or high watermarks?

This is something I would be interested in as well, a bit more background is that on clusters with a lot of partitions, 10k+ some of the topics having 500 partitions, scraping all of these metrics at the partition level has a heavy toll on the prometheus server doing the scraping.
Although I understand these metrics aren't emitted at the topic level, it would be helpful to have topic level metrics summarized.
kafka_minion_topic_partition_high_water_mark{topic, partition}
kafka_minion_topic_partition_low_water_mark{topic, partition}
kafka_minion_topic_partition_message_count{topic, partition}
```
Could all be summed across partitions at the topic level in the exporter itself, and then either a flag could be added to determine whether someone wants partition level, or topic level metrics emitted, or even without that as long as the summary metrics existed clients could do a metric relabel with a `keep` or `drop`, to take the metrics they want.

I'm not very familiar with go, but likely need to dive in soon, but I think for larger clusters this functionality would be quite nice. Although in a large cluster I potentially would skip scraping message count, and compute that on the prometheus end.

This is something I would be interested in as well, a bit more background is that on clusters with a lot of partitions, 10k+ some of the topics having 500 partitions, scraping all of these metrics at the partition level has a heavy toll on the prometheus server doing the scraping.

Agreed, this can potentially cause problems for Prometheus! Just adding more metrics as this issues suggests doesn't fix this though.

and then either a flag could be added to determine whether someone wants partition level, or topic level metrics emitted

Makes sense to me. I am not sure whether it should be done exactly like that or if there could be more elloborated config settings.
I am not sure whether @kaushal1562 intentions align with @wulfuric wishes? Until we figure out, could you (@wulfuric ) create a separate issue for that?

@weeco , I am looking for the same details which @wulfuric added.  Both requirements are aligned.

Agreed, I think this would have a lot of value but the implementation of it is tricky.
You're right, if topic summary metrics are added this will lead to more time series metrics each scrape.  This would be the simplest on the exporter end, but it would push the burden to the customer, they could include in their prometheus scrape a list of metrics they do want to scrape.
metric_relabel_configs:
--
 - source_labels: [__name__]
    separator: ;
    regex: kafka_minion_topic_log_dir_size|kafka_minion_topic_topic_low_water_mark|kafka_minion_topic_topic_high_water_mark
    replacement: $1
    action: keep

Otherwise the exporter could be configured with flags, potentially a boolean flag for TOPIC_SUMMARY_METRICS, and PARTITION_LEVEL_METRICS, I could see cases where someone could want both enabled, which would give you the granularity of partition level metrics for deeper debugging but quicker queries with summary level metrics without needing to sum at the prometheus level.

I implemented that feature in v2 and the "granularity" is now configurable, see: https://github.com/cloudhut/kminion/blob/753af5610f21c576ee0fec71bb5280456d2e4439/docs/reference-config.yaml#L57

This looks awesome, excited to try it out thank you!

V2 is basically ready to use. I'll close this issue :)


When HTML Imports and  used together, sometimes it will cause page pending.

desc:
I've been met this problem for several months. It is very hard to reproduce it in previous version. But since v0.33.4 (and a cool thing is v0.33.3 will not meet this issue), it become 100% to reproduce the problem.
The issue relates with HTML import and webview, when they work together, a webview ipc.sendSync function will happen to insert into the HTML import request, and eventually it will block the HTML imports, which make the page stun there.
example code:
To help Electron team easily locate the problem, I create a repo for this issue: https://github.com/jwu/electron-html-import-pending, you can follow the README to run the app, it also shows how to reproduce it.
If things go right, we will see a window like this:

But if a pending happens, it will give you a blank white window like this:

Note, in this time, if the dev-tools opened (open by default), go to the Network session, and you will see a pending item remains in it. Plus the call stack is as strange as what I describe --- a syncing ipc after an event emitter:

test environment:

System: Mac OSX
Electron ver: v0.34.0. It starts quite easy to reproduce it since v0.33.4. versions below it including v0.33.3 is hard to reproduce in simple case, but the problem already there, and I can only reproduce it in complicated code.

Hope the sample code and the information can help you locate the problem.

This should have been fixed by #3833.

Hi @zcbenz ,
I test this in Electron v0.36.2, and it works! Thank you very much.
Regards,
Wu


Add config for showing username in window title

Fixes #1918
Solved my own issue as @rherms was working on #1951. Either solution is probably equally effective, this one just includes a config to make the behaviour optional.

I thought the choice would be appreciated, because appending the username at the end will cause it to be obscured on a Windows taskbar button, especially if many windows are open.

Rebased, and rewritten. It can now recognise username changes mid-game.
I still believe it's important to provide options for this feature like I mentioned in my last comment, so that toggle is unchanged.

Rebased on post-deob history.


Static runtime libraries with MinGW-w64/GCC on Windows

The recent introduction of full MinGW-w64 support in the core webview library introduced new runtime dependencies for users of our Go bindings:

libgcc_s_seh-1.dll
libstdc++-6.dll

This PR links these libraries statically by adding the -static linker flag when building the Go bindings for a better default user experience.
Instead of -static I tried to add -static-libstdc++ -static-libgcc which got rid of the need for those DLLs but instead introduced a dependency on libwinpthread-1.dll.
Closes #779.
Windows CI build is apparently green but not actually healthy. #785 fixes the build when paired with this PR.

In case this helps, I ported over some of the fixes from #785 to this PR just to show that the Windows build will become healthy again. I will revert those fixes again to reduce the scope of this PR.


I am facing a issue while CD pipeline is running. Below is the screenshot.

application is not deployed in namespace.


Disable selection of checkboxes without disabling the opening of the …

…multiselection
Basically in the previous version you couldn't disable the selection without disabling the opening of the multiselection, meaning that if a user wanted to compare selected items with not-selected items it wasn't possible.
Now, just by adding a field in the settings, you can navigate the items without modifying the selected ones. This hides automatically the 'Select all' and 'Remove all' buttons, if present. Also, it adds custom css to show the default cursor instead of the pointer.
This is usefull when you want to show already compiled forms without allowing the user to modify it (which is exactly the scenario I'm currently on).
Hope you like this little update!

Bug found, I'll refactor and then ask for another pull request


Issue at Startup

When I initially set this program up, it was working perfectly but now, it's giving me an error.  Here's the debug log.

debug log
2022-05-15 12:34:18,172 [INFO] Trying to restore the connection to the Plex server...
2022-05-15 12:34:18,681 [INFO] Successfully connected as user 'CardcaptorRLH85' (id: 1)
2022-05-15 12:34:18,682 [DEBUG] [Cache] Loading server cache from file
2022-05-15 12:34:18,687 [INFO] Starting alert listener
2022-05-15 12:34:18,717 [WARNING] Alert listener had an unexpected error
2022-05-15 12:34:19,689 [DEBUG] [Cache] Saving server cache to file
2022-05-15 12:34:19,698 [INFO] Trying to restore the connection to the Plex server...
2022-05-15 12:34:20,275 [INFO] Successfully connected as user 'CardcaptorRLH85' (id: 1)
2022-05-15 12:34:20,276 [DEBUG] [Cache] Loading server cache from file
2022-05-15 12:34:20,284 [INFO] Starting alert listener
2022-05-15 12:34:20,335 [WARNING] Alert listener had an unexpected error
2022-05-15 12:34:21,285 [DEBUG] [Cache] Saving server cache to file
2022-05-15 12:34:21,295 [INFO] Trying to restore the connection to the Plex server...

This continues for hundreds of lines.
Here's my docker-compose for this container.

docker-compose
...
  plexautolanguages:
    image: remirigal/plex-auto-languages:latest
    container_name: Plex-Auto-Languages
    environment:
      - TZ=America/Detroit
    volumes:
      - /volume1/docker/Plex-Auto-Languages:/config
    depends_on:
      - plex
    restart: unless-stopped
...

Finally, here's a redacted version of my config.yaml.

config.yaml
plexautolanguages:
  # Update language for the entire show or only for the current season
  # Accepted values:
  #   - show (default)
  #   - season
  update_level: "show"

  # Update all episodes of the show/season or only the next ones
  # Accepted values:
  #   - all (default)
  #   - next
  update_strategy: "all"

  # Whether or not playing a file should trigger a language update, defaults to 'true'
  trigger_on_play: true

  # Whether or not navigating the Plex library should trigger a language update, defaults to 'false'
  # Set this to 'true' only if you want to perform changes whenever the default track of an episode is updated, even when the episode is not played.
  # Setting this parameter to 'true' can result in higher resource usage.
  trigger_on_activity: false

  # Plex configuration
  plex:
    # A valid Plex URL (required)
    url: "<plex.example.com>"
    # A valid Plex Token (required)
    token: "<REDACTED>"

  scheduler:
    # Whether of not to enable the scheduler, defaults to 'true'
    # The scheduler will perform a deeper analysis of all recently played TV Shows
    enable: true
    # The time at which the scheduler start its task with the format 'HH:MM', defaults to '02:00'
    schedule_time: "04:30"

  notifications:
    # Whether of not to enable the notifications through Apprise, defaults to 'false'
    # A notification is sent whenever a language change is performed
    enable: false
    # An array of Apprise configurations, see Apprise docs for more information: https://github.com/caronc/apprise
    apprise_configs: []

  # Whether or not to enable the debug mode, defaults to 'false'
  # Enabling debug mode will significantly increase the number of output logs
  debug: true

I have a registered domain name that I host all of my local services on in the form <service.domain.tld>.  Since the logs show PAL connecting to my Plex server as me, I don't think this is the problem.  Also, I specifically turned debug on for diagnosing this issue but it doesn't seem to have added any detail about what the unexpected error was.

I've updated my config file to the most recent version with all of the variables set as default and that hasn't helped.
My current file follows.

config.yaml
plexautolanguages:
  # Update language for the entire show or only for the current season
  # Accepted values:
  #   - show (default)
  #   - season
  update_level: "show"

  # Update all episodes of the show/season or only the next ones
  # Accepted values:
  #   - all (default)
  #   - next
  update_strategy: "all"

  # Whether or not playing a file should trigger a language update, defaults to 'true'
  trigger_on_play: true

  # Whether or not scanning the library for new files should trigger a language update, defaults to 'true'
  # A newly added episode will be updated based on the most recently watched episode, or the first episode of the show if it has never been watched
  trigger_on_scan: true

  # Whether or not navigating the Plex library should trigger a language update, defaults to 'false'
  # Only the Plex web client and the Plex for Windows app support this feature
  # Set this to 'true' only if you want to perform changes whenever the default track of an episode is updated, even when the episode is not played.
  # Setting this parameter to 'true' can result in higher resource usage.
  trigger_on_activity: false

  # Whether or not to refresh the cached library whenever the Plex server scans its own library, defaults to 'true'
  # Disabling this parameter will prevent PlexAutoLanguages from detecting updated files for an already existing episode
  # It is recommended to disable this parameter if you have a large TV Show library (10k+ episodes)
  refresh_library_on_scan: true

  # Plex configuration
  plex:
    # A valid Plex URL (required)
    url: "<plex.example.com>"
    # A valid Plex Token (required)
    token: "<REDACTED>"

  scheduler:
    # Whether of not to enable the scheduler, defaults to 'true'
    # The scheduler will perform a deeper analysis of all recently played TV Shows
    enable: true
    # The time at which the scheduler start its task with the format 'HH:MM', defaults to '02:00'
    schedule_time: "04:30"

  notifications:
    # Whether of not to enable the notifications through Apprise, defaults to 'false'
    # A notification is sent whenever a language change is performed
    enable: false
    # An array of Apprise configurations, see Apprise docs for more information: https://github.com/caronc/apprise
    apprise_configs: []

  # Whether or not to enable the debug mode, defaults to 'false'
  # Enabling debug mode will significantly increase the number of output logs
  debug: true

I've tested with 1.1.3, 1.1.2, 1.1.1, 1.1.0, 1.0.15, and 1.0.0.  This issue seems to persist across them all even though I didn't have any issues when I initially installed it on April 24th.

Thank you for all the information you provided, it will greatly help to identify the source of the problem !
Based on the timestamp of your logs, the connection for the alert listener seems to fail immediately (less than 100ms between Starting alert listener and Alert listener had an unexpected error).
The alert listener is based on a websocket connection, so my guess would be to check your network configuration first, maybe your reverse proxy is blocking the websocket ?
Would it be possible for you to use a more direct Plex URL ?
I have improved the amount of information logged when this error occurs, can you use the docker image remirigal/plex-auto-languages:master and see if the error you get is more informative ? Thanks !

Switching my url to http://<IP_ADDRESS>:32400 seems to have fixed the problem (the container reports healthy now).  However, is the log supposed to just stop after the alert listener starts?
Here's the latest cycle of my log.  It includes the last error while I was editing my config file and the first working(?) version after the edit.

Current Log
2022-05-16 13:46:47,798 [INFO] Trying to restore the connection to the Plex server...
2022-05-16 13:46:48,313 [INFO] Successfully connected as user 'CardcaptorRLH85' (id: 1)
2022-05-16 13:46:48,314 [DEBUG] [Cache] Loading server cache from file
2022-05-16 13:46:48,320 [INFO] Starting alert listener
2022-05-16 13:46:48,321 [DEBUG] [Cache] Saving server cache to file
2022-05-16 13:46:48,330 [INFO] Stopping scheduler
2022-05-16 13:46:48,357 [ERROR] Alert listener had an unexpected error
2022-05-16 13:46:48,358 [ERROR] Handshake status 400 Bad Request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/websocket/_app.py", line 327, in run_forever
    self.sock.connect(
  File "/usr/local/lib/python3.8/site-packages/websocket/_core.py", line 248, in connect
    self.handshake_response = handshake(self.sock, url, *addrs, **options)
  File "/usr/local/lib/python3.8/site-packages/websocket/_handshake.py", line 57, in handshake
    status, resp = _get_resp_headers(sock)
  File "/usr/local/lib/python3.8/site-packages/websocket/_handshake.py", line 148, in _get_resp_headers
    raise WebSocketBadStatusException("Handshake status %d %s", status, status_message, resp_headers)
websocket._exceptions.WebSocketBadStatusException: Handshake status 400 Bad Request
2022-05-16 13:47:09,017 [INFO] Parsing config file '/config/config.yaml'
2022-05-16 13:47:09,026 [INFO] The provided configuration has been successfully validated
2022-05-16 13:47:09,026 [DEBUG] Debug mode enabled
2022-05-16 13:47:09,028 [INFO] Starting scheduler
2022-05-16 13:47:09,630 [INFO] Successfully connected as user 'CardcaptorRLH85' (id: 1)
2022-05-16 13:47:09,630 [DEBUG] [Cache] Loading server cache from file
2022-05-16 13:47:09,651 [INFO] Starting alert listener


Fix nix and qemu downloads.

The nix URL is returning HTTP 301.  We add the -L arg to make curl follow it.
Also, the qemu download URL is returning 503.  We update the URL to the one listed on the Fedora package page.
$ curl -I https://nixos.org/nix/install
HTTP/2 301
content-length: 0
date: Fri, 19 Jun 2020 06:04:07 GMT
location: https://releases.nixos.org/nix/nix-2.3.6/install
server: Netlify
via: 1.1 7452590c60991e4e4499f2a0095052b8.cloudfront.net (CloudFront)
x-amz-cf-id: 89g-QDSN-Gqa0KjKycxvhvWW_klAodcqziFnB1CqdHMS_z2e6danjw==
x-amz-cf-pop: NRT51-C2
x-cache: Miss from cloudfront
age: 2422
x-nf-request-id: 3b573f7b-eeef-41a8-93c8-0f61a4b7d1cd-1399326

$ curl -I https://kojipkgs.fedoraproject.org/packages/qemu/5.0.0/2.fc33/x86_64/qemu-user-static-5.0.0-2.fc33.x86_64.rpm
HTTP/1.1 503 Service Temporarily Unavailable
Date: Fri, 19 Jun 2020 06:45:39 GMT
Server: Apache
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
content-length: 3476
cache-control: no-cache
pragma: no-cache
content-type: text/html; charset=UTF-8
AppTime: D=360
X-Fedora-ProxyServer: proxy01.iad2.fedoraproject.org
X-Fedora-RequestID: XuxfE4uqacSqzgYDzUzqeQAABc8
Connection: close

Hey man, so sorry that you went through these issues 😞 It looks like Fedora's kojipkgs is undergoing some big maintenance and is working in limited capacity -- I was hoping the URL was going to work permanently.
Thanks a lot!


Build fails on expo web

After installed and configured, the build fails for web.
LOG OUTPUT:
/.../..../node_modules/material-bread/dist/Components/AppbarBottom/AppbarBottom.svg.js
Module not found: Can't resolve 'react-native-svg' in '/.../..../design-system-rn/node_modules/material-bread/dist/Components/AppbarBottom'
To Reproduce
Steps to reproduce the behavior:

Expo init
install material-bread
expo start -w
See error

Should i configure a webpack.config.js file? Can anyone help me?

@jascaniojah You need to install react-native-svg, just run
npm i react-native-svg

And it should be working

You can see a working example here
https://github.com/codypearce/material-bread-expo-example


Sage build is broken

Describe the bug
Whenever I run:
nixos-rebuild switch --upgrade

with sagemath listed in systemPackages the sagemath package is rebuilt, even when the latest version of SageMath available (8.9) is installed on my system. This is very irritating given that SageMath's test suite uses up a lot of CPU and RAM and takes around half an hour to an hour to complete and that's assuming it's successful, it frequently fails to run successfully on my PC.
To Reproduce
Steps to reproduce the behaviour:

Add sagemath to /etc/nixos/configuration.nix under systemPackages.
Run nixos-rebuild switch --upgrade
After your system has been successfully upgraded run the above command again and if you're observing this bug you should find that nix-build attempts to build sagemath again, despite SageMath already being installed.

Expected behaviour
I expected SageMath to only be built by Nix when it was a new install (i.e. right after I had added it to systemPackages) or when a new version of SageMath was out.
Metadata

system: "x86_64-linux"
host os: Linux 5.4.22, NixOS, 20.09pre215183.7948e6785b4 (Nightingale)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.3.3
channels(fusion809): "nixos-20.09pre215183.7948e6785b4, nixos-unstable"
channels(root): "nixos-20.09pre215183.7948e6785b4, nixos-unstable"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos-unstable

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: sagemath
# a list of nixos modules affected by the problem
module: ?

(Not sure what 'module' to list above).

/cc @timokau: thought I should ping you as this package's maintainer.

Dependencies have been updated causing a rebuild, that's the whole concept behind Nix. If you want to pin sage or any package to a specific build, you can do that as well.

Looks like this is actually caused by some breakage: https://hydra.nixos.org/job/nixos/trunk-combined/nixpkgs.sageWithDoc.x86_64-linux
It rebuilds every time because unfortunately nix doesn't cache failures. It only notices that sage is not available in the binary caches (since the build fails) and then tries to build it locally.
Do those local builds actually succeed?

Usually not. Not sure if they've ever succeeded but usually they don't.

I fixed the build in #82240 and #82239. sage should be available in the binary channels again soon, so no local rebuilds will be necessary. Thanks for the report!

Guessing the fact that tests are still failing is something I should report elsewhere?

No, here is fine. Looks like I broke it again while fixing it: https://github.com/NixOS/nixpkgs/pull/81950
Should work after https://github.com/NixOS/nixpkgs/pull/82311. Thanks again!

The build should be fixed now. This time I have double checked that it builds on master (a3735dcbd7b2530ae3bcf73fd91e1ca6aa9449e7).


Feature/294 styling issues

Description
Styling issues on rule details
Issues Resolved
Closes #294
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@amsiglan
https://user-images.githubusercontent.com/1564068/211664629-00197c7d-1363-4011-814c-7be0d5f739bc.mov


Explore landscape view filters on the same screen in a right col

Lots of changes to get in line with the designs. Here's what a list view should look like:

So an incomplete todo list might be

[ ] Show filters next to Explore content
[ ] Get rid of filters button
[ ] Make sure all filters still work, but especially every filter that opens a modal or pushes a new screen
[ ] Make sure everything still works on a phone

@abhasinat this is still the state of the current designs in Figma, but I know you've presented a lot of alternatives of late. Is this still worth doing?

Let's hold off on this for now.


How to handle permissions?

First off, great library.
I'm trying to test my CLI commands that modify the file system, but the tests get a permission denied error, while I'm able to run it locally just fine. Is there anyway to support this?

I think some more context is needed to be able to help with what you are doing.  Where you are writing to and how you are doing it are important factors.
In case you weren't aware, I tend to use https://github.com/assert-rs/assert_fs or https://github.com/assert-rs/trycmd for creating a scratchpad to write to.

@epage Not using either of those for this specific test, which is this: https://github.com/milesj/monolith/pull/5/files#diff-9a1f0aa2064ff70580494869ae0fa8a4f7c3e9b26cf3eabdfee35a0d6b17baf7R6-R9
I want to test that this code, which the CLI runs, actually works in a real world use case: https://github.com/milesj/monolith/blob/master/crates/toolchain/src/tools/node.rs#L189-L215 Right now I either get a "Permission denied" or "No such file or directory" error.

Under what conditions do you get each of those errors?
Is this local, CI, or both?
What platforms are you seeing the failure?  Know of platforms that work?
btw this is most likely a very specific setup failure or a program bug.  I'd recommend tracing in to see which call is failing and verifying the inputs to that call and make sure you are reproducing with those same inputs.

@epage It fails locally and in CI, for both mac/linux.
I'll try and trace to debug more.

@milesj I think I found the problem, but it does bring up a question. Does assert() wait for the entire child process to complete? Because this test could take many seconds.

Yes, this is for one-shot testing rather than interactive testing, so it waits for the program to finish.

This is still happening, but I've narrowed it down to this line: https://github.com/milesj/moon/blob/master/crates/toolchain/src/helpers.rs#L78
It seems like the dirs aren't being created on line 71 and I have no idea why.
This is the entire test https://github.com/milesj/moon/pull/5/files#diff-a31902d06ac6afdddf64feb76dc589f59e39a33434c352de60ba84f8beca4427R16

Not sure what changed, but this isn't an issue anymore. Closing.


[C#] refactor: update action system and AI class

Linked issues
closes: #718
Details
Change details

move history from AI class to OpenAI planner
modify the return type of action handler to string
remove AllowOverrides from ActionAttribute, add IsDefault instead
make ActionEntry internal
add TooManySteps action, STOP command
update AIOptions, AI class
TODO:

uncomment moderator review input/output, pending #805
support json schema in separate PR afterwards

Attestation Checklist

[x] My code follows the style guidelines of this project

I have checked for/fixed spelling, linting, and other errors

I have commented my code for clarity

I have made corresponding changes to the documentation (we use TypeDoc to document our code)

My changes generate no new warnings

I have added tests that validates my changes, and provides sufficient test coverage. I have tested with:

Local testing
E2E testing in Teams

New and existing unit tests pass locally with my changes

Additional information

Feel free to add other relevant information below

Code looks good to me, but still confuse on action and defaultAction. For developers, any doc/comment to understand "when to register action and when to register default action"?

I have the confusion as well. It seems that RegisterDefaultAction is designed to be used internally. I cannot think of any scenario when developers will use it. @singhk97 Any information on that?


Introduction to improve the flow of the text, and an image to keep it engaging and colorful. Note image is hypothetical of programming books as in
https://cdn.arstechnica.net/wp-content/uploads/2013/11/8396104239_ab93c016b8_z.jpg

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Improved General Flow


chore(drive): temporary disable credit transfer transitions

Issue being fixed or feature implemented

Blocks are falling on testnet when we are getting credit transfer transitions:
  2023-12-12T09:50:09.402439Z ERROR tenderdash_abci::server::generic: error processing request, error: "execution: corrupted credits not balanced error: credits are not balanced after block execution TotalCreditsBalance { total_credits_in_platform:<PHONE_NUMBER>070950, total_in_pools:<PHONE_NUMBER>4, total_identity_balances:<PHONE_NUMBER>797906 } off by 10000000", request: Request { value: Some(PrepareProposal(RequestPrepareProposal { max_tx_bytes: 22018983, txs: [[7, 0, 59, 103, 131, 253, 131, 2, 255, 161, 81, 35, 31, 43, 64, 115, 244, 37, 129, 164, 73, 60, 175, 35, 36, 93, 127, 48, 249, 8, 61, 176, 25, 207, 59, 103, 131, 253, 131, 2, 255, 161, 81, 35, 31, 43, 64, 115, 244, 37, 129, 164, 73, 60, 175, 35, 36, 93, 127, 48, 249, 8, 61, 176, 25, 207, 252, 0, 152, 150, 128, 2, 65, 31, 15, 131, 101, 178, 36, 20, 162, 77, 172, 19, 100, 178, 162, 121, 49, 109, 231, 176, 6, 155, 231, 247, 27, 45, 101, 57, 158, 44, 154, 119, 237, 167, 48, 32, 73, 155, 15, 250, 246, 183, 228, 40, 108, 145, 191, 85, 164, 211, 111, 188, 98, 18, 213, 95, 87, 138, 148, 227, 190, 30, 175, 136, 216, 112]], local_last_commit: Some(CommitInfo { round: 1, quorum_hash: [0, 0, 0, 60, 253, 45, 211, 80, 215, 228, 229, 232, 208, 64, 45, 10, 141, 223, 36, 73, 243, 71, 71, 58, 126, 15, 217, 168, 146, 69, 129, 128], block_signature: [169, 57, 68, 247, 132, 28, 176, 47, 237, 74, 178, 228, 113, 113, 97, 207, 46, 210, 57, 159, 125, 212, 128, 184, 245, 224, 86, 90, 211, 226, 37, 19, 69, 174, 220, 47, 191, 190, 42, 30, 146, 178, 110, 75, 107, 97, 211, 207, 14, 158, 107, 94, 128, 174, 91, 105, 113, 31, 25, 150, 24, 152, 209, 232, 203, 224, 24, 14, 200, 85, 99, 233, 89, 160, 51, 78, 58, 207, 91, 144, 1, 18, 211, 160, 72, 48, 104, 40, 166, 206, 236, 236, 75, 70, 86, 42], threshold_vote_extensions: [] }), misbehavior: [], height: 10224, time: Some(Timestamp { seconds:<PHONE_NUMBER>, nanos: 395000000 }), next_validators_hash: [124, 129, 104, 78, 32, 216, 186, 20, 27, 187, 9, 88, 136, 223, 119, 108, 112, 194, 225, 132, 208, 2, 16, 89, 103, 214, 24, 89, 195, 64, 219, 201], round: 0, core_chain_locked_height: 929843, proposer_pro_tx_hash: [57, 116, 26, 216, 61, 215, 145, 225, 231, 56, 241, 158, 218, 232, 45, 108, 3, 34, 151, 46, 106, 69, 89, 129, 66, 77, 163, 118, 155, 61, 189, 74], proposed_app_version: 1, version: Some(Consensus { block: 13, app: 1 }), quorum_hash: [0, 0, 0, 60, 253, 45, 211, 80, 215, 228, 229, 232, 208, 64, 45, 10, 141, 223, 36, 73, 243, 71, 71, 58, 126, 15, 217, 168, 146, 69, 129, 128] })) }

To keep testnet available for testing and benchmarks we do not accept credit transfer transitions to mempool until the bug is fixed
What was done?

Reject credit transfer transition on check tx
Skip credit transfer tests in test suite

How Has This Been Tested?

None
Breaking Changes

None
Checklist:

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have added or updated relevant unit/integration/functional/e2e tests
[x] I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
[x] I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

[x] I have assigned this pull request to a milestone

@shumkov why don't you just make a release based off this branch?


Fix issue with Snyk scan report processor

Similar to issue in hedera-services repo https://github.com/hashgraph/hedera-services/issues/11266
The source provided is not a valid json! Please validate that the input provided to the CLI is an actual JSON

Tip: To find more information, try running `snyk-to-html` in debug mode by appending to the CLI the `-d` parameter

Error running `snyk-to-html`. Please check you are providing the correct parameters. Is the issue persists contact<EMAIL_ADDRESS> html-to-markdown: Error: File /runner/_work/hedera-services/hedera-services/snyk-code.html does not exist!
    at file:///opt/hostedtoolcache/node/16.20.2/x64/lib/node_modules/@wcj/html-to-markdown-cli/lib/cli.js:53:23
    at file:///opt/hostedtoolcache/node/16.20.2/x64/lib/node_modules/@wcj/html-to-markdown-cli/lib/cli.js:76:3
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:5[30](https://github.com/hashgraph/hedera-services/actions/runs/7717428954/job/21037352777?pr=11233#step:53:31):24)
    at async loadESM (node:internal/process/esm_loader:91:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12)

:tada: This issue has been resolved in version 0.20.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Implemented the connection pool and added basic tests.

See lukeb-uk/node-promise-mysql/pull/9

At the moment only simple basics tests are implemented using mocha. To run the tests, you need to connect to a running MySQL server. A database or write permissions are not required.
Start the test suite with
DB_HOST=localhost DB_USER=user DB_PWD=pwd npm test

Looks good to me and test all pass, merging!

Can you push this to npm as well?

I have a little merging to do between master and the pool branch, which I'm doing now, but I will push as soon as I've finished

@naturalethic this has been pushed to npm now.

:+1: Thanks!

big ups!

Oops -- I just noticed there is no way to release connection.  I see the incomplete tests in there for that as well.  @pdaether you want to add that?

Yes, i can add this.
At the moment, i only use this module in an high traffic  service without transaction and other stuff, so I only use the query-method directly on the pool object.
Will add this asap.


Turning off receiveShadows with VSM causes crash

Describe the bug
If a gltf lit shader has receiveShadows turned off when VSM is enabled, there is a consistent crash when rendering the scene.
To Reproduce
Steps to reproduce the behavior:
Turn off receiveShadows on an imported gltf model with the lit shader on while using VSM type shadows.
Expected behavior
Rendering without a crash.
Logs
Panic
in filament::backend::Program filament::FMaterial::getProgramBuilderWithVariants(uint8_t, uint8_t, uint8_t) const:434
reason: The material 'ubershader_lit_opaque' has not been compiled to include the required GLSL or SPIR-V chunks for the fragment shader (variant=0x41, filtered=0x41).

Backend: Metal

I'm able to reproduce with gltf_viewer. The relevant assertion is:
reason: ../../filament/src/Material.cpp:366: failed assertion `!Variant::isReserved(variantKey)'


Redundant quotation marks included in the version string

I'm not sure whether this is a problem with my Serilog config but I have the following issue.
When the Serilog Raygun sink emits a log event it sets the Version property of the Raygun message details before sending the error message to Raygun. However, an unnecessary set of quote marks gets included in the version string (e.g. "\"1.0.39\""). When viewed in Raygun the error event looks like this. This is incorrect with the result being that Raygun is unable to figure out which deployment the error belongs to. Seems to be related to the ToString() this line.

Hi, thanks for the report. Looks like it should be ToString("l", null).

I've submitted pr #10

Great - thanks! Build should be out of CI shortly 👍


Suggestions

Please consider updating the template to target net6.0. It would also be helpful if you mentioned TypeScript is included as part of the README or in the description of the extension in the Visual Studio marketplace.

Thank you for the Suggestions, I will try to migrate later : )
Maybe end of this month, A little bit busy, lol.

#2

Support Vite2.x and Vite3.xSupport Asp.Net Core 6 (Minimal API and Controllers)Support 9 templates -- both Vue and React (list below)
| Vue + TypeScript | Vue | React + TypeScript | React
-- | -- | -- | -- | --
Asp.Net Core MVC 5 | O | X | X | X
Net 6 Use Controllers | O | O | O | O
Net 6 + Minimal API | O | O | O | O


Xapid 634 (Perf testing via docker)

Greg,
This will issue artillery based perf test for change/snapshot server, and cleanup. The json files can then be fed into grafana.
Thanks
Sundar

Can you change your github email to your Google.com address and re-submit?
On Wed, Feb 15, 2017 at 7:46 PM Sundar Ramamoorthy<EMAIL_ADDRESS>wrote:

Greg,
This will issue artillery based perf test for change/snapshot server, and
cleanup. The json files can then be fed into grafana.
Thanks
Sundar
You can view, comment on, or merge this pull request online at:
https://github.com/apigee-labs/transicator/pull/48
Commit Summary

artillery based load gen files for change/snapshot server.
New Tests for performance. Fixed shell script for docker invocation.
More modifications to perf tests.
Include the DB data generating docker file.
Added docker based perf testing using artillery.

File Changes

A test/Dockerfile.dbdatagen
https://github.com/apigee-labs/transicator/pull/48/files#diff-0 (8)
A test/Dockerfile.postgrest
https://github.com/apigee-labs/transicator/pull/48/files#diff-1 (15)
A test/artillery/change-test.yaml
https://github.com/apigee-labs/transicator/pull/48/files#diff-2 (72)
A test/artillery/changehandler.js
https://github.com/apigee-labs/transicator/pull/48/files#diff-3 (78)
R test/dockerperftest.sh
https://github.com/apigee-labs/transicator/pull/48/files#diff-4 (78)
D test/dockerperftest_cs.sh
https://github.com/apigee-labs/transicator/pull/48/files#diff-5 (87)
M test/loadgen.go
https://github.com/apigee-labs/transicator/pull/48/files#diff-6 (97)

Patch Links:

https://github.com/apigee-labs/transicator/pull/48.patch
https://github.com/apigee-labs/transicator/pull/48.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/apigee-labs/transicator/pull/48, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAf0a8lguB7hcxXt8UlfMM6WMCZzGvIVks5rc8aSgaJpZM4MCjRb
.

Thanks, I was logged in with my apigee account, hence the warning.


Config list command

Implement command to list all detected settings along with the read paths.

Implemented in https://github.com/resin-io/resin-cli/pull/265.


set: refuse from getMaskInv

Codecov Report

Merging #76 into master will decrease coverage by 0.08%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #76      +/-   ##
==========================================
- Coverage   79.85%   79.77%   -0.09%
==========================================
  Files          22       22
  Lines         988      984       -4
==========================================
- Hits          789      785       -4
  Misses        180      180
  Partials       19       19

Impacted Files
Coverage Δ

set/sparsebitset.go
100% <100%> (ø)
:arrow_up:

set/bitset.go
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 75cc1a2...28d7bb3. Read the comment docs.


Proper way to get the diff in a CSV file

Hi,
I like your tool!
How can I diff two files and export the difference in a CSV file (without patching any file)?

Ok, there was a mistake in my setup, sorry.


table of contents not working

This doesn't work in markdown documents:
```eval_rst
.. contents:: Table of Contents
  :depth: 3
```.

This error occurs:
ERROR: The "contents" directive may not be used within topics or body elements.

This error is appearing for me as well.


Motherlode Mine - Gem overlay

When mining with an alumet of glory the chance of receiving uncut gems increases significantly.
This introduces an overlay to the Motherlode Mine Plugin which shows the numer of gems found during the current mining session.
Screenshot:

We mentioned this on discord, but isn't this change applicable to gem mining in general, not just the Motherlode Mine?

I agree it should probably be in a mining specific plugin. I can create a mining specific plugin instead and PR that.
If I create a mining specific plugin, should I merge the Motherlode Mine plugin with it or leave it as is and exclude Motherlode Mine from the Mining plugin?

IMHO I think it's worth keeping them in the same plugin, and just renaming it to Mining, unless anyone else has an objection. I don't think the Motherlode's mechanics are unique to deserve their own plugin, and it's not even listed on the Minigames list.


image lazy loading is broken when use with thumbnail plugin

https://github.com/VLZH/react-lightgallery/blob/f844b9f68e6b3ca74963114791bcc2c9e5478bc3/src/LightgalleryItem.js#L36
src and thumb are the same => thumbnail plugin will use scaled full images to create its thumbs => it will load full list of images.
May be use different property for thumb?
thumb: this.props.thumb || this.props.src
workaround with extend:
export class LightgalleryItemWithThumbs extends LightgalleryItem {
    register = () => {
        this.context.registerPhoto(this.state.id, this.props.group, {
            src: this.props.src,
            thumb: this.props.thumb || this.props.src,
            subHtml: this.props.subHtml || "",
            downloadUrl: this.props.downloadUrl || ""
        });
    };
}

@shadowreplicant Thank you for this issue! You were absolutely right; Version with fix: 0.6.1


Customize validation message when looking through project contents

Should clarify the "Invalid input" message

Prompt code is here: https://github.com/ScilifelabDataCentre/dds_cli/blob/3396e7275dc008e15a2acea906f31e6b205b2ab9/dds_cli/main.py#L179-L184
I can't immediately see an option to customise the error message (though I feel like I have done this before 🤔 ), so the easy option is to just print an additional log message making it super clear what is expected.
I'm actually not sure if the validate  function is required here as we are already providing a list of choices. So it could be that simply removing this line gives a nicer error message.

Moving to Jira.


Authentication is not working.

Is authenticating but redirects to the dashboard page.

yes, that is it, works! did you see your name logged on the top?

You are right.


Fix  Entity Type References on merge

Is your feature request related to a problem? Please describe.
After entity type merging, some fields refer to the entity extension type object and some to Base Entity Type.  All type references should be on the merged Entity Type
Describe the solution you'd like
In XtextGraphFolder, use Xtext's UsageCrossReferencer to update type references on the operation  (Query, Mutation, Subscription).   References to Type extension should be replaced with the base type since, the current algorithm, extensions are merged to the base type.

Already fixed


Bubbles are Pumpkins and Signs are Oak Posts

Bubbles are Pumpkins and Signs are Oak Posts just saw thought it would help to let you know.

Fixed in next update


Docs fixes + Code coverage

LICENSE converted to md file
Updated README
Code consistency - removed empty lines before license headers
Added coveralls configuration
Added .gitattributes
Update travis configuration to run test coverage and test over PHP 7.1
Changed PHPCodeSniffer to zendframework/zend-coding-standard ~1.0.0
Updated badges to use SVG images and master branch

Please enable coveralls for the package

Changes Unknown when pulling 91c202e0326b93c017c9969417b47b909bc51adb on webimpress:fix/docs into ** on zfcampus:master**.

Thanks, @webimpress.


kubelet test is flaky

Appears to deadlock or otherwise hang after a few seconds and then hit the 5-minute timeout.
From start of log:
W0406 21:21:50.436464   21680 docker.go:482] found a container with the "k8s" prefix, but too few fields (2): "k8s_unidentified"
...
E0406 21:21:53.397806   21680 server.go:614] Timed out waiting for client to create streams
panic: test timed out after 5m0s

followed by stack traces of zillions of goroutines that mostly report being blocked for 4+ minutes.
Recent instances:
https://travis-ci.org/GoogleCloudPlatform/kubernetes/jobs/57351393
https://travis-ci.org/GoogleCloudPlatform/kubernetes/jobs/57389027
cc @dchen1107

I tried to reproduce the issue by run kubelet unittests 200 times (100 for v1beta1, and 100 for v1beta3), but failed to reproduce the issue.
$ hack/test-go.sh -i 100 pkg/kubelet/
Running tests for APIVersion: v1beta1
+++ [0406 16:42:23] Running 100 times
+++ [0406 16:42:23] github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet/
...
+++ [0406 16:55:17] 100 / 100 passed
Running tests for APIVersion: v1beta3
+++ [0406 16:55:17] Running 100 times
...
+++ [0406 17:08:11] 100 / 100 passed

I poked around, and found the only difference between my local machine and test machine is that I am running go 1.4.2 and test machine is running 1.3 (both failure cases posted by @bgrant0607 are with 1.3.0). I remembered from online I saw a deadlock issue reported against old golang version, but haven't found that issue yet. I will continue trying to reproduce the issue.

Saw this in travis runs too:
https://travis-ci.org/GoogleCloudPlatform/kubernetes/jobs/57428080

Yes, there have been multiple failures today. Check shippable, also.

FWIW, these are not all kubelet tests failing, but all of them have many kubelet go routines in the trace - mostly stuck in managePodLoop().

 Yu-Ju just pointed out that the failure posted above by @jnagal is go1.4. Shippable is in a really bad shape today, tons of different tests are failed, which should be handled separately.

Ok, with some hacky debugging message added into kubelet package, even I couldn't reproduce the very failure report here, I made some progress on understanding the issue here. Basically there are two issues:

The flaky tests reported here are TestServeExecInContainer. All reports posted here have the same stack trace below:

goroutine 364 [chan send, 2 minutes]:
github.com/docker/spdystream.(*idleAwareFramer).WriteFrame(0xc20810d560, 0x7f80bb632478, 0xc208470460, 0x0, 0x0)
	/home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/Godeps/_workspace/src/github.com/docker/spdystream/connection.go:116 +0x18e
github.com/docker/spdystream.(*Stream).WriteData(0xc208377400, 0xc2085b2640, 0x5, 0x8, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/Godeps/_workspace/src/github.com/docker/spdystream/stream.go:63 +0x389
github.com/docker/spdystream.(*Stream).Write(0xc208377400, 0xc2085b2640, 0x5, 0x8, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/Godeps/_workspace/src/github.com/docker/spdystream/stream.go:68 +0x78
github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet.TestServeExecInContainer(0xc208038510)
	/home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet/server_test.go:856 +0x1cac
testing.tRunner(0xc208038510, 0x1665e40)
	/usr/local/go/src/pkg/testing/testing.go:422 +0x110
created by testing.RunTests
	/usr/local/go/src/pkg/testing/testing.go:504 +0xb46

There are tons of goroutines, especially one created for each pod. Kubelet leak those goroutines.

I0407 12:40:29.015028   23740 server_test.go:681] ???? BEGINE TestServeExecInContainer with 106 goroutines
I0407 12:40:29.037029   23740 server_test.go:914] ???? ENDING TestServeExecInContainer with 147 goroutines

One can see when test initial run, there are 106 goroutines. Test itself create another 41 goroutines, but none of them are created for podWorkder.

FWIW, the tests in server_test.go were creating go routines it didn't need at all. I just removed them in #6527

@yujuhong Thanks for cleanup. Now the number down to 91:
I0407 12:50:20.218265   25918 server_test.go:676] ???? BEGINE TestServeExecInContainer with 91 goroutines
I0407 12:50:20.241959   25918 server_test.go:909] ???? ENDING TestServeExecInContainer with 125 goroutines

cc/ @ncdc

Did something change related to this recently? I haven't heard any flakiness reports for a long time, and now they're starting up again. Any ideas?
Sent from my iPhone

On Apr 7, 2015, at 7:37 PM, Dawn Chen<EMAIL_ADDRESS>wrote:
cc/ @ncdc
—
Reply to this email directly or view it on GitHub.

I didn't see any changes related to this feature. Actually I even couldn't reproduce it on my desktop.
Please note that the comment I made above https://github.com/GoogleCloudPlatform/kubernetes/issues/6490#issuecomment-90709952:
all those leftover goroutines are separate issue thought.

This has exposed a deadlock in spdystream.
I'll work with @dmcgowan to come up with a fix.

Upstream issue https://github.com/docker/spdystream/issues/49

Cool ncdc@ !  dchen1107@, by the way, I've run across a few similar bugs recently where their reproducibility hinges mainly on how the test cluster was brought up.  If the test cluster comes up right, then the tests pass 100% forever.  If the cluster comes up bad, the tests fail, sometimes forever.  Were you rerunning your tests against the same test cluster?  Or rebuilding your test cluster between every test invocation?

@quinton-hoole This is unittests, not e2e. We saw test flakiness through travis and shippable, but never on my desktop.

This will hopefully fix the deadlock: https://github.com/docker/spdystream/pull/50. I will do a bump commit once it's merged.

@ncdc Thanks for your quick fix. I will clean up goroutines after that.

@dchen1107 it's entirely possible there's some bugs in my exec & port forwarding code around leaking goroutines. Are you going to take an initial look to see what's sticking around?

Yes, your test leak some goroutine, but other tests too.