_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m 20s
master passed

+1 :green_heart:
compile
0m 31s
master passed

+1 :green_heart:
checkstyle
0m 14s
master passed

+1 :green_heart:
spotbugs
0m 35s
master passed

_ Patch Compile Tests _

-1 :x:
mvninstall
1m 45s
root in the patch failed.

-1 :x:
compile
0m 10s
hbase-http in the patch failed.

-0 :warning:
javac
0m 10s
hbase-http in the patch failed.

-0 :warning:
checkstyle
0m 11s
hbase-http: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

-1 :x:
hadoopcheck
1m 36s
The patch causes 10 errors with Hadoop v3.1.2.

-1 :x:
hadoopcheck
3m 15s
The patch causes 10 errors with Hadoop v3.2.1.

-1 :x:
hadoopcheck
4m 55s
The patch causes 10 errors with Hadoop v3.3.0.

-1 :x:
spotbugs
0m 10s
hbase-http in the patch failed.

_ Other Tests _

+1 :green_heart:
asflicense
0m 12s
The patch does not generate ASF License warnings.

15m 39s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3549

JIRA Issue
HBASE-26160

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile

uname
Linux d07bcb77d02f 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / f3175c5ab6

Default Java
AdoptOpenJDK-1.8.0_282-b08

mvninstall
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/patch-mvninstall-root.txt

compile
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/patch-compile-hbase-http.txt

javac
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/patch-compile-hbase-http.txt

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-http.txt

hadoopcheck
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/patch-javac-3.1.2.txt

hadoopcheck
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/patch-javac-3.2.1.txt

hadoopcheck
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/patch-javac-3.3.0.txt

spotbugs
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/artifact/yetus-general-check/output/patch-spotbugs-hbase-http.txt

Max. process+thread count
86 (vs. ulimit of 30000)

modules
C: hbase-http U: hbase-http

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/3/console

versions
git=2.17.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@jojochuang let me know if this looks good with the latest commit. If so, I'll squash and clean up the commits for merging.
I'd also like to get this merged into branch-2 and branch-2.4. The LogLevel changes apply cleanly, but TestLogLevel has a bunch of merge conflicts. I'll need to submit a PR which reconciles those once ready.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m  8s
master passed

+1 :green_heart:
compile
0m 28s
master passed

+1 :green_heart:
checkstyle
0m 11s
master passed

+1 :green_heart:
spotbugs
0m 31s
master passed

_ Patch Compile Tests _

-1 :x:
mvninstall
1m 44s
root in the patch failed.

-1 :x:
compile
0m 10s
hbase-http in the patch failed.

-0 :warning:
javac
0m 10s
hbase-http in the patch failed.

-0 :warning:
checkstyle
0m 10s
hbase-http: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

-1 :x:
hadoopcheck
1m 38s
The patch causes 10 errors with Hadoop v3.1.2.

-1 :x:
hadoopcheck
3m 16s
The patch causes 10 errors with Hadoop v3.2.1.

-1 :x:
hadoopcheck
4m 51s
The patch causes 10 errors with Hadoop v3.3.0.

-1 :x:
spotbugs
0m 10s
hbase-http in the patch failed.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 48s
master passed

+1 :green_heart:
compile
0m 33s
master passed

+1 :green_heart:
checkstyle
0m 16s
master passed

+1 :green_heart:
spotbugs
0m 39s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 49s
the patch passed

+1 :green_heart:
compile
0m 31s
the patch passed

+1 :green_heart:
javac
0m 31s
the patch passed

-0 :warning:
checkstyle
0m 13s
hbase-http: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
18m 50s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.

+1 :green_heart:
spotbugs
0m 45s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 14s
The patch does not generate ASF License warnings.

37m 57s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/5/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3549

JIRA Issue
HBASE-26160

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile

uname
Linux b3c5d5a56f3f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / f3175c5ab6

Default Java
AdoptOpenJDK-1.8.0_282-b08

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-http.txt

Max. process+thread count
96 (vs. ulimit of 30000)

modules
C: hbase-http U: hbase-http

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/5/console

versions
git=2.17.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Looks like all checks are passing now. Thanks whoever fixed that :)

@jojochuang thanks for the review! As promised, I've squashed the commits here and submitted #3558 and #3559 for porting to branch 2 and 2.4.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m 21s
master passed

+1 :green_heart:
compile
0m 31s
master passed

+1 :green_heart:
checkstyle
0m 14s
master passed

+1 :green_heart:
spotbugs
0m 35s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m  2s
the patch passed

+1 :green_heart:
compile
0m 29s
the patch passed

+1 :green_heart:
javac
0m 29s
the patch passed

-0 :warning:
checkstyle
0m 12s
hbase-http: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
20m  3s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.

+1 :green_heart:
spotbugs
0m 44s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 12s
The patch does not generate ASF License warnings.

40m 48s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/6/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3549

JIRA Issue
HBASE-26160

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile

uname
Linux 00c6e52a11e4 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / f3175c5ab6

Default Java
AdoptOpenJDK-1.8.0_282-b08

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/6/artifact/yetus-general-check/output/diff-checkstyle-hbase-http.txt

Max. process+thread count
86 (vs. ulimit of 30000)

modules
C: hbase-http U: hbase-http

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3549/6/console

versions
git=2.17.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


Why precompute mask for local neighborhood?

I interest your work but have two questions in 'main.py'.
First, why precompute masks for local neighborhood?
Second, why leverage this manner to precompute the mask?
I'm a fresher for GCN. Thank you.

The mask is used to exclude the local pixel from the nearest neighbor search. Since the local pixels are already used in the local 2D convolution we don't want to also use them in the nonlocal aggregation. The mask is constructed to be multiplied times the pairwise distance matrix D computed in the layers and sets the distances of the local pixels to zero so that we know that they will always be the first ones when we sort by distance and we can exclude them.

Thank you so much. By the way, I think the precomputed mask matrix is similar to the adjacent matrix because the dimension of the matrix is N*N.
Is it right?

Yes, it is.

Thank you. I see.


Make handler functions return status value

Public-Facing Changes
None
Description

Refactors the way handler functions are run in a separate thread. It's better to do that in a unified way in the server rather than doing that for ROS 1 & 2 differently as it was implemented in #165
Change handler function signature to return a success status and an additional error message which will be sent to the client if the handler did not succeed.

Fixes #148

This seems like a good change overall. I have seen C++ codebases benefit greatly from a Status type, e.g. https://abseil.io/docs/cpp/guides/status, so that would be a nice improvement/cleanup to consider. I'm not sure if there is one already in the ROS package ecosystem?

I wonder if we should not simply throw exceptions if something went wrong. We already use exceptions in other places, so there is no good reason to not use them. This seems to be also what the C++ core guidelines recommend: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#e25-if-you-cant-throw-exceptions-simulate-raii-for-resource-management


debugger: TypeError: Cannot read property 'map' of undefined

VSCode Version: 1.24.1
OS Version: windows 8.1

Steps to Reproduce:

clone Microsoft/Typescript
npm i
set launch.json that add a mocha test config

{
      "type": "node",
      "request": "launch",
      "name": "Mocha Tests",
      "program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
      "args": [
        "-R",
        "mocha-fivemat-progress-reporter",
        "-g",
        "tests/cases/compiler/reversedWordsWithBindingElement.ts",
        "${workspaceFolder}/built/local/run.js"
      ],
      "internalConsoleOptions": "openOnSessionStart"
    },

debug

SourceMaps.getMapForGeneratedPath: exception while processing path: c:\Users\xxx\Documents\TypeScript\built\local\run.js, sourcemap: run.js.map
TypeError: Cannot read property 'map' of undefined
    at new SourceMap (c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\ms-vscode.node-debug2\node_modules\vscode-chrome-debug-core\out\src\sourceMaps\sourceMap.js:40:35)
    at sourceMapContentsP.then.contents (c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\ms-vscode.node-debug2\node_modules\vscode-chrome-debug-core\out\src\sourceMaps\sourceMapFactory.js:53:28)
    at <anonymous>

Does this issue occur when all extensions are disabled?: Yes

1.24.1 on osx also has this problem

Seems like this would happen only if the sourcemap has no sources array which would make it not a very useful sourcemap. I'll try it...

I see that they are now outputting sourcemaps that use sourcemap 'sections' which I haven't implemented. Got https://github.com/Microsoft/vscode-chrome-debug-core/issues/217 for this.


Add fallback logic to githubusercontent.com

Every version of the registry is committed to github before publishing (https://github.com/graphprotocol/networks-registry/tree/main/public), so we can use that location in case request to registry.thegraph.com fails.

Implemented for TS and Rust libs


Feedback Quick Start page

OVERVIEW :
Create a clear separation in between BBN aims / properties,
and the example of MoviLens dataset afterwards (under the voice 'Example').
In Ingest data section as well.
LIST DATA:

"The full payload of the resources are not retrieved when listing them: only identifier, type as well as Nexus added metadata are. But the result list can be scrolled and each resource fetched by identifier." ?
JSON-LD link goes back to the beginning of the page
QUERY DATA:

in the tab = ElasticSearch is linked to the beginning of the page
https://nexus-sandbox.io/v1/views/tutorialnexus/$PROJECTLABEL/documents/_search and
https://nexus-sandbox.io/v1/views/tutorialnexus/$PROJECTLABEL/graph/sparql
not working:
{"@context":"https://bluebrain.github.io/nexus/contexts/error.json","@type":"AuthorizationFailed","reason":"The supplied authentication is not authorized to access this resource."}

some more additional comments for a basic user friendly system :)
in LOGIN SECTION:
add the command for create the variable TOKEN
export TOKEN=[copy here your token]
load datasets takes around 40 mins
with the command
nexus resources fetch https://nexus-sandbox.io/v1/resources/tutorialnexus/$PROJECTLABEL/_/Movie_1
i always crash into 404 error
Should be mentioned how to connect ElasticSearchView\SparqlView to Nexus?


Add license checker scripts

Description
License verification automation + Use latest eslint.
License list:

Fixes # (issue)
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have signed the Adobe Open Source CLA.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Same comment as in https://github.com/adobe/asset-compute-sdk/pull/7#pullrequestreview-410289952 - I think we should move this into a CI job.


Feedly support?

I know, newsbeuter not intended to be Feedly client but still would be nice to have Feedly integration.

Yeah, it's been actively requested back in #63. I took a look at Feedly API some time ago—they use OAuth, for which we don't have support in Newsbeuter yet, so adding Feedly isn't quite as simple as adding some other APIs was.
I'm not working on this at the moment, but if you—or anyone else—want to, I'm glad to discuss and help with refactoring Newsbeuter to add Feedly to the mix.

+1

@lony, you just mailed 55 people. Please use reactions instead.


Don't show an empty list of findings after project is opened

The reload of findings was breaking because of a threading issue.
The non-blocking ReadAction cancels the operation after the results are calculated. We used to call onSuccess(), which unfortunately may happen after or during the cancellation is triggered. This could cause an empty or an incomplete list of findings at startup.
This fix is hopefully fixing this for good. Unfortunately, I don't know of a reliable way to test this with a unit test.

Maybe there is a way to use mocking to setup the race condition that you're
trying to test?
On Fri, Jun 16, 2023 at 3:14 PM Joachim Ansorg @.***>
wrote:

The reload of findings was breaking because of a threading issue.
The non-blocking ReadAction cancels the operation after the results are
calculated. We used to call onSuccess(), which unfortunately may happen
after or during the cancellation is triggered. This could cause an empty or
an incomplete list of findings at startup.
This fix is hopefully fixing this for good. Unfortunately, I don't know of
a reliable way to test this, because it's happening during project init and
only happens on some machines (e.g. machines with many cpu threads).
You can view, comment on, or merge this pull request online at:
https://github.com/getappmap/appmap-intellij-plugin/pull/307
Commit Summary

de2bc54
https://github.com/getappmap/appmap-intellij-plugin/pull/307/commits/de2bc542fb85232c61100f8836ba704fffd3f298
fix: avoid empty list of findings after a project is opened
7b1277e
https://github.com/getappmap/appmap-intellij-plugin/pull/307/commits/7b1277ed79e495f8e0c2543a759a06f183a45684
chore: attempt to fix unstable test AppmapMetadataSampleFileTest

File Changes
(2 files
https://github.com/getappmap/appmap-intellij-plugin/pull/307/files)

M
plugin-core/src/main/java/appland/problemsView/FindingsManager.java
https://github.com/getappmap/appmap-intellij-plugin/pull/307/files#diff-dd45c0de0c7813b883b5537d795677bb0d0b435d172886817d1d703ace30e6cf
(22)
M plugin-core/src/test/java/appland/AppMapBaseTest.java
https://github.com/getappmap/appmap-intellij-plugin/pull/307/files#diff-eab6aaadec9d9f9b31877b377e4fd413a9ae1af17170802e17770f01a74c94dc
(11)

Patch Links:

https://github.com/getappmap/appmap-intellij-plugin/pull/307.patch
https://github.com/getappmap/appmap-intellij-plugin/pull/307.diff

—
Reply to this email directly, view it on GitHub
https://github.com/getappmap/appmap-intellij-plugin/pull/307, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAVC63Z6C37FTJ7GBCLYVTXLSWABANCNFSM6AAAAAAZJUKN5A
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

:tada: This PR is included in version 0.37.0 :tada:
The release is available on:

v0.37.0
GitHub release

Your semantic-release bot :package::rocket:


[Wiki module design] Azure Resource Reference (ARR) alignment - Parameter names (with what exceptions, e.g., cmk or networking input parameters that we are trying to align across all resources supporting those)

One idea to test:
For a block like
    "keyVault": {
      "type": "Microsoft.KeyVault/vaults",
      "apiVersion": "2022-07-01",
      "name": "[parameters('name')]",
      "location": "[parameters('location')]",
      "tags": "[parameters('tags')]",
      "properties": {
        "enabledForDeployment": "[parameters('enableVaultForDeployment')]",
        "enabledForTemplateDeployment": "[parameters('enableVaultForTemplateDeployment')]",
        "enabledForDiskEncryption": "[parameters('enableVaultForDiskEncryption')]",
        "enableSoftDelete": "[parameters('enableSoftDelete')]",
        (...)
}

Check if there are any occurrences like (singular) resource type (in this case vault) + the property name.
Example

Are there any properties like vault*name, vault*enabledForDeployment etc.

If so it should throw an error
If not, it should be accepted

We should test this and see if we find any false positives or if it works as intended


Fix 22

Pull Request Checklist:

[x] pre-commit hooks are installed/active in my local clone ($ pre-commit install)
[x] This PR addresses an already opened issue (for bug fixes / features)

This PR fixes #22

[x] (If applicable) Documentation has been added / updated (for bug fixes / features)
[x] HISTORY.rst has been updated (with summary of main changes)

[x] Link to issue (:issue:number) and pull request (:pull:number) has been added

[ ] bumpversion patch has been called on this branch

What kind of change does this PR introduce?

new function clean_up

convert to the right units using xscen.finalize.change_units
call the xscen.common.maybe_unstack function
add a february 29th by linear interpolation
remove a list of attributes
remove everything but a list of attributes
add attributes
change the prefix of the catalog attrs

Does this PR introduce a breaking change?
no
Other information:

We should add this step to the Notebook, I think?

We should add this step to the Notebook, I think?

yes, good idea. I will do that.

J'ai fait des changements en réponse au commentaire de @aulemahal sur gitlab. Le notebook a aussi été changé en conséquence. Que pensez-vous des nouveaux arguments?


Questions about HuBERT/Wav2Vec2 pre-training

🐛 Describe the bug
[This is not really a bug report, more a request for clarification/discussion...]
I'm training a HuBERT BASE-sized model, first iteration targeting 100 clusters of MFCC, on a custom 1000-hr dataset (i.e. similar size to Librispeech).  See the intriguing plots below:

Note that the triangles in the Tensorboard plots indicate there were NaN values.  I'm perplexed as to why that leads the loss curve and unit prediction accuracies have a "hiccup" around step 180K, but then seem to recover by step 200K.  My hypothesis is that it's related to the special treatment of feature penalty and layer normalization in the BASE-sized models.
As noted in the Wav2Vec2 paper:

For the smaller Librispeech dataset, we regularize the model by applying an L2 penalty to the activations of the final layer of the feature encoder and scale down the gradients for the encoder by a factor of 10.  We also use a slightly different encoder architecture where we do not use layer normalization, and instead of normalizing the raw waveform, the output of the first encoder layer is normalized.

(Inspecting the code, it's worth clarifying that the L2 "feature penalty" is in fact always included in the loss function, and it is scaled up by a hardcoded factor of 10x -- regardless of dataset or model size, and also irrespective of any masking -- although the 10x downscaling of the feature encoder gradients and the specialized layer normalization are only enabled in configurations for BASE-sized models.  So I think a perhaps more straightforward interpretation is that the feature penalty is effectively unscaled for BASE models, and 10x upscaled for LARGE/XLARGE models.  Am I reading that correctly?)
So my first question: what changes might be suggested to avoid the loss "hiccup" that I've observed?  Should I try adjusting the feature penalty scale and/or enabling standard layer normalization in the BASE model configuration?
My second concern is the drop in unmasked accuracy, which seems to start declining somewhat prior to the peak learning rate warmed up by step 20K.  I suspect this is because the implementation of the HuBERT loss function does not give any weight to the unmasked logits.  The HuBERT paper explored weightings of 0.0, 0.5, 1.0 and found that it was generally best to give zero weight to the unmasked loss component, especially when the targets are relatively low quality in terms of phonemic correlation.  However, I wonder: might it be worthwhile to consider some small but non-zero weighting, say 0.1, for the unmasked loss, to prevent the under-fitting dip seen in these plots?
I also wonder about the length normalization when combining masked and unmasked losses.  It seems that the current TorchAudio implementation will first combine these weighted masked and unmasked losses (which are summed from masked and unmasked logits of different lengths, depending on the masking parameterization), and add the feature penalty (averaged over all frames, irrespective of masking; this is later scaled by the length of masked logits) before later normalizing the overall summation of losses by the length of the masked logits.  By contrast, the fairseq implementation would normalize by the sum of lengths of the masked plus unmasked logits (i.e. the full sequence length) if the weight of the unmasked loss is non-zero.  Should the TorchAudio implementation be updated to match the fairseq implementation?
Moreover, I wonder: would it be a sensible improvement to instead normalize the masked and unmasked losses by their respective lengths prior to their weighted summation, and to also compute the feature penalty with respect to the weighting of the masked and unmasked losses (e.g unmasked feature frames should not contribute to the penalty if the unmasked weight is zero)?  The advantage of this is that the weighting becomes decoupled from the effect of the masking parameterization and thus it's easier to tune this hyperparameter independently.
Versions
I'm using a slightly modified local fork of the main branch.  The principal change is to refactor the training_step to (re-)enable automatic_optimization=True in Lightning (specifically for the gradient accumulation functionality, see #2918), rather than having a manual backward step.

Hi @arlofaria, thanks for sharing these questions, which I think are very inspiring.
To question 1, the advantage of using group normalization instead of layer normalization is the faster training speed, at the same time the gradient can be unstable, like what you observed in the loss curve. I think you can use layer normalization in Base model to stabilize the training. NOTE you need to also enable normalizing the waveform before feeding it to hubert model (see https://github.com/pytorch/audio/pull/2873)
To question 2: I actually tried with 0.001 weight on unmasked frames and it indeed help increase the unmasked accuracy. However, it doesn't seem to be helpful when fine-tuning the pre-trained model on ASR task. In both experiments (0 weight and 0.001 weight) I didn't meet the "hiccup" in the loss curve, but it's worth trying to see if it works in your case.
To question 3: This is true. Since the HuBERT paper mentioned that 0 weight for unmasked frames is optimal, I hardcoded the sample_size to be the number of frames in the masked frames. I can make it more flexible to let the loss function return two sample sizes and make the value to be 0 if the weight is 0.
To question 4: I think the feature penalty loss is to make feature extraction layers more sparse or avoid overflow, so both masked and unmasked frames can be helpful to such purpose. Regarding normalization of masked and unmasked losses by respective lengths, I think it's a good idea. Would you like to run another experiments to see if that is beneficial to the training? If so we can add it to the current recipe. Thanks!

Thanks for the helpful replies!

I'll try enabling full layer normalization for the BASE-sized model, hopefully improving training stability while not degrading training speed too much.  Thanks for the note about enabling waveform normalization as well, as that would have been quite easy for me to overlook.
My intuition is that an unmasked weight of 0.001 is rather close to zero, so I might start by trying 0.1 instead... but this value will have a somewhat different interpretation due to (4) below...
Sounds good!
I'll run this experiment and let you know how it goes!  However, note that this change to length-normalize the masked and unmasked losses before their weighted combination (instead of after) will modify the semantics of these weight hyper-parameters, so it won't be comparable to prior experiments that FAIR or you might have run.

And one more question, which seems like it may be related to layer normalization:

The fairseq Wav2Vec2 configurations disable conv_bias for BASE models, but enable it for LARGE; meanwhile, conv_bias is always disabled for all sizes of HuBERT models.  Is there some rationale for that difference?

Just following up on a few of these questions:
1. Regarding the training instability with NaN loss, and the "hiccup" is loss curve
I figured this out: it had nothing to do with layer normalization.  Instead, I found that using Lightning's Trainer(precision="bf16") instead of Trainer(precision=16) resolved the problem.  I'm not sure if this same problem would happen if I hadn't refactored the training step to enable Trainer(automatic_optimization=True) and use Lightning's half-precision AMP instead of the manual backward approach that is implemented on the main branch.
I don't have a clear explanation for the "hiccup" shape in the curves, but I think it might be an artifact of Lightning's logging: perhaps it appropriately zeros the gradients or skips the batches when encountering a NaN loss, but does not correctly update the batch size when logging the accumulated metric?
2. Regarding the drop in unmasked accuracy
Curiously, this does seem to be affected by layer normalization: replacing the group_norm with layer_norm (while also implementing waveform normalization) results in curves that monotonically increase the unmasked accuracy.  I'm not yet sure whether that results in improved downstream ASR accuracy.
4. Regarding the weighted combination of masked and unmasked losses
I've confirmed that using a non-zero weight for the unmasked loss is indeed detrimental for downstream ASR, particularly for BASE-sized models trained in iterations 1 and 2 with relatively poor targets.  However, I found that an unmasked weight of 0.1 (where the masked and unmasked losses are each length-normalized before combination) provided a very slight improvement (or perhaps statistically insignificant difference) for a LARGE-sized model at iteration 3.
5. Regarding the bias vectors for the CNN feature extractor
I found that having bias vectors was critical for stability when using Trainer(precision=16), but they are not necessary when using Trainer(precision="bf16").  This result, along with the first point above, suggests to me that there might be something sub-optimal about how Lightning implements AMP training with half-precision floats -- but it's kind of a non-issue if you have hardware that supports bfloat16. See also: https://github.com/Lightning-AI/lightning/pull/5359
And lastly another question:
6. Regarding examples/hubert and examples/ssl ...
... is there a plan to maintain both, or is the latter going to replace the former?
Also, is there a plan for newer SSL recipes to be added, e.g. Data2Vec2?

Yeah bf16 is more robust compared to normal 16-bit precision, although it is only supported on Ampere architecture GPU. My guess on why turning off bias in CNN feature extractor is that it may avoid the value overflow in the weights during optimization.
The plan for the examples/ssl recipe is to make it flexible for users to customize each components (loss function, data module, model architecture, etc). There will be new recipe added based on the ssl example. I will start with the Wav2Vec2 recipe with Conformer as the encoder. For Data2Vec I don't have bandwidth to implement it now, but would like to hear opinions from you and other recipe users. If there are enough interests on a specific SSL recipe we can put it in a planned work :)

I would certainly be very interested in a Data2Vec(2) recipe!
One of the major practical drawbacks of the HuBERT approach is the bottleneck of dumping of large hidden activations to disk storage before kmeans clustering; the multi-iteration training is also slightly inconvenient .  AFAICT, Data2Vec seems to avoid those problems by keeping everything in memory and doing a single training; it seems that Data2Vec2 incorporates some very significant speedups and incremental accuracy improvements also.  That's particularly helpful in comparison to HuBERT, where I've had to explore various shortcuts to scale experiments.
I think the main concern is whether the approach is still being refined; if so, it might be best to wait a bit.  For example, is there currently a Data2Vec3 being researched?
A Wav2Vec2 recipe would also be nice.  A specific use case to consider is the concept of further pre-training, where a well-trained large model is used as an initial checkpoint.  For HuBERT, this required a bit of hacking for me to find the original k-means model and also patch in a few missing components from Fairseq models that were missing from their repackaged bundles in TorchAudio.
However, I largely chose to explore HuBERT instead of Wav2Vec2 because of the availability of the recipe in TorchAudio, which seemed a bit easier to follow and perhaps more actively maintained than what's published in Fairseq.  Also, I have been a very big fan of its use of Lightning, and would encourage structuring future recipes to continue using that, ideally with the enabled automatic optimization.

I would certainly be very interested in a Data2Vec(2) recipe!

Data2vec 2.0 is indeed promising as it outperforms both HuBERT and WavLM. Let me check with them to see if there is new iteration.
I will look into the details when I get more bandwidth. Seems the key component is the teacher-student training and loss functions. The model architecture is pretty much the same as Wav2Vec2.


Maintain fork_targets

This PR fixes an issue with announced blocks being accumulated in fork_targets while doing a full sync. Each entry in fork_target is periodically checked with an expensive DB query, which slows down the full sync over time.
There are two changes:

We don't add fork targets until full sync is complete, as they won't be downloaded anyway.
When peer disconnects we make sure to cleanup fork targets that are not advertised by other peer.
This ensures that  fork_targets won't be building up over time and also prevents an attack that floods with invalid targets.

bot merge


Publish artifacts with source branch as the NPM tag

Fixes #492

By default, publish artifacts with source branch as the NPM tag
If provided, --npmtag takes precedence
Artifact version is no longer validated in the release definition. Instead, NPM will return error for non-existent version.

Codecov Report

Merging #513 (05ca07e) into develop (b95ce64) will increase coverage by 0.88%.
The diff coverage is 91.95%.

@@             Coverage Diff             @@
##           develop     #513      +/-   ##
===========================================
+ Coverage    78.59%   79.47%   +0.88%
===========================================
  Files           35       38       +3
  Lines         1210     1296      +86
  Branches       212      228      +16
===========================================
+ Hits           951     1030      +79
- Misses         259      266       +7

Impacted Files
Coverage Δ

...powerscripts-cli/src/impl/changelog/OrgsUpdater.ts
88.88% <88.88%> (ø)

...werscripts-cli/src/impl/changelog/CommitUpdater.ts
91.66% <91.66%> (ø)

...rscripts-cli/src/impl/changelog/WorkItemUpdater.ts
100.00% <100.00%> (ø)

...rscripts-cli/src/impl/release/ReleaseDefinition.ts
92.85% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09bb61f...05ca07e. Read the comment docs.


"Add providers" example doesn't match current ASP.NET Core Template

None of the configuration examples matches the code found in the current ASP.NET Core template used by Visual Studio.
public static class Program
{
    /// Creates the web host builder.
    /// <param name="args">The arguments.</param>
    /// <returns></returns>
    public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
        WebHost.CreateDefaultBuilder(args)
            .UseStartup<Startup>();

    /// Defines the entry point of the application.
    /// <param name="args">The arguments.</param>
    public static void Main(string[] args)
    {
        CreateWebHostBuilder(args)
            .Build().Run();
    }
}

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7184a308-a254-9350-a5cd-5bced1f369ae
Version Independent ID: 726e3bf1-f367-d733-8933-bccc04da0e16
Content: Logging in ASP.NET Core
Content Source: aspnetcore/fundamentals/logging/index.md
Product: aspnet-core
GitHub Login: @tdykstra
Microsoft Alias: tdykstra

@Grauenwolf ... It's somewhat common across the repo for a few reasons. For one thing, the sample apps trail the framework releases. Another is that samples concentrate on the concepts that the topics cover, which often don't strictly require the app to follow the template conventions. In this case (logging), the common thread is just hanging the ConfigureLogging method calls off of the host builder ... that's the same pattern regardless of the 2.x flavor.
I'll mark this issue on the sample update tracking issue. The doc author who takes on the 2.2 updates for this sample will see this at that time. Thanks for commenting on the topic.


fix build warnings [-Wunused-variable]

Signed-off-by: Remi Collet<EMAIL_ADDRESS>

Please add the hacktoberfest topic

@TysonAndre sorry, but can you add the  ‘hacktoberfest-accepted' topic...
From https://hacktoberfest.digitalocean.com/

This pull request was submitted to a repository that is not participating in Hacktoberfest. Add the ‘hacktoberfest’ topic to the repository to opt-in the whole repository, or the ‘hacktoberfest-accepted’ label to the PR to opt-in just the PR.

P.S. OK, this is a very small PR... btw, I probably open enough PR in the past in so much projects... and I love new Tshirt ;)


Add a "Featured In" page

We've been included in some news articles and stuff, let's showcase them somewhere on our website.
Maybe somewhere in our blog?

I'm down for it


feat: add network component types and update const…

…ructors

This change is

Codecov Report
Attention: Patch coverage is 0% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 51.37%. Comparing base (5496458) to head (a9a5952).

Files
Patch %
Lines

crates/gateway/src/gateway.rs
0.00%
3 Missing :warning:

crates/mempool/src/mempool.rs
0.00%
3 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #97      +/-   ##
==========================================
- Coverage   52.15%   51.37%   -0.79%
==========================================
  Files          13       13
  Lines         395      401       +6
  Branches      395      401       +6
==========================================
  Hits          206      206
- Misses        181      187       +6
  Partials        8        8

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

crates/mempool_types/src/mempool_types.rs line 30 at r7 (raw file):

    None,                                     // Input.
    AddTx(InternalTransaction, AccountState), // Output.
}

Is the type None really used somewhere? Is it needed?
Suggestion:
#[derive(Debug)]
pub enum GatewayMessage {
    AddTx(InternalTransaction, AccountState), // Output.
}

crates/gateway/src/gateway.rs line 45 at r7 (raw file):

            network,
        }
    }

All members of Gateway are public so there is no need for a constructor.
Code quote:
    pub fn new(
        config: GatewayConfig,
        stateless_transaction_validator_config: StatelessTransactionValidatorConfig,
        network: GatewayNetworkComponent,
    ) -> Self {
        Self {
            config,
            stateless_transaction_validator_config,
            network,
        }
    }


Run HTTPS by default

Given the trouble some browsers have with redirecting HTTP to HTTPS (Firefox, log in with custom WebID), maybe we should serve the local instance of the generated app using HTTPS?

I have updated the README to include a description of how to run in HTTPS mode. For the record, it's HTTPS=true npm start.


Panic (likely when something is odd with a frame)

Hi @balliegojr
Just encounter it accidentally.
I tend to believe it should return a Error but don't panic in any case?
Sadly I can't provide a reproducible.
thread 'main' panicked at 'range end index 12 out of range for slice of length 10', /home/maxim/.cargo/registry/src/github.com-1ecc6299db9ec823/simple-dns-0.5.0/src/dns/packet.rs:108:41

Take care.

Hi @zhiburt
Thanks for reporting this.

This commit should fix the issue


Lots of fake accounts has been created in my website.

Hi There,
Lots of fake accounts are getting generated on my website.
The one who is doing this maybe is bots because of below 2 reasons.

I disabled the registration page but still, fake registrations are showing up.
The time difference between Get and Post request of customer/registration page is 2-3 seconds which is not human behavior.

I tried using captcha V2 but it not stopping the fake registrations.
Note:- I don't want to use social media login or otp confirmation features.
Is there any way I can stop this registration?
Thanks in advance

@akshaypitale
could you please provide your website URL?

@akshaypitale,
Captcha V2 is fine. Maybe the problem is coming from the API. Did you disable the register route in API?

@akshaypitale,
Captcha V2 is fine. Maybe the problem is coming from the API. Did you disable the register route in API?

No.

@akshaypitale
could you please provide your website URL?

@akshaypitale
could you please provide your website URL?

https://naturaltilestone.co.uk/

@akshaypitale,
I am able to register my account via. API. You just need to disable the registered route via API. In API routes just comment the customer's register route.

@akshaypitale,
I am able to register my account via. API. You just need to disable the registered route via API. In API routes just comment the customer's register route.

Thanks for the reply.
But it can't be API because I checked the access logs where it shows at the time of fake registrations there is get request on customer/register and after 2-3 seconds there is post request on the same route.

@akshaypitale,
It's look like google captcha V2 has not been integrated properly. I am able to register the user without verifying the robot section.
Please check the integration as well.

@akshaypitale,
It's look like google captcha V2 has not been integrated properly. I am able to register the user without verifying the robot section.
Please check the integration as well.

@devansh-webkul thanks for the reply.
I have not integrated captcha V2. I will observe registrations a couple of days for a fake account.
Thanks again for the help.

@akshaypitale,
Well, this is not the problem with the code. To avoid spamming and bots google captcha are used but in your case captcha v2 has not been integrated properly, please check your integration.


[Bugs] QosRules in the PDU Session establishment Accept is wrong coded

Describe the bug
The default Qos Rule sent by the AMF in the payload container of the pDUSessionNAS-PDU of the PDU session establishment accept is wrong coded :
- The length value of the default QoS rule is set to 1585 bytes instead of the expected value 6 bytes.
- The packet filter component type is coded with an unspecified value (0 – Unknown) instead of one of the values specified in the 3gpp 24.501.
- The last byte of the Qos Rule which define the Qos Flow Identifier and segregration is missing.
- In consequence, the length values of the of QoS rules and Payload containers IEs are also wrong coded.
The current hexa stream of the pDUSessionNAS-PDU is set as follow (the wrong code in bold ) :
7e006801001d2e0a00c21100080106313101010009xx0601e80301e8032905013c3c0001120a
To Reproduce
Steps to reproduce the behavior:

Change config 'default Config'
Code patch 'no patch'
Run command './test.sh TestPDUSessionReleaseRequest or ./run.sh''
Dump packet 'Qos Rule in the pDUSessionNAS-PDU of the PDU session establishment accept'

Expected behavior
the AMF sends a QoS rule right coded in the pDUSessionNAS-PDU of the PDU session establishment accept as the following proposition (the wrong code in bold) :
7e006801001e2e0a00c21100090100063101010109060601e80301e8032905013c3c0001120a
Screenshots

QoS rule is set to 1585 bytes

The packet filter component type is coded with an unspecified value (0 – Unknown)

Environment (please complete the following information):

free5GC Version: [v3.0.2]
OS: [Ubuntu 18.04 Server]
Kernel version: [5.0.0-23-generic]
go version: [1.12.9 linux/amd64]

Trace File
Configuration File
config.zip
PCAP File
20200821_TestPDUSessionReleaseRequest_f5gc302.zip
Log File
20200821_TestPDUSessionReleaseRequest_f5gc302.txt

The issue seems to relate to the PR
https://github.com/free5gc/smf/pull/3
We'll check and merge it.

Sorry, seems that this is not the same issue as SMF PR#3.
How about using free5gc v3.0.3, will there still remind the same bug?

Hi,
The bug seems to be fixed in the free5gc V3.0.3

However, a fix in the v3.0.2  could be interesting for UE simulators which don't support yet the NAS security mechanisms (AMF doesn't check security mode #60)

We're sorry about that we don't maintain different branch(version) now due to the lack of our resource.
You can make the patch yourself and it might be easy due to free5GC is totally open source. 😃


fix: 🐛 pad tickers in Scope claims

:tada: This PR is included in version 9.0.0-alpha.2 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 9.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


hugepage usage with kubernetes and kata-container

Description of problem
I summarized some usage scenarios of hugepages
1. Configuration
The following two options have an impact on huge pages, where enable_hugepages is always configured as true, the default_memory always configured as 2048and sandbox_cgroup_only is divided into two cases: (1) true (2) false
sandbox_cgroup_only = true / false
enable_hugepages = true
default_memory = 2048

config hugepage on host
if the nr_hugepageshas changed,the kubeletshould restart.
[root@host-69 yaml]# sysctl vm.nr_hugepages=1024
vm.nr_hugepages = 1024
[root@host-69 yaml]# cat /proc/meminfo | grep -i huge
AnonHugePages:    280576 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
[root@host-69 yaml]# systemctl restart kubelet

2. Scenarios
2.1 pod does not configure huge page parameters and sandbox_cgroup_only=true
The yamlfile:
apiVersion: v1
kind: Pod
metadata:
  name: busybox-one
spec:
  runtimeClassName: kata-shimv1
  containers:
  - name: busybox-1
    image: busybox:latest
    stdin: true
    tty: true
    imagePullPolicy: IfNotPresent

Restart kubeletservice
systemctl restart kubelet

Start pod,and then i get the error:
Warning  FailedCreatePodSandBox  10s        kubelet, host-69   Failed create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: OCI runtime create failed: Failed to check if grpc server is working: rpc error: code = Unavailable desc = transport is closing: unknown

In this case, the usage scenario is incorrect. Because the huge page parameter is not configured in the yaml file, the limit at the pod level is 0. At this time, the qemu process is placed in a subdirectory under the pod level because sandbox_cgroup_only = true.Thus the qemu porcess will be restricted.More detailed analysis can refer to the following link:
https://github.com/kata-containers/runtime/issues/2172
2.2 pod does not configure huge page parameters and sandbox_cgroup_only = false
sandbox_cgroup_only = false

apiVersion: v1
kind: Pod
metadata:
  name: busybox-one
spec:
  runtimeClassName: kata-shimv1
  containers:
  - name: busybox-1
    image: busybox:latest
    stdin: true
    tty: true
    imagePullPolicy: IfNotPresent

Start pod:
[root@host-69 yaml]# systemctl restart kubelet
[root@host-69 yaml]# kubectl apply -f busybox-one_no_hugepage.yaml
root@host-69 yaml]# kubectl get pods
NAME          READY   STATUS    RESTARTS   AGE
busybox-one   1/1     Running   0          60s

At this point, the pod can be successfully started, because when sandbox_cgroup_only = false, although the pod level limit is still 0, the qemu process is not restricted because it is not placed in the control group under the pod.
2.3 pod configuration huge page parameters and sandbox_cgroup_only=true
2.3.1 pod started failed
podconfig:
apiVersion: v1
kind: Pod
metadata:
  name: busybox-one
spec:
  runtimeClassName: kata-shimv1
  containers:
  - name: busybox-1
    image: busybox:latest
    stdin: true
    tty: true
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - mountPath: /dev/hugepages
      name: hugepage
    resources:
      limits:
        hugepages-2Mi: 4096Mi
        memory: 4096Mi
      requests:
        memory: 4096Mi
  volumes:
  - name: hugepage
    emptyDir:
      medium: HugePages

modify host hugepage configuration:
[root@host-69 yaml]# sysctl vm.nr_hugepages=2048
vm.nr_hugepages = 2048
[root@host-69 yaml]#

Restart kubeletservice:
systemctl restart kubelet

Run pod,and then i get the error:
[root@host-69 yaml]# kubectl apply -f busybox-one.yaml
pod/busybox-one created
[root@host-69 yaml]# kubectl get pods
NAME          READY   STATUS              RESTARTS   AGE
busybox-one   0/1     RunContainerError   1          5s
[root@host-69 yaml]# kubectl describe pods busybox-one
 Warning  Failed     25s (x3 over 41s)  kubelet, host-69   Error: failed to create containerd task: OCI runtime create failed: QMP command failed: unable to map backing store for guest RAM: Cannot allocate memory: unknown

k8s+qemu = 4096Mi + 2048Mi = needHugePages
HostHugepages = 2048 * 2Mi

The above error is because there is only 4096M huge page memory on the host, but when kata calculates, 4096Mi in the pod and 2048Mi configured by kata itself are accumulated. When the virtual machine is started, the qemu process detects that the requested huge page memory exceeds Host configuration value, so an error is reported.Also,k8s does not know that kata itself has configured huge pages for virtual machines.
2.3.2 pod started successfully
podconfig:
apiVersion: v1
kind: Pod
metadata:
  name: busybox-one
spec:
  runtimeClassName: kata-shimv1
  containers:
  - name: busybox-1
    image: busybox:latest
    stdin: true
    tty: true
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - mountPath: /dev/hugepages
      name: hugepage
    resources:
      limits:
        hugepages-2Mi: 128Mi
        memory: 128Mi
      requests:
        memory: 128Mi
  volumes:
  - name: hugepage
    emptyDir:
      medium: HugePages

Host hugepage conig:
modify host hugepage configuration:
[root@host-69 yaml]# sysctl vm.nr_hugepages=1088
vm.nr_hugepages = 1088
[root@host-69 yaml]#

Restart kubeletservice:
systemctl restart kubelet

Run pod, pod started successfully:
root@host-69 yaml]# kubectl apply -f busybox-one.yaml
pod/busybox-one created
[root@host-69 yaml]# kubectl get pods
NAME          READY   STATUS    RESTARTS   AGE
busybox-one   1/1     Running   0          4s
[root@host-69 yaml]# cat /proc/meminfo | grep -i huge
AnonHugePages:    331776 kB
HugePages_Total:    1088
HugePages_Free:     1034
HugePages_Rsvd:     1034
HugePages_Surp:        0
Hugepagesize:       2048 kB

Although the above pods can be successfully launched, there are also hidden problems. Because the hugepages-2Mi: 128Mi value is written in the pod-level control group, the virtual machine is configured with a huge page of 2Gi; when the business in the virtual machine uses a large amount of memory, at the same time the qemu process is placed in the pod's child control group which will exceed the limit of the pod control group. This issue may cause the virtual machine to go down.And i guess,the error is same as 2.1
[root@host-69 pod5e2d430f-1ebe-4d80-a6fc-5ae52d897b6e]# pwd
/sys/fs/cgroup/hugetlb/kubepods/burstable/pod5e2d430f-1ebe-4d80-a6fc-5ae52d897b6e
[root@host-69 pod5e2d430f-1ebe-4d80-a6fc-5ae52d897b6e]# cat hugetlb.2MB.limit_in_bytes
134217728
[root@host-69 pod5e2d430f-1ebe-4d80-a6fc-5ae52d897b6e

[root@host-69 pod5e2d430f-1ebe-4d80-a6fc-5ae52d897b6e]# cd ./kata_7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b/
[root@host-69 kata_7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b]# cat cgroup.procs
43436
43438
43444
43458
[root@host-69 kata_7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b]# ps aux | grep 43436
root      4052  0.0  0.0  16340  1112 pts/2    S+   09:49   0:00 grep --color=auto 43436
root     43436  0.0  0.0 2759216 67008 ?       Sl   12月12   0:42 /usr/bin/qemu-vanilla-system-x86_64 -name sandbox-7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b -uuid c13cf4c5-21a3-45bf-ba0b-77dbdef716b4 -machine pc,accel=kvm,kernel_irqchip,nvdimm -cpu host -qmp unix:/run/vc/vm/7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b/qmp.sock,server,nowait -m 2048M,slots=10,maxmem=97558M -device pci-bridge,bus=pci.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2,romfile= -device virtio-serial-pci,disable-modern=false,id=serial0,romfile= -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b/console.sock,server,nowait -device nvdimm,id=nv0,memdev=mem0 -object memory-backend-file,id=mem0,mem-path=/usr/share/kata-containers/kata-containers-image_clearlinux_1.10.0-rc0_agent_8a4d901772.img,size=134217728 -device virtio-scsi-pci,id=scsi0,disable-modern=false,romfile= -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng,rng=rng0,romfile= -device virtserialport,chardev=charch0,id=channel0,name=agent.channel.0 -chardev socket,id=charch0,path=/run/vc/vm/7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b/kata.sock,server,nowait -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared,romfile= -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b,security_model=none -netdev tap,id=network-0,vhost=on,vhostfds=3,fds=4 -device driver=virtio-net-pci,netdev=network-0,mac=26:e5:a9:23:9e:9a,disable-modern=false,mq=on,vectors=4,romfile= -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic -daemonize -object memory-backend-file,id=dimm1,size=2048M,mem-path=/dev/hugepages -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinuz-<IP_ADDRESS>-3.1.container -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro ro rootfstype=ext4 quiet systemd.show_status=false panic=1 nr_cpus=8 agent.use_vsock=false systemd.unit=kata-containers.target systemd.mask=systemd-networkd.service systemd.mask=systemd-networkd.socket -pidfile /run/vc/vm/7f3be98b62b18f29acbe93aea60c04e0168d9edafce698ff4184bda8a0c1c38b/pid -smp 1,cores=1,threads=1,sockets=8,maxcpus=8
root     43438  0.0  0.0      0     0 ?        S    12月12   0:00 [vhost-43436]
root     43442  0.0  0.0      0     0 ?
Did you test pod overhead in scenario 2.3.1?

I make sum in https://github.com/kata-containers/runtime/issues/2353

@liangxianlong
When you finished to deploy your pod using kata-runtime,  you can checked host's hugepages are allocated to guest(kata).
But i can't find any hugepages when i try to check hugepages in pod.
I mean, i attached to pod and then execute below cmd line for check hugepages.
. cat /proc/meminfo | grep Huge
: HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
I expected to be able to check on the pod as many numbers as kata-runtime uses in the number of hugepages allocated on the host.
In your case, can you find any hugepages in pod?

What a comprehensive report about kata hugepage and k8s this issuses is !
Did you test pod overhead in scenario 2.3.1?

Pod Overhead won't work here, since it currently only accounts  for overhead associated with memory and cpu cgroups.
In 2.3.1, I expect things to be killed because we are using more the hugetlbfs than Kubelet expects us to (ie, more than is specified in the pod cgroup within the  hugetlbfs subsystem).  When SandboxCgroupOnly is set to false, the VMM isn't put into the pod cgroup -- that's why it succeeds.
Even with that succeeding, it is definitely doing the wrong thing to the node. Node thinks it has N huge pages available, but it actually has much less since our VMM isn't being accounted for.

What a comprehensive report about kata hugepage and k8s this issuses is !
Did you test pod overhead in scenario 2.3.1?

Pod Overhead won't work here, since it currently only accounts  for overhead associated with memory and cpu cgroups.
In 2.3.1, I expect things to be killed because we are using more the hugetlbfs than Kubelet expects us to (ie, more than is specified in the pod cgroup within the  hugetlbfs subsystem).  When SandboxCgroupOnly is set to false, the VMM isn't put into the pod cgroup -- that's why it succeeds.
Even with that succeeding, it is definitely doing the wrong thing to the node. Node thinks it has N huge pages available, but it actually has much less since our VMM isn't being accounted for.

I think we would have to make some unfortunate changes in Kubernetes in order to facilitate this working, or make an admission controller/webhook which will update the PodSpec if/when huge pages are used and a Kata runtimeclass is specified. With that, we'd need to:

Augment the huge page limit to include: emptyDir limit specified +  all memory limits + memory limit within the runtime class's PodOverhead field.
Find a way to communicate to the runtime (annotation?) what the limit should be for the ephemeral volume that is created in the guest.

@bergwolf @gnawux @sameo curious if ya'll have input here?

I think we would have to make some unfortunate changes in Kubernetes in order to facilitate this working, or make an admission controller/webhook which will update the PodSpec if/when huge pages are used and a Kata runtimeclass is specified. With that, we'd need to:

Augment the huge page limit to include: emptyDir limit specified +  all memory limits + memory limit within the runtime class's PodOverhead field.
Find a way to communicate to the runtime (annotation?) what the limit should be for the ephemeral volume that is created in the guest.

@bergwolf @gnawux @sameo curious if ya'll have input here?

@lifupan @bergwolf

@lifupan @bergwolf

I think we would have to make some unfortunate changes in Kubernetes in order to facilitate this working, or make an admission controller/webhook which will update the PodSpec if/when huge pages are used and a Kata runtimeclass is specified. With that, we'd need to:

Augment the huge page limit to include: emptyDir limit specified +  all memory limits + memory limit within the runtime class's PodOverhead field (or add a special/magic emptyDir with the overhead accounting)
Find a way to communicate to the runtime (annotation?) that the VM should be backed by huge pages and what the limit should be for the ephemeral volume that is created in the guest.

@bergwolf @gnawux @sameo curious if ya'll have input here?

Hi @egernst
I do agree on some change of k8s/controller/webhook to tell the PodSpec to add some HugePage overhead to hugetlb cgroup, but why we need to account all memory limits into the hugepage overhead?   Is it possible to split the guest memory into two parts/slots: the first one is normal page size memory with memory limits size; and the other one is the huge page with emptyDir limit + PodOverhead. Thus we can make the hugepage request as small as possible.
Yeah, we can use the annotation to pass the hugepage request to runtime.

I think we would have to make some unfortunate changes in Kubernetes in order to facilitate this working, or make an admission controller/webhook which will update the PodSpec if/when huge pages are used and a Kata runtimeclass is specified. With that, we'd need to:

Augment the huge page limit to include: emptyDir limit specified +  all memory limits + memory limit within the runtime class's PodOverhead field (or add a special/magic emptyDir with the overhead accounting)
Find a way to communicate to the runtime (annotation?) that the VM should be backed by huge pages and what the limit should be for the ephemeral volume that is created in the guest.

@bergwolf @gnawux @sameo curious if ya'll have input here?

Hi @egernst
I do agree on some change of k8s/controller/webhook to tell the PodSpec to add some HugePage overhead to hugetlb cgroup, but why we need to account all memory limits into the hugepage overhead?   Is it possible to split the guest memory into two parts/slots: the first one is normal page size memory with memory limits size; and the other one is the huge page with emptyDir limit + PodOverhead. Thus we can make the hugepage request as small as possible.
Yeah, we can use the annotation to pass the hugepage request to runtime.

@lifupan If we want to do such split, we need to pass give hugepages and normal pages to the guest. It is possible? I thought a guest can be using either hugepages or not at all. Also even if it's possible, we'd have to figure out how to hand out different memory region to different applications (guest kernel, tmpfs, agent, containers). I don't think it's simply do-able.
I think we can make all guest part of memory use hugepage and all host part of podoverhead (vmm + runtime) use normal size memory (that's the current situation IIUC.) The problem is that k8s hugetlb control group doesn't take kata's podoverhead into account. A webhook might just work by altering the container memory field but kata runtime should find a way to revert webhook's modification so that a container still uses its original claimed size of memory in the guest.
A more intrusive way is to make k8s aware of such detail by adding hugetlb overhead to podoverhead so that node and scheduler know how to properly handle it.
Just my 2 cents.

@lifupan If we want to do such split, we need to pass give hugepages and normal pages to the guest. It is possible? I thought a guest can be using either hugepages or not at all. Also even if it's possible, we'd have to figure out how to hand out different memory region to different applications (guest kernel, tmpfs, agent, containers). I don't think it's simply do-able.
I think we can make all guest part of memory use hugepage and all host part of podoverhead (vmm + runtime) use normal size memory (that's the current situation IIUC.) The problem is that k8s hugetlb control group doesn't take kata's podoverhead into account. A webhook might just work by altering the container memory field but kata runtime should find a way to revert webhook's modification so that a container still uses its original claimed size of memory in the guest.
A more intrusive way is to make k8s aware of such detail by adding hugetlb overhead to podoverhead so that node and scheduler know how to properly handle it.
Just my 2 cents.

@lifupan If we want to do such split, we need to pass give hugepages and normal pages to the guest. It is possible? I thought a guest can be using either hugepages or not at all. Also even if it's possible, we'd have to figure out how to hand out different memory region to different applications (guest kernel, tmpfs, agent, containers). I don't think it's simply do-able.
I think we can make all guest part of memory use hugepage and all host part of podoverhead (vmm + runtime) use normal size memory (that's the current situation IIUC.) The problem is that k8s hugetlb control group doesn't take kata's podoverhead into account. A webhook might just work by altering the container memory field but kata runtime should find a way to revert webhook's modification so that a container still uses its original claimed size of memory in the guest.
A more intrusive way is to make k8s aware of such detail by adding hugetlb overhead to podoverhead so that node and scheduler know how to properly handle it.
Just my 2 cents.

@lifupan If we want to do such split, we need to pass give hugepages and normal pages to the guest. It is possible? I thought a guest can be using either hugepages or not at all. Also even if it's possible, we'd have to figure out how to hand out different memory region to different applications (guest kernel, tmpfs, agent, containers). I don't think it's simply do-able.
I think we can make all guest part of memory use hugepage and all host part of podoverhead (vmm + runtime) use normal size memory (that's the current situation IIUC.) The problem is that k8s hugetlb control group doesn't take kata's podoverhead into account. A webhook might just work by altering the container memory field but kata runtime should find a way to revert webhook's modification so that a container still uses its original claimed size of memory in the guest.
A more intrusive way is to make k8s aware of such detail by adding hugetlb overhead to podoverhead so that node and scheduler know how to properly handle it.
Just my 2 cents.

I think we would have to make some unfortunate changes in Kubernetes in order to facilitate this working, or make an admission controller/webhook which will update the PodSpec if/when huge pages are used and a Kata runtimeclass is specified. With that, we'd need to:
1. Augment the huge page limit to include: emptyDir limit specified +  all memory limits + memory limit within the runtime class's PodOverhead field (or add a special/magic emptyDir with the overhead accounting)

2. Find a way to communicate to the runtime (annotation?) that the VM should be backed by huge pages and what the limit should be for the ephemeral volume that is created in the guest.

@bergwolf @gnawux @sameo curious if ya'll have input here?

@egernst I'm trying to understand your above comments and putting some questions here. Please be patient as I try to deconstruct and understand :-)
Usecase-1: Trying to deploy an existing pod.yaml with hugepage req/limit using Kata. For this to functionally work the Kata VM needs to have same number and type of hugepages that are requested by the pod. The Kata VM memory itself might not be backed up by hugepages
Usecase-2: Trying to deploy an existing pod.yaml with hugepage req/limit using Kata. For this to functionally work the Kata VM needs to have same number and type of hugepages that are requested by the pod. Also for optimal performance the Kata VM memory needs to be backed up by hugepages.
My understanding is your comments refer to Usecase-2.  Please correct me if wrong ?

No problem -- want to make sure you understand and we're on the same page!
Yeah, I'm talking about use-case #2. I was under the assumption that you'd want to be be backed by huge pages on the host. This is what Kubernetes is accounting for when creating your huge page volume, right?  There isn't a way to "provide this" to the application.

No problem -- want to make sure you understand and we're on the same page!
Yeah, I'm talking about use-case #2. I was under the assumption that you'd want to be be backed by huge pages on the host.
Thanks @egernst. Yeah ideally I would want VM memory to be backed up hugepages to actually reap the benefit of hugepages. However my primary goal for the runtime and agent changes was to ensure a regular pod.yaml works and doesn't break when running with Kata (from user experience stand point).  And we document the limitations.

This is what Kubernetes is accounting for when creating your huge page volume, right? There isn't a way to "provide this" to the application.
Right, In Kata case, the hugepage is actually not (cannot be) consumed.  We'll need to look at some of the options that you had already mentioned - like introducing Pod overhead for hugepages so that the VM memory gets accounted for in hugepage limits etc. Also things like how to handle different hugepage sizes since afaik VM can only be backed by a specific hugepage size.
This will be a long lead item to figure out a workable approach given these complexities.

@egernst @bpradipt do I understand correctly that both of you got itnto a consensus? :-)
If so, I'd like proceed with https://github.com/kata-containers/runtime/pull/3109 and https://github.com/kata-containers/agent/pull/872

I'm going ahead and proceeding with #3109 and kata-containers/agent#872


documentation: how it works

It would be great to have something like a design document: what problems does this library solve, how does it solve them, how does it avoid various pitfalls, etc.
I came across this reply from @elBoberido on HN and it's emblematic of what's lacking; the parent talks about shared memory challenges in the presence of process death and the response simply states

We found a few ways on how to make this feasible, though.

Another point of confusion for me: how does this work without a central broker? According to https://iceoryx.io/ there is no need for one.
Anyway, something that describes the design of the software would go a long way in increasing (at least my) confidence in it.

@tamird I appreciate and completely get it.
Just to give you some context, we have all been working together for the last seven years. Starting with a Bosch research project, the predecessor of iceoryx, then we fought for over a year to make it open source. In the last years, we have certified parts of iceoryx1 according to ISO 26262 and realized that some things make the zero-copy communication hard to certify - one key element was the central broker.
So we started redesigning iceoryx again and came up with the iceoryx2 architecture. It just took us five years to realize that everyone has a central broker - it is the operating system.
At the moment we have the goal to reach feature parity with iceoryx and complete the user documentation with a good getting started guide. When this is done, we will present the iceoryx2 design in detail. Maybe it would be a great topic for a FOSDEM 2025 talk but I think we need a bit more than just one hour.
Just out of curiosity, what topic would you be most interested in:

Architecture without a central daemon
Decentralized process monitoring and resource cleanup
Decentralized communication and service discovery
Zero trust approach for shared memory communication
Indestructible processes
Incorruptible data structures

All of these would be interesting topics. I would probably start with the first 3 which sound least fantastical 🙂 .


Consider adding axum-seaorm-todo to SeaORM showcase!

Hey @Taillook, feel free to submit a PR to showcase your project!

https://github.com/SeaQL/sea-orm/issues/403

First of all, Should I write README.

Yeah, why not loll :)


Fix memory leaks in MAUI views

Description of Change
It is very easy to create a circular reference in iOS and we had a few. When you have an event from a platform view to another instance that then references that platform view, you get an un-collectable set.
This PR breaks this chain by making the platform view have an event on a proxy object that has a weak reference to handlers and/or virtual views.
Bugs Fixed

Fixes #2923

API Changes
None.

Behavioral Changes
None.

Required skia PR
None.

PR Checklist

[ ] Has tests (if omitted, state reason in description)
[ ] Rebased on top of main at time of PR
[ ] Merged related skia PRs
[ ] Changes adhere to coding standard
[ ] Updated documentation

/rebase

/rebase

@mattleibow this one is only on 3.0.0 right? 2.88 is not possible?

Preview 4 is out: https://www.nuget.org/packages/SkiaSharp/3.0.0-preview.4.1


Make workflow steps optional and allow them to be ordered

As a UX designer I may want to change the workflow of the feedback mechanism to fit my particular context.
The current workflow is modeled off of Google's feedback mechanism and is somewhat static. For example in the context of Google+ it may be very useful to black out sensitive information. If this were implemented for site like www.NewYorker.com however, there might be no need to block out sections of the DOM. These steps should be modular, optional and have the ability to be ordered.

Hi, great work greatly appreciated.
But now is there a way to skip the blackout/highlight step and take a screenshot of all page only?
The code you refer is not working anymore:
Feedback({
    blackout: false; // defaults to true
});


Fix build instructions for Go 1.18+

Reference: https://golang.org/doc/go-get-install-deprecation
I’m unable to run the REPL by following the build instructions in the README.md, but using go install instead of go get works for me:
$ podman run --pull always -it --rm docker.io/library/golang:latest
root@445c025009b7:/go# go version
go version go1.18.4 linux/amd64
root@445c025009b7:/go# starlark
bash: starlark: command not found
root@445c025009b7:/go# go get -u go.starlark.net/cmd/starlark
go: go.mod file not found in current directory or any parent directory.
	'go get' is no longer supported outside a module.
	To build and install a command, use 'go install' with a version,
	like 'go install example.com/cmd@latest'
	For more information, see https://golang.org/doc/go-get-install-deprecation
	or run 'go help get' or 'go help install'.
root@445c025009b7:/go# go install go.starlark.net/cmd/starlark@latest
go: downloading go.starlark.net v0.0.0-20220714194419-4cadf0a12139
go: downloading github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e
go: downloading golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f
root@445c025009b7:/go# starlark
Welcome to Starlark (go.starlark.net)
>>> ^D

Thank you for the merge!


Dockerfile.contrib

Dockerfile to build from local node-zwave-js git repo.

@larstobi Is this actually working?
Did you checked https://zwave-js.github.io/node-zwave-js/#/development/installing-from-github ?

It works on my machine, at least. I haven't tried on anybody elses. Yes, I used that guide from your link as a starting point.

@larstobi I think that you copuld merge step 1 and 2 into one

Also I'm not sure that the links will work in that way

First, npm link in a package folder will create a symlink in the global folder {prefix}/lib/node_modules/ that links to the package where the npm link command was executed. It will also link any bins in the package to {prefix}/bin/{name}. Note that npm link uses the global prefix (see npm prefix -g for its value). Next, in some other location, npm link package-name will create a symbolic link from globally-installed package-name to node_modules/ of the current folder.

So essentially the link will be lost when building the last container (as the sym link is added globally). I suggest do to everything in only one build stage

@larstobi I think that you copuld merge step 1 and 2 into one

Not sure what you mean by step 1 and 2. Do you mean:
RUN npm install && npx lerna init

I have corrected my comment.
I mean you should not build them in separete build steps but keep only one, firstly build zwavejs then z2m and after building it just run the link command and you don't need to override package.json, the link will override the dep automatically in node modules.
Also having a one unique step will fix the link problem I mentioned before :)

I'm not sure i really follow for the need for this, but can we add to the ci build, even if we don't push somewhere so that we know when we break it?

I'm not sure i really follow for the need for this, but can we add to the ci build, even if we don't push somewhere so that we know when we break it?

@chrisns I don't get it, in order this to be broken it means z2m or zwave-js masters are broken someway. I see this a developer tool for the ones that are looking for testing latest builds of both projects using master or specific branches

I'm not sure i really follow for the need for this, but can we add to the ci build, even if we don't push somewhere so that we know when we break it?

@chrisns I don't get it, in order this to be broken it means z2m or zwave-js masters are broken someway. I see this as a developer tool for the ones that are looking for testing latest builds of both projects using master or specific branches (with docker, this is lot easier using nodejs but I see there are lot users that still prefer docker)

agree its a dev tool, I don't see the real usecase of why you'd do it /this/ way. if it does need to exist I feel like it could easily fall out of sync however, so would recommend exercising the use case in a CI job which would double up as documentation on how to use it 😄

I don't see the real usecase of why you'd do it /this/ way

IMO docker isn't really usefull at all now that there isn't OZW lib to compile

Well you do have a confined environment where the libraries are used with a fixed Node.js version without influences from the outside world. I can see the benefit in that.

Well you do have a confined environment where the libraries are used with a fixed Node.js version without influences from the outside world. I can see the benefit in that.

Yes that actually is good for every project that you run with docker, it's docker main feature. I mean that if you can use nodejs in your machine it would be much easier and faster, expecially for testing. But as said I understand that someone could still want docker so I'm ok to also provide it as it is already managed :)

if it were me, i'd just docker run -v ${PWD}:/app zwave-js/zwavejs2mqtt bash and then run everything in the container starting with npm i.
It's a different use case since you'll want dev dependencies etc in there so will be different. but do agree with having a representative version in the same arch, i guess you can probably port forward through to attach to a debugger etc

i guess you can probably port forward through to attach to a debugger etc

Yep you can do remote debugging

but your editor+git client is still (presumably) gonna be in the host machine, so wouldn't you want to volume mount the workspace?

but your editor+git client is still (presumably) gonna be in the host machine, so wouldn't you want to volume mount the workspace?

:man_shrugging:

🤷♂

@larstobi can you explain a bit more about how you're using this and why it's better/different/preferred than mounting in the workspace with docker run -v ?
Thanks

Building a self-contained containers instead of running just a node container with mounted source code is a good way of getting immutability for the service. It's the way I prefer to run my production containers. It's also distributable, I can store it in a docker registry, and easily run the exact same container on a different server.
Also, it's a really easy way for new users to get going. Just point them to the container on Docker Hub, and run it. No need to check out the source repo and build it.
There are many benefits to it as I see it, but those mentioned are some. :-)

I've made some changes to the Dockerfile based on input from you. It builds successfully on my machine. I updated it to use the newer npm version along with the new build instructions for zwave-js. I wasn't easily able to get it to work with Alpine Linux, and it doesn't have the more recent node and npm.
I'm getting some new runtime errors from missing dependencies after the updates, though. For instance triple-beam and reflect-metadata.  Will try to work it out tomorrow.

I'm getting some new runtime errors from missing dependencies after the updates, though. For instance triple-beam and reflect-metadata. Will try to work it out tomorrow.

Weird... reflect-metadata is a dependency of zwave-js, triple-beam should come with winston.

Building a self-contained container instead of running just a node container with mounted source code is a good way of getting immutability for the service. It's the way I prefer to run my production containers. It's also distributable, I can store it in a docker registry, and easily run the exact same container on a different server.
Also, it's a really easy way for new users to get going. Just point them to the container on Docker Hub, and run it. No need to check out the source repo and build it.
There are many benefits to it as I see it, but those mentioned are some. :-)

I'm with you a million percent on the "why containerize" I just don't really follow to what distinguishes this from the existing, and why we need both https://github.com/zwave-js/zwavejs2mqtt/blob/master/docker/Dockerfile or can we remove one?

@chrisns The main difference is that this actually allows to use a custom version of zwavejs. We could optionally add an arg on the other one to support this?

Yeah. That's kinda what I was leaning towards, or a super slim one that inherits from the base.
But given the pace of change of the zwavejs I'm not sure if there's even a strong case for that.
So it's not for sharing and only as a dev tool, again fine but I still don't follow the real 'why'. It feels like an anti pattern or I've just misunderstood the original intention

It feels like an anti pattern or I've just misunderstood the original intention

I'm unable to see how this is an anti-pattern. Could you explain?

It feels like an anti pattern or I've just misunderstood the original intention

I'm unable to see how this is an anti-pattern. Could you explain?

heya
the reasons you mentioned in here https://github.com/zwave-js/zwavejs2mqtt/pull/28#issuecomment-733970144 don't seem to stand up because this surely(?) isn't for running something that is for production or sharing, by definition its a dev enabler that there doesn't appear to be an appetite to maintain by even adding a CI job/step to test or push the image anywhere.
I'm probably totally missing something but I don't really grasp who this is for and why it is needed.
Sorry I don't mean to come across like a ... and it's more likely me than you thats wrong, I just don't get it - I suspect others will be the same. So would be great if you could you could really spell it out :)
I'm on the z2m slack if it's easier to chat about it there rather than bat github comments back and forth?

FYI I'm ok for this to land, personally I would never use such thing but if could help some users with testing I'm totally :+1:

This last version builds and runs on my machine.

@larstobi Could you delete that Readme and add a section to the already existing readme in docker folder with the steps needed to build?  Like:
mkdir -p testing && cd testing
git clone https://github.com/zwave-js/node-zwave-js
git clone https://github.com/zwave-js/zwavejs2mqtt

## Checkout repos to any branch/commit you need to test

cp zwavejs2mqtt/docker/Dockerfile.contrib ./Dockerfile
docker build -t zwavejs2mqtt .

@larstobi Sorry I just saw that lint fails, could you run npm run lint-fix ?

would be great to build with actions and maybe push somewhere

There is a Docker repo here https://hub.docker.com/r/zwavejs/zwavejs2mqtt
This can be used to set up Docker Hub Automated Builds with some modifications.

would be great to build with actions and maybe push somewhere

Pull Request Test Coverage Report for Build 386844662

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 28.335%

Totals

Change from base Build 386823720:
0.0%

Covered Lines:
1725

Relevant Lines:
6256

💛  - Coveralls

would be great to build with actions and maybe push somewhere

@chrisns Where should be pushed? Why? This in inteded to be used just locally

earlier comments suggested it was useful for sharing, eitherway, it needs to be in CI if we're going to be able to maintain it. even if its just a amd64 build that doesn't get pushed

@larstobi Are you done here?


Tags Do Not Work Correctly in karate-gatling:1.1.0 And Higher

In karate-gatling version 1.1.0 and higher, karateFeature("classpath:mock/cats-delete.feature@name=delete") works correctly, but karateFeature("classpath:mock/cats-delete.feature", "@name=delete") does not.  It appears that the tags  var-args parameter is not being honored.
Symptoms
Given: karateFeature("classpath:mock/cats-delete.feature", "@name=delete")
Expected:  Only the scenarios that match @name=delete in cats-delete.feature are executed
Actual:  All scenarios in cats-delete.feature are executed
Testing
The easiest way to repeat this bug is to modify line 27 of CatsSimulation.scala
val delete = scenario("delete").group("delete cats") {
   exec(karateFeature("classpath:mock/cats-delete.feature", "@name=delete"))
}

Run the tests for karate-gatling and inspect the output for the following line.  This confirms that all scenarios were executed, even those without @name=delete.
GatlingSystem-akka.actor.default-dispatcher-94] INFO  com.intuit.karate - [print] this should not appear in the logs !

Fix
I think I have an idea of what's causing this, so I can investigate and submit a PR.

I have confirmed that this is fixed in 1.2.0.RC2.  However, I found two new bugs in the process.  I'll open new issues as soon as I can assemble the minimal examples to reproduce them.

@packleader thanks - if possible, can you also see if migration to gatling latest is straightforward, in which case we can do that too: https://gatling.io/docs/gatling/reference/current/upgrading/3.6-to-3.7/

I reviewed the release notes for Gatling 3.7, and I don't think any of the changes will impact us.  Specifically, we don't seem to be using any of the newly-deprecated methods.
I upgraded to 3.7.3 in the karate-gatling pom, and the build still passes. I'll do some more thorough testing, but so far it looks good.

@packleader I upgraded to 3.7.3 and the gatling maven plugin to 4.0.1. as you said it all looked ok.
but couldn't get the examples/gatling project to run via maven at all. which is very strange, tried a lot and gave up for now. that 4.0.1 gatling maven plugin simply cannot find the simulation files

1.2.0 released


[FEATURE] Hover effect for get results background colour change

Description
hey

Can we add hover effect to get results button, background colour changer when user hover it .
I am working on this feature.
can you assign me this.

Screenshots
https://user-images.githubusercontent.com/97652040/193565261-17880a83-0cf1-45ea-b252-3af3e9dcd74c.mp4

It's great having you contribute to this project by creating an issue Make sure you are assigned to this before you work on it and you read the Contributing.md file, Thank you! Welcome to the community :nerd_face:

Definitely, thanks for wanting to contribute

:tada: This issue has been resolved in version 2.4.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Peak shaving and rotor speed

Hi all,
I am tuning my controller using ROSCO toolbox, and when I plot my operational points coming from steady wind simulations statistics, I see that when peak shaving applied (less than 20% load reduction) the TSR is changing (rotor speed slope lower than what should be) from where peak shaving is used, which is not desired. I reduced and increased my rated wind speed in the yaml file, but it did not change the controller behavior at operational points where there is peak shaving.
Is there a way to ensure that before reaching rated rotor speed the TSR is kept constant, so that peak shaving is applied only by changing the pitch angle? Cf picture

Thanks
Best regards

Hi, this looks like an issue with the ROSCO controller that we addressed here.
What version of the ROSCO controller are you using and what model/DISCON.IN file are you using?

Ok, why is this difference in rotor speed between Peak shaving / not Peak shaving occuring for some simulations only in the pdf from your link?
I am using the ROSCO 2.2.0 (libdiscon.dll), and I am using the DISCON.IN file created from the ROSCO toolbox (installed from the ROSCO 2.4.0) using the parameters I would like to have for my controller. The wind turbine is close to the IEA 15MW UMaine, but I am simulating it as onshore WT and with Orcaflex (which can read the .dll).

This should fix the min rpm problem https://github.com/NREL/ROSCO_toolbox/pull/64/commits/d14964772831f26e961386775743140a5a257970

Thanks for the update.


Dropout for CPU

@Krovatkin Just two last things:

I think you still need to fix the mode change on hybrid/unit_test.manifest. (If I'm wrong just tell me, GitHub's UI is confusing to me here.)

Please delete state/state.cpp, or add a copyright header (even though the file is otherwise empty) so we don't forget it if code is added to the file later.


UR robot dual arm manipulation

Hope to realize dual arm manipulation by two UR 10 robot with ur_modern_drive. Each UR arm has a different task but coordinates with each other, rather than master-slave. Already refer to https://answers.ros.org/question/269074/dual-arm-ur5-connection-with-universal-robot-stack/
Look forward to a generic method. Thanks.

There isn't anything really special about ur_modern_driver, and the same approach can be taken as with the old ur_driver: run two instances in two different namespaces and make sure to configure the reverse_port parameter with a unique value for each instance (it is the TCP port on which the driver will communicate with the controller, so two instances of the driver cannot share the same port).
At this point you'd have two copies of all topics, services and action servers.
If you have nodes that would benefit from a "global JointState" message, use joint_state_publisher together with the source_list parameter to arrange that.

I'm going to close this as it's a question about configuration of the node, not an actual issue.
You are free to keep commenting of course.

Someone suggests using the 'Moveit! Setup Assistant' to configure the other UR10 to avoid the namespace issue, as if they are a PR2 and alike. If you disagree, could you please teach me a systematic method to run two UR10 independently on one PC? Thank you. Our goal is to use one UR10 as a turntable, while the other UR10 operates on it, scanning, picking and etc.
Thank you. I look forward to your help.

Someone suggests using the 'Moveit! Setup Assistant' to configure the other UR10 to avoid the namespace issue, as if they are a PR2 and alike.

I don't know what this means.

A software engineer suggests using the Moveit! Setup Assistant http://docs.ros.org/hydro/api/moveit_setup_assistant/html/doc/tutorial.html
to configure two ur10 into a dual arm system, from scratch.

That might be a good idea in any case, especially if you want to plan for both arms in a single scene.
That is almost orthogonal of how you configure the driver though.

from scratch.

If "from scratch" means: following the wizard to create a new MoveIt configuration pkg, then I would recommend to always do that.
The MoveIt pkgs we provide here are almost just examples. It's very rare that you would have your robot in isolation in free space, so a rerun of the MSA is almost always needed.

Btw: I'd recommend to use the latest version of the MSA documentation / tutorials: here.
The Hydro version you link is rather antiquated by now.

Thank you and happy weekend.
I am using Kinect. Sorry for the casual google results.
I will appreciate it if you can teach me your method for dual arm. I tried to modify the mentioned files in my first post. But it broke the ur_modern_drive. I cannot use Moveit! however purging and reinstalling the package. I reinstalled my laptop. Could you please teach me a systematic work flow to make two UR10 cooperate, in your understanding. Thank you very much. I can try to carry it out step by step.
PS: Ubuntu 16 now uses the kernel 4.15. The reinstallation is the cause of failing to build the realsense2 package.


add __builtins__ to globals in py.run() if they're missing

Python code doesn't like to run without __builtins__, so adding them if missing seems to be a good idea. Also that's what CPython >3.10 does.
See https://github.com/python/cpython/pull/24564
Resolves #3370

[x] Agreed to dual-license (https://github.com/PyO3/pyo3/pull/3108#issuecomment-1674518411)
[x] Entry in newsfragments
[x] Tests for changed functions

Many thanks, implementation looks great. My only thought: after evaluation should we attempt to remove builtins from the globals dictionary if we added it?

I think that’s more confusing than the current option (especially given possibility of code like __builtins__ = new_value). CPython uses some private APIs to specify __builtins__ for the new frame explicitly instead of adding them to globals, so we can’t easily mimic its behaviour here (in particular, we’d need _PyFunction_FromConstructor: I don’t see a way to manually specify __builtins__ otherwise).

Fixed the docs.


Completed the Weather App / Project

Related Issue
Fixes #147
Description
WeatherLy App Deployed
Type of PR

[ ] Bug fix
[x] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)

Checklist:

[x] I have performed a self-review of my code
[x] I have read and followed the Contribution Guidelines.
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[x] I have commented my code, particularly in hard-to-understand areas.

Additional context:
@SUGAM-ARORA , WeatherLy app is completed , Kindly review

@SUGAM-ARORA kindly review this PR, and if there are any changes to be made do let me know.


First released version 1.0.7 cannot be installed anymore

Bug Description
After deleting newest testnet app v1.2.3(66), first released version 1.0.7 cannot be installed anymore. It fails with message App not installed, as if you'd try to downgrade from a newer version to an older version. Do we not delete all data?
Or can the old version not be installed by different reasons?
Steps to Reproduce

install v1.2.3(66)
delete it
install v1.0.7

Expected Result
Old version can be installed.
Actual Result
Old version cannot be installed; App not installed
Versions

see above
Android 9
Huawei Y6s

@concordium-cl I see this from time to time. It hasn't been related to the actual build, but a hiccup in the OS itself. Restarting the phone may help, or try install the new version and uninstall it again may help. Using ADB (Android Debug Bridge - a command line tool that comes with eg. Android Studio) to uninstall may also be an option, as sometimes it isn't uninstalled although up cannot see the app. Uninstalling and installing through ADB may reveal more info as to why it cannot install/uninstall on the particular phone.

Restarting phone and installing newest version did not work. Used v1.0.8 for testing now, which was released internally very soon after 1.0.7.


[runtime/fix] fix locale wstring conversion error

Fix issue https://github.com/wenet-e2e/wenet/issues/745


0.23

Preparing to release 0.23. Here are the changes:

Pull 11 Added Observable support for Item Events
Pull 15 Added support for property change events
Pull 17 Removed SwingScheduler's invokeLater when called from EDT
Pull 18 Automatically subscribe/unsubscribe on the EDT
Pull 21 Adding unsubscribeOn EDT calls to all event sources
Pull 22 Added DocumentEvent support

I released it, but accidentally released as 0.24 :-/

No problem, thanks Ben.

Edited this release to be version 0.24


FastAPI

Integrate FastAPI usage sample to template. The example should implement base functionality. You should submit setup docs to USAGE_GUIDE.md.
Teleplate with FastAPI integration must develop in fastapi branch, apart from the main branch.

Hey @devkarych  , I'm interested in contributing to this issue, so before I start working on it, would you mind sparing your time explaining what the issue is about and pointing me to some resources to get started .


tag_excluding not managed

Describe the bug
When setting retention rules to tag_excluding, rule is created but without decoration.
Expected behavior
Add tag_excluding management
Additional context
In client/retention.go, tag_matching is tested but not tag_excluding

Add PR #111


Feature request: sphinx

Hi,  this is a feature request to support the sphinx docstring style.
Thanks

I procrastinated a bit but this day has eventually come 😂
Yes, this is something that I could add.
Could you confirm that this is the official guide/rules of the Sphinx docstring style?  https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html
Thanks!

Thanks!
pydoclint uses docstring-parser to parse Google-style docstrings, and I know it can also parse Sphinx-style docstrings:
https://github.com/jsh9/pydoclint/blob/062bb451c3b22cd8ce460417d88f1690c9710037/setup.cfg#L21
The reason I'm asking for a style guide is for me to write test cases.  I haven't really used the Sphinx style before.

Hi @wwuck:
I added support for the Sphinx docstring style.  It's in the latest release (v0.1.0).
Please feel free to try it out.  Since I'm not very familiar with this style, I don't know what different variations of this style can there be.
If you find any issues or false positives, please feel free to open other issues.  Thanks!