chore: fix pre-commit script to work on Linux and macOS

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Pre-commit script fix

What is the current behavior? (You can also link to an open issue here)
Failing on Linux and macOS.

What is the new behavior (if this is a feature change)?
Use $OSTYPE to detect the OS.
Force meteor to run in x86 on ARM Macs. (to be removed when upgrading to meteor that supports arm64)

Other information:
Tested only on Windows and macOS, not tested on Linux.

Status

[ ] Code documentation for the relevant parts in the code have been added/updated by the PR author
[ ] The functionality has been tested by the PR author
[ ] The functionality has been tested by NRK

Codecov Report

Merging #628 (01620f1) into release38 (1e75825) will increase coverage by 15.18%.
The diff coverage is n/a.

:exclamation: Current head 01620f1 differs from pull request most recent head 3ca9752. Consider uploading reports for the commit 3ca9752 to get more accurate results

@@              Coverage Diff               @@
##           release38     #628       +/-   ##
==============================================
+ Coverage      67.28%   82.46%   +15.18%
==============================================
  Files            268       12      -256
  Lines          18669     1015    -17654
  Branches        3944      198     -3746
==============================================
- Hits           12561      837    -11724
+ Misses          6065      177     -5888
+ Partials          43        1       -42

Impacted Files
Coverage Δ

meteor/lib/collections/Timeline.ts

...teor/server/api/ingest/mosDevice/mosIntegration.ts

meteor/server/api/peripheralDevice.ts

meteor/server/api/playout/playout.ts

meteor/server/api/playout/timeline.ts

meteor/server/migration/1_12_0.ts

meteor/server/migration/1_3_0.ts

meteor/server/migration/1_14_0.ts

meteor/server/cache/lib.ts

meteor/server/security/organization.ts

... and 246 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e75825...3ca9752. Read the comment docs.

To the reviewers: plz comment below with which platforms you have tested this on

[ ]  Windows
[ ] MacOS
[ ] Linux

Not working on linux

TIL that [[ is a bash thing, not a POSIX standard. And bash is not the default shell on Ubuntu. I chose to stick to the default shell and use a different method of comparing strings.

It works on linux now


fix, chore, perf, style, feat, improv, refactor, test, docs: update dispatcher.h/.cpp

This PR allows events to be move and copy assigned.
That's it. Definitely nothing else in 800 line changes.
Just kidding :

Fixes style issues in dispatcher.h and dispatcher.cpp
Adds unit tests checking that the types can be copy constructed and assigned as well as move constructed and assigned
Changes the thread_local stop_event for event cancellation into a mutable bool inside of event_dispatch_t (yes, it still works)
Adds various small optimizations in dispatcher.cpp
Fixes some weird things (e.g. message_create_t::send taking with a non-const lvalue reference to dpp::message
Makes VS2022 use PCH because why not

API remains the same for everything
Code change checklist

[x] I have ensured that all methods and functions are fully documented using doxygen style comments.
[x] My code follows the coding style guide.
[x] I tested that my change works before raising the PR.
[x] I have ensured that I did not break any existing API calls.
[x] I have not built my pull request using AI, a static analysis tool or similar without any human oversight.

squeaky clean new code, when can we merge it 😄

now plz, with a release too if possible <3

When I fix ARM 😭
And tests apparently?

your new unit test fails


Adopt PEP-621 for packaging

Contributor checklist

[x] Provided the tests for the changes.
[x] Assure PR title is short, clear, and good to be included in the user-oriented changelog

Maintainer checklist

[x] Assure one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
[x] Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

FYI it looks like this org wants to standardize on packaging and the first few migrated to Hatchling https://github.com/jazzband/help/issues/313


DNS provider configuration: alternatives to secretKeyRef

/kind design
Introduction
Currently, there are several DNS providers for the acme issuer. Each provider is configured differently, mostly due to the different requirements of cloud providers.
Each provider's required credentials and configuration is modelled as part of the issuer spec. Fields are modeled as either a reference to a secret (via something like apiKeySecretRef) or as a string value (e.g. email).
In this issue, I'd like to suggest some other possible ways to model this configuration and advantages / disadvantages of them.
Note that there may be a broader issue to discuss the configuration of issuers in general, but I'd like to focus on specifically acme dns provider configuration in this issue.
Options
I will use the following AWS configuration as an example in examining various options. It is representative, I think, since it has secret references, optional configuration (hostedZoneID), and non-secret inline values (region, accessKeyID).
Currently, it looks like the following:

kind: Issuer
spec:
  acme:
    dns01:
      providers:
      - name: r53
        route53:
          region: us-west-2
          hostedZoneID: optional-zoneid
          accessKeyID: AKIDEXAMPLE
          secretAccessKeySecretRef:
            name: route53-creds
            key: secret-key
---

apiVersion: v1
kind: Secret
metadata:
  name: route53-creds
data:
  secret-key: base64("SECRET_KEY")

Option 1: value/valueFrom everywhere
One option would be to convert everything to have a value / valueFrom style reference value.
The above example would then look like:

kind: Issuer
spec:
  acme:
    dns01:
      providers:
      - name: r53
        route53:
          region:
            value: us-west-2
          hostedZoneID:
            value: optional-zoneid
          accessKeyID:
            value: AKIDEXAMPLE
          secretAccessKey:
            valueFrom:
              secretKeyRef:
                name: route53-creds
                key: secret-key
---

apiVersion: v1
kind: Secret
metadata:
  name: route53-creds
data:
  secret-key: base64("SECRET_KEY")

Advantages:

Flexible, allowing people to store values in secret or as inline literals as they prefer (xref https://github.com/jetstack/cert-manager/pull/197)
Similar to how env vars are configured in the kubernetes pod spec, and thus we can lean on upstream types, docs, and knowledge

Disadvantages:

Flexible; it provides more ways to configure a provider than now, which means some additional complexity
Difficult to fully validate, though no more difficult than existing secretKeyRefs

Option 2: New CRD types per provider config
In an ideal world, Kubernetes might allow a CRD-like extension to secret types. Right now, there are secret types like kubernetes.io/tls and kubernetes.io/basic-auth and so on. Each of these types allows convergence so that those secrets may be shared.
If we could have a secret type such as aws.kubernetes.io/credentials, that would allow the whole K8s ecosystem to converge on a similar way of storing those credentials and reference them.
Since that isn't available presently, we could attempt to create such a standard format by creating a new type for each provider config. For the AWS example, this might look like:

apiVersion: certmanager.k8s.io/v1alpha1
Kind: AWSCredentials
metadata:
  name: r53-creds
spec:
  region: us-west-2
  accessKeyID: AKIDEXAMPLE
  secretAccessKey: SECRET

----

kind: Issuer
spec:
  acme:
    dns01:
      providers:
      - name: r53
        route53:
          awsCredentialsName: r53-creds
          hostedZoneID: optional-zoneid # zone-id could fit in a "services: route53: zone-id" section in AWSCredentials, or it could be a more specific AWSRoute53Config or something

Advantages:

Slightly better validation possible (the new types can be validated, and the user just has to get the reference to its name right, while before there are more total references the user must get right and the secret is not validated on its own).
Potential for integration with other projects.

Disadvantages:

More complex, possibly needlessly so.
Probably the wrong project to try and create such standard types in.
Storing secret information in this new type is more complex than making a simple CRD (I think it's possible with an aggregate api and some difficulty).
Initially less compatible with other tooling/projects; e.g. no equivalent to kubectl create secret will be there.

Option 3: Always just reference a configmap or secret for everything, document expected structure
Option 1 defines all the keys as part of the issuer CRD, and then allows referencing configmaps or secrets. This one inverts it and asks users to create the configmap or secret themselves with well-defined keys, and then only asks for them to reference its name.

kind: Issuer
spec:
  acme:
    dns01:
      providers:
      - name: r53
        route53:
          config:
            secretName: route53-config
            # or configMapName: ....
---

apiVersion: v1
kind: Secret
metadata:
  name: route53-config
data:
  # all keys must be named exactly like so
  accessKey: base64("AKIDEXAMPLE")
  secretKey: base64("SECRET_KEY")
  region: base64("us-west-2")
  hostedZoneID: base64("optional zone-id")

Advantages:

Simplest issuer CRD of all of them since fields are not defined there.

Disadvantages:

Less "type-safe" / harder to validate due to keys being a convention, not a part of the CRD.
May require users to duplicate data if we pick different key names from any other tools which do something similar.

Conclusion
I personally think that Option 1 is probably the best to transition to. It's not significantly different from what we have now, but is a little more flexible and a little more "kubernetes-ey" in my eyes.
Unfortunately, I couldn't find much in the way of prior art. Tools like external-dns and kube2iam, as far as I can tell, don't allow referencing a configmap or secret for configuration.

If anyone has additional options that are worth considering, I'd welcome them, preferably with an example of what a provider's configuration might look like with that option.
@munnerz I think you may have had a more broad issue in mind for issuer configuration. If you think it's more valuable to also try and look at things from a little higher up first, we could do that.
After looking through other existing issuers/PRs, it seems like there's fairly little noise about the existing format (other than my pr, https://github.com/jetstack/cert-manager/pull/197), so it's possible no one really cares all that strongly about this detail of configuration.
Given there's much more noise about additional (possibly remote) providers/issuers, it may also be worth thinking about how they might be configured first to see if this is relevant or not.


Adding a way to overrride the default image name on paste.

Hi there I'm loving your plugin alot. I wanted a way to name the image on paste so I added some code to do that.
Its a little bit tacked on and messy but please consider incorporating this feature if you don't like the code.

Or maybe I should have taken a look at your tips page.


WFS200 FutureWarning

Line https://github.com/geopython/OWSLib/blob/f48d8f69ada2a746072b16c8b3cba1b62c5fd1ec/owslib/feature/wfs200.py#L153
raises
FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.

@huard would you like to provide a fix?

@huard ok, already fixed by #595.


freeze app for 3  4 seconds for 50 lines or more

when set text that have more than 50 lines app freeze for 3  4 seconds , how solve this issue?

same problem

same problem please resolve issue. thanks


Error on decoding ISO-8859-1 words

Version
3.0.1
Reproduction link
https://codepen.io/anon/pen/ZPdOxa?editors=1010
Steps to reproduce
First click on the "Test Ok" link and see the route populate the $this.route.query.q. Then, click on the "Test fail" and see the route chash and not populate the $this.route.query.q.
What is expected?
It was expected that the router converted the "test%E9" to "testé" and populate the this.$route.query.q.
What is actually happening?
Vue-router is failing to populate the this.$route.query.q.

By debugging the code,  I realized  that the decodeURIComponent does not work properly with  ISO-8859-1 words. In other hand, the unescape method works well.
Example below:
text = 'test%E9'
> "test%E9"

decodeURIComponent(text)
> VM568:1 Uncaught URIError: URI malformed
    at decodeURIComponent (<anonymous>)
    at <anonymous>:1:1

unescape(text)
> "testé"

Yeah, the correct encoding of é is %C3%A9. You can check that with the encodeURI/decodeURI function. As the error says, that's a malformed URI and is not allowed (by the browser)

Hey @posva  thanks for answer. I'm working with Vuejs on a old legacy project that's use latin-1 encode (ISO-8859-1), so, unfortunately, I couldn't use the correct encoding.
For those who might have the same problem: By searching on the docs I discovered how to pass a  parseQuery property on the route file that receives a function, so that a I  could "override" the original decodeURIComponent. I don't know if it's the best way to do it, but it's worked for me. =)
Here is my code as an example:
parseQuery: (query) => {
    const res = {};
    query = query.trim().replace(/^(\?|#|&)/, '');
    if (!query) {
      return res;
    }
    query.split('&').forEach((param) => {
      const parts = param.replace(/\+/g, ' ').split('=');

      const key = decodeURIComponent(parts.shift());

      let val = '';
      if (parts.length > 0) {
        try {
          val = decodeURIComponent(parts.join('='));
        } catch (e) {
          val = unescape(parts.join('='));
        }
      } else {
        val = null;
      }

      if (res[key] === undefined) {
        res[key] = val;
      } else if (Array.isArray(res[key])) {
        res[key].push(val);
      } else {
        res[key] = [res[key], val];
      }
    });

    return res;
  },

fdsbxcbxc


Wrong urls in Releases documentation

The Cyanite homepage at http://cyanite.io/ says
The latest release is available at https://github.com/pyr/cyanite/releases/latest

Clicking through to that url results in this message from GitHub
There aren’t any releases here

The url http://cyanite.io/quickstart.html#obtaining-cyanite says
Binary release are the simplest way to get started and are hosted on github: https://github.com/exoscale/cyanite/releases/latest.

Clicking through to that url results in a 404 error from GitHub
Could you please update these with the correct location of the binary releases?

Hi @alangibson,
This is because there are no binary releases yet, there's a - admitedly small - warning on main page of cyanite.io. The release is imminent though.


Add missing NOTE_* constants to apple

Source link: https://github.com/apple-oss-distributions/xnu/blob/94d3b452840153a99b38a3a9659680b2a006908e/bsd/sys/event.h#L318

Actually this needs the demver test, can you add these to libc-test/semver/apple.txt?

LGTM, please squash and I'll merge

@rustbot label +stable-nominated


Batch MPI pool example

Adds a module to create a Batch Node Pool with VMs configured to run MPI workloads with Intel MPI and (optionally) with an NFS share pre-mounted on each VM.
Submission Checklist
Please take the following actions before submitting this pull request.

Fork your PR branch from the Toolkit "develop" branch (not main)
Test all changes with pre-commit in a local branch #
Confirm that "make tests" passes all tests
Add or modify unit tests to cover code changes
Ensure that unit test coverage remains above 80%
Update all applicable documentation
Follow Cloud HPC Toolkit Contribution guidelines #

We are moving forward (in separate PRs) to make this functionality as part of the batch-job-template module.


Fix client testing

Well what do you know, the pull request template works!

Description
This pull request comments out a couple of troublesome client tests that were breaking the test runner. I think it is fine to leave them in a commented out-of-state, as the mockups for the next version of the client does not include the functionality that they cover.
It also puts back the line in the circle.yml file that will run grunt karma
Related Issue
closes #516
Motivation and Context
False negatives with Karma testing was breaking builds.
How Has This Been Tested?
I have run the test runner locally several times in succession and have not seen the previously breaking issues.
Types of changes

[x] Bug fix (non-breaking change which fixes an issue).o
[ ] New feature (non-breaking change which adds functionality).
[ ] Breaking change (fix or feature that would cause existing functionality to change).

Checklist:

[ ] My change requires new model migrations.
[ ] I have executed makemigrations (if necessary).
[ ] My change requires new docker images for development.
[ ] I have created and pushed new docker images (if necessary).
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly (if necessary).
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed locally.
[ ] Test coverage percentage has improved.
[ ] PEP8 passes (for python code).
[ ] Pylint score has improved (for python code).

[Current coverage][cc-pull] is 69.01%

Merging [#558][cc-pull] into [develop][cc-base-branch] will not change coverage

@@            develop       #558   diff @@
==========================================
  Files            37         37
  Lines          1336       1336
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits            922        922
  Misses          414        414
  Partials          0          0

[][cc-pull]

Powered by Codecov. Last updated by [e8e3a28...6bf07e8][cc-compare]
[cc-base-branch]: https://codecov.io/gh/Solinea/goldstone-server/branch/develop?src=pr
[cc-compare]: https://codecov.io/gh/Solinea/goldstone-server/compare/e8e3a28d49f4a56f898396f05c7258fa1fe27388...6bf07e8d92ae184161b20a0bd15a6857ae9b5ffc
[cc-pull]: https://codecov.io/gh/Solinea/goldstone-server/pull/558?src=pr

👍


Can't Install Kaolin on linux

Hello, I'm trying to install kaolin on a remote linux machine but when I run python setup.py I get the following error:
gcc -pthread -B /home/IIT.LOCAL/arosasco/mambaforge/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/IIT.LOCAL/arosasco/mambaforge/include -fPIC -O2 -isystem /home/IIT.LOCAL/arosasco/mambaforge/include -fPIC -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -I/home/IIT.LOCAL/arosasco/libs/kaolin/third_party/cub -I/home/IIT.LOCAL/arosasco/libs/kaolin/.eggs/torch-1.9.0-py3.9-linux-x86_64.egg/torch/include -I/home/IIT.LOCAL/arosasco/libs/kaolin/.eggs/torch-1.9.0-py3.9-linux-x86_64.egg/torch/include/torch/csrc/api/include -I/home/IIT.LOCAL/arosasco/libs/kaolin/.eggs/torch-1.9.0-py3.9-linux-x86_64.egg/torch/include/TH -I/home/IIT.LOCAL/arosasco/libs/kaolin/.eggs/torch-1.9.0-py3.9-linux-x86_64.egg/torch/include/THC -I/usr/local/cuda-8.0/include -I/home/IIT.LOCAL/arosasco/mambaforge/include/python3.9 -c kaolin/csrc/metrics/sided_distance.cpp -o build/temp.linux-x86_64-3.9/kaolin/csrc/metrics/sided_distance.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
/usr/local/cuda-8.0/bin/nvcc -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -I/home/IIT.LOCAL/arosasco/libs/kaolin/third_party/cub -I/home/IIT.LOCAL/arosasco/libs/kaolin/.eggs/torch-1.9.0-py3.9-linux-x86_64.egg/torch/include -I/home/IIT.LOCAL/arosasco/libs/kaolin/.eggs/torch-1.9.0-py3.9-linux-x86_64.egg/torch/include/torch/csrc/api/include -I/home/IIT.LOCAL/arosasco/libs/kaolin/.eggs/torch-1.9.0-py3.9-linux-x86_64.egg/torch/include/TH -I/home/IIT.LOCAL/arosasco/libs/kaolin/.eggs/torch-1.9.0-py3.9-linux-x86_64.egg/torch/include/THC -I/usr/local/cuda-8.0/include -I/home/IIT.LOCAL/arosasco/mambaforge/include/python3.9 -c kaolin/csrc/metrics/sided_distance_cuda.cu -o build/temp.linux-x86_64-3.9/kaolin/csrc/metrics/sided_distance_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_60,code=compute_60 -gencode=arch=compute_60,code=sm_60 -std=c++14
nvcc fatal   : Value 'c++14' is not defined for option 'std'
error: command '/usr/local/cuda-8.0/bin/nvcc' failed with exit code 1

Also, even though I already have installed pytorch 1.8 I get this:
/home/IIT.LOCAL/arosasco/libs/kaolin/setup.py:24: UserWarning: Couldn't find torch installed, so this will try to install it. If the installation fails we recommend to first install it.
  warnings.warn("Couldn't find torch installed, so this will try to install it. "

Thanks for your interest in using Kaolin!
It seems from the error that you're using CUDA 8.0. The minimum supported version of CUDA for Kaolin is CUDA 10.0, so I would reccomend upgrading. I believe this error is happening because cpp14 is not supported in old versions of CUDA.

Oh ok, sadly CUDA 8.0 is the latest version of CUDA on the remote machine and I don't have superuser permission. Thanks anyway!


Deleted files appear in directory after server restart

Hello,
so like said in the title, when I delete files in the directory that came with the plugin, they reappear after every 2nd server restart. First picture is what I want to have and the second is when files reappear. How can I make them not reappear after deletion?
Hope you have any idea why this happens, otherwise I'm loving the plugin.
Best regards

They should not re-appear after restart/reload if you disable example qualities in the Config. Do you have it enabled?

Yeah that fixed it, thanks man.

Of course! Thanks for your report either way. Have a great day!


Add verbiage for VM size restrictions on disk utilization & bandwidth

The size of a VM (that supports premium disk) also has a disk performance limitation. The article does not state whether the VM size can inhibit or otherwise throttle overall disk performance capability. I'm assuming regardless of the disk performance tier, the VM size will still have a performance ceiling that the new disk performance tiers cannot override.
Please consider adding a bullet to the section https://docs.microsoft.com/en-us/azure/virtual-machines/disks-performance-tiers#restrictions that indicates whether this limitation applies.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5846046f-fa6a-5440-76dd-73108299a0f6
Version Independent ID: 974145a5-3d9b-76cf-2f7f-3bd3e42cfdfa
Content: Change the performance of Azure managed disks - Azure Virtual Machines
Content Source: articles/virtual-machines/disks-performance-tiers.md
Service: virtual-machines
Sub-service: disks
GitHub Login: @roygara
Microsoft Alias: rogarana

@CHDAFNI-MSFT
Thanks for the feedback! I have assigned the issue to content author to check and update the document as appropriate.

@roygara
Can you please check and add your comments on this doc update request as applicable.

Hello, @CHDAFNI-MSFT, thanks for reaching out. We appreciate your feedback.
We won't be adding this to the restriction section. However, I have updated the conceptual article to include a link to our article which covers how performance works.
That change will be live sometime tomorrow.
Accordingly, I'll close this out.
#please-close


Register source_id BESM-2-6

"BESM-2-6":{
    "activity_participation":[
        "CMIP"
    ],
    "aerosol":"None",
    "atmosphere":"BAM (v1.0; T062L28, 192 x 96 longitude/latitude; 28 levels; top level 3 hPa)",
    "atmospheric_chemistry":"None",
    "cohort":[
        "CMIP5"
    ],
    "institution_id":[
        "INPE"
    ],
    "label":"BESM 2.6",
    "label_extended":"BESM 2.6 (This entry is free text for users to contribute verbose information)",
    "land_ice":"None",
    "land_surface":"SSiB",
    "nominal_resolution_atmos":[
        "1.875 Degree"
    ],
    "nominal_resolution_landIce":[
        "None"
    ],
    "nominal_resolution_ocean":[
        "1.0 Degree"
    ],
    "ocean":"MOM-5 (MOM5; tripolar primarily 1 deg, 360 x 300 longitude/latitude; 50 levels; top grid cell 0-10 m)",
    "ocean_biogeochemistry":"TOPAZ",
    "release_year":"2012",
    "sea_ice":"SIS",
    "source_id":"BESM-2-6"
},

@besm-inpe pinging you here

@besm-inpe we need some additional information from you to accurately register this source_id, I have amended your entry above and added a ### where additional information is required:
"BESM-2-6":{
    "activity_participation":[
        "CMIP"
    ],
    "aerosol":"None",
    "atmosphere":"BAM (v1.0; T062L28, 192 x 96 longitude/latitude; 28 levels; top level 3 hPa)",
    "atmospheric_chemistry":"None",
    "cohort":[
        "Registered"
    ],
    "institution_id":[
        "INPE"
    ],
    "label":"BESM 2.6",
    "label_extended":"BESM 2.6",
    "land_ice":"None",
    "land_surface":"SSiB", ### What is the version number? i.e. SSiB 1.0
    "nominal_resolution_aerosol":"None"
    "nominal_resolution_atmos":"1.875 Degree" ### This is not a valid entry
    "nominal_resolution_atmosChem":"None"
    "nominal_resolution_land":"" ### Please enter this value
    "nominal_resolution_landIce":"None"
    "nominal_resolution_ocean":"1.0 Degree" ### This is not a valid entry
    "nominal_resolution_ocnBgChem":"" ### Please enter this value
    "nominal_resolution_seaIce":"" ### Please enter this value
    "ocean":"MOM-5 (MOM5; tripolar primarily 1 deg, 360 x 300 longitude/latitude; 50 levels; top grid cell 0-10 m)",
    "ocean_biogeochemistry":"TOPAZ", ### What is the version number? i.e. TOPAZ 1.0
    "release_year":"2012",
    "sea_ice":"SIS", ### What is the version number? i.e. SIS 1.0
    "source_id":"BESM-2-6"
},

For the nominal_resolution entries, please select the closest value from the list found in CMIP6_CVs/CMIP6_nominal_resolution.json.
Just for clarity, all the CMIP6 endorsed activities are listed at CMIP6_CVs/CMIP6_activity_id.json, please add all relevant MIPs that you will be contributing to using this model

@besm-inpe have you had a chance at obtaining the requested information above?

Dear Paul
Thank you for your msg. Yes, will send the requested information asap.
Thank you for your patience,
...paulo
On Fri, Apr 21, 2017 at 10:34 PM, Paul J. Durack<EMAIL_ADDRESS>wrote:

@besm-inpe https://github.com/besm-inpe have you had a chance at
obtaining the requested information above?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/WCRP-CMIP/CMIP6_CVs/issues/287#issuecomment-296338220,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AZO__zE1ZZlp8tjJMnfM1eiMVXs6JORmks5ryVkNgaJpZM4M9Iku
.

  "BESM-2-7":{
        "activity_participation":[
            "CMIP"
        ],
        "aerosol":"None",
        "atmosphere":"BAM (v1.0; T062L28, 192 x 96 longitude/latitude; 28 levels; top level 3 hPa)",
        "atmospheric_chemistry":"None",
        "cohort":[
            "CMIP5"
        ],
        "institution_id":[
            "INPE"
        ],
        "label":"BESM 2.7",
        "label_extended":"BESM 2.7 (This entry is free text for users to contribute verbose information)",
        "land_ice":"None",
        "land_surface":"SSiB 2.0",
        "nominal_resolution_land":[
            "250 km",
        ],
        "nominal_resolution_atmos":[
            "250 km",
        ],
        "nominal_resolution_landIce":"None",
        "nominal_resolution_ocean":[
            "1x1 degree",
        ],
        "ocean":"MOM-5 (MOM5; tripolar primarily 1 deg, 360 x 300 longitude/latitude; 50 levels; top grid cell 0-10 m)",
        "ocean_biogeochemistry":"TOPAZ 1.0",
        "nominal_resolution_ocnBgChem":"1x1 degree",
        "release_year":"2012",
        "sea_ice":"SIS 1.0",
        "nominal_resolution_seaIce":"1x1 degree",
        "source_id":"BESM-2-7"
    },

@besm-inpe thanks for this, I note you've amended the registration from BESM-2-6 to BESM-2-7 did you want to replace the existing entry, or add another?
I have rearranged some of the entries above to make it clearer what information has been provided.
Regarding the nominal_resolution identifier, if you do not have an explicit 1x1 degree grid (which would is exactly 360 x 180 longitude/latitude), then this should be more correctly labeled to the closest entry in CMIP6_nominal_resolution.json for the average resolution across your grid. For your examples (ocean: 360 x 300) it would most likely be 100 km or 250 km. The Appendix 2 in the CMIP6 Global Attributes, DRS, Filenames, Directory Structure, and CV’s file provides more comprehensive information

Dear Paul
Thank you for your msg and rearrangement of our entries to BESM.
Yes, please change the model version to BESM-2-7.
The ocean resolution shall then be 100 Km.
It is in fact 1 degree longitude and  1/4 degree latitude between 10S-10N,
varying to 1 degree at 40S and 40N, and then uniform 1 x 1 to the poles.
Best,
Paulo Nobre
On Tue, Apr 25, 2017 at 4:08 PM, Paul J. Durack<EMAIL_ADDRESS>wrote:

@besm-inpe https://github.com/besm-inpe thanks for this, I note you've
amended the registration from BESM-2-6 to BESM-2-7 did you want to
replace the existing entry, or add another?
I have rearranged some of the entries above to make it clearer what
information has been provided.
Regarding the nominal_resolution identifier, if you do not have an
explicit 1x1 degree grid (which would is exactly 360 x 180
longitude/latitude), then this should be more correctly labeled to the
closest entry in CMIP6_nominal_resolution.json
https://github.com/WCRP-CMIP/CMIP6_CVs/blob/master/CMIP6_nominal_resolution.json
for the average resolution across your grid. For your examples (ocean: 360
x 300) it would most likely be 100 km or 250 km. The Appendix 2 in the CMIP6
Global Attributes, DRS, Filenames, Directory Structure, and CV’s
https://goo.gl/v1drZl file provides more comprehensive information
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/WCRP-CMIP/CMIP6_CVs/issues/287#issuecomment-297133908,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AZO__5crH8XXjrjslbnLH4jXmTw5UadFks5rzkScgaJpZM4M9Iku
.

@besm-inpe no problem, is release_year = 2012 correct?

@besm-inpe I updated this to be release_year = 2017. If you want to take a peek at CMIP6_source_id.html and double check the entries, that would be great

Hi Paul
Thank you for the inclusion of changes. In fact, some further changes
needed.
Will to it tomorrow morning.
Best,
...paulo
On Tue, Apr 25, 2017 at 4:52 PM, Paul J. Durack<EMAIL_ADDRESS>wrote:

@besm-inpe https://github.com/besm-inpe I updated this to be release_year
= 2017. If you want to take a peek at CMIP6_source_id.html
http://rawgit.com/WCRP-CMIP/CMIP6_CVs/master/src/CMIP6_source_id.html
and double check the entries, that would be great
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/WCRP-CMIP/CMIP6_CVs/issues/287#issuecomment-297145557,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AZO__xtsCVd2UAUWGE3pDXA9s9twjrBnks5rzk8ZgaJpZM4M9Iku
.

@besm-inpe, no problem. The current listing shows the following:
"BESM-2-7":
"activity_participation":[
   "CMIP"
],
"cohort":[
   "Registered"
],
"institution_id":[
   "INPE"
],
"label":"BESM 2.7",
"label_extended":"BESM 2.7",
"model_component":
"aerosol":
   "description":"none",
   "nominal_resolution":"none"
"atmos":
   "description":"BAM (v1.0, T062L28; 192 x 96 longitude/latitude; 28 levels; top level 3 hPa)",
   "nominal_resolution":"250 km"
"atmosChem":
   "description":"none",
   "nominal_resolution":"none"
"land":
   "description":"SSiB 2.0",
   "nominal_resolution":"250 km"
"landIce":
   "description":"none",
   "nominal_resolution":"none"
"ocean":
   "description":"MOM-5 (MOM5, tripolar primarily 1 deg; 360 x 300 longitude/latitude; 50 levels; top grid cell 0-10 m)",
   "nominal_resolution":"100 km"
   "ocnBgchem":
   "description":"TOPAZ 1.0",
   "nominal_resolution":"100 km"
"seaIce":
   "description":"SIS 1.0",
   "nominal_resolution":"100 km"
"release_year":"2017",
"source_id":"BESM-2-7"

Just make whatever amendments are required and reopen the issue below

    "BESM-2-7":{
        "activity_participation":[
            "CMIP", "DCPP", "HighResMIP", "ScenarioMIP",
        ],
        "aerosol":"None",
        "atmosphere":"BAM (v1.0; T062L28, 192 x 96 longitude/latitude; 28 levels; top level 3 hPa)",
        "atmospheric_chemistry":"None",
        "cohort":[###?
        ],
        "institution_id":"INPE"
        "label":"BESM 2.7",
        "label_extended":"BESM 2.7 (This entry is free text for users to contribute verbose information)",
        "land_ice":"None",
        "land_surface":"SSiB 2.0",
        "nominal_resolution_land":"250 km",
        "nominal_resolution_atmos":"250 km",
        "nominal_resolution_landIce":"None",
        "nominal_resolution_ocean":"100 Km",
        "ocean":"MOM-5 (MOM5; tripolar primarily 1 deg, 1/4 deg between 10S-10N, 360 x 300 longitude/latitude; 50 levels; top grid cell 0-10 m)",
        "ocean_biogeochemistry":"TOPAZ 2.0",
        "nominal_resolution_ocnBgChem":"100 Km",
        "release_year":"2017",
        "sea_ice":"SIS 1.0",
        "nominal_resolution_seaIce":"100 Km",
        "source_id":"BESM-2-7"
    },

@besm-inpe the revisions have been made, take a peek at CMIP6_source_id.html and let me know if any other tweaks are required

Dear Paul!
It looks OK now.
Thank you!
...paulo
On Wed, Apr 26, 2017 at 1:21 PM, Paul J. Durack<EMAIL_ADDRESS>wrote:

@besm-inpe https://github.com/besm-inpe the revisions have been made,
take a peek at CMIP6_source_id.html
http://rawgit.com/WCRP-CMIP/CMIP6_CVs/master/src/CMIP6_source_id.html
and let me know if any other tweaks are required
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/WCRP-CMIP/CMIP6_CVs/issues/287#issuecomment-297464900,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AZO__3nBfjigNJCxmdaS66STVl--Hd6Cks5rz28SgaJpZM4M9Iku
.

@besm-inpe no problem, if you have any further tweaks before starting to write CMIP6 data we can always amend the entry..


enable github hosted docs

Looks good! Obviously, we want the same for esp-idf-svc and embedded-svc


onError callback prop

Is your feature request related to a problem? Please describe.
I am looking a way to handle errors when images don't load (cloudflare moment)
Describe the solution you'd like
an onError callback function would be really helpful to do so, that way i know when to have a error message appear or something like that
Describe alternatives you've considered
I've considered modifying this libraries source to do so but I will not because of time constraints
Additional context
No response

Duplicate #183


Improve Raster.reproject() performance by changing rio.warp.reproject kwargs

rio.warp.reproject has two kwargs that we should expose (and possibly have defaults for): num_threads and warp_mem_limit. The first is an obvious improvement, and just defaults to 1 right now with no way to change it. The second needs more checking, but since rio is so careful with memory, it might speed up things considerably to allow for more memory to be used.
Also, the **kwargs for Raster.reproject() is not used and should be removed or be given a use!
https://rasterio.readthedocs.io/en/latest/api/rasterio.warp.html?highlight=reproject#rasterio.warp.reproject
So:

Expose num_threads as a keyword argument. I think it should default to os.cpu_count() - 1 or similar. No reason to not use multithreading apart from quite specific use-cases!
Experiment with the warp_mem_limit kwarg and either just expose it, and/or assign a more aggressive default value.
Remove **kwargs or give it a use.

Closed by #181


Download llama-3-8b-bnb-4bit  jupyter and run it locally，Error: tensors found at least two devices

RuntimeError                              Traceback (most recent call last)
Input In [8], in <cell line: 1>()
----> 1 trainer_stats = trainer.train()
File ~/.conda/envs/llama/lib/python3.9/site-packages/trl/trainer/sft_trainer.py:361, in SFTTrainer.train(self, *args, **kwargs)
358 if self.neftune_noise_alpha is not None and not self._trainer_supports_neftune:
359     self.model = self._trl_activate_neftune(self.model)
--> 361 output = super().train(*args, **kwargs)
363 # After training we make sure to retrieve back the original forward pass method
364 # for the embedding layer by removing the forward post hook.
365 if self.neftune_noise_alpha is not None and not self._trainer_supports_neftune:
File ~/.conda/envs/llama/lib/python3.9/site-packages/transformers/trainer.py:1859, in Trainer.train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
1857         hf_hub_utils.enable_progress_bars()
1858 else:
-> 1859     return inner_training_loop(
1860         args=args,
1861         resume_from_checkpoint=resume_from_checkpoint,
1862         trial=trial,
1863         ignore_keys_for_eval=ignore_keys_for_eval,
1864     )
File :361, in _fast_inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
File ~/.conda/envs/llama/lib/python3.9/site-packages/transformers/trainer.py:3138, in Trainer.training_step(self, model, inputs)
3135     return loss_mb.reduce_mean().detach().to(self.args.device)
3137 with self.compute_loss_context_manager():
-> 3138     loss = self.compute_loss(model, inputs)
3140 if self.args.n_gpu > 1:
3141     loss = loss.mean()  # mean() to average on multi-gpu parallel training
File ~/.conda/envs/llama/lib/python3.9/site-packages/transformers/trainer.py:3161, in Trainer.compute_loss(self, model, inputs, return_outputs)
3159 else:
3160     labels = None
-> 3161 outputs = model(**inputs)
3162 # Save past state if it exists
3163 # TODO: this needs to be fixed and made cleaner later.
3164 if self.args.past_index >= 0:
File ~/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
1510 else:
-> 1511     return self._call_impl(*args, **kwargs)
File ~/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
1515 # If we don't have any hooks, we want to skip the rest of the logic in
1516 # this function, and just call forward.
1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1518         or _global_backward_pre_hooks or _global_backward_hooks
1519         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520     return forward_call(*args, **kwargs)
1522 try:
1523     result = None
File ~/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py:185, in DataParallel.forward(self, *inputs, **kwargs)
183     return self.module(*inputs[0], **module_kwargs[0])
184 replicas = self.replicate(self.module, self.device_ids[:len(inputs)])
--> 185 outputs = self.parallel_apply(replicas, inputs, module_kwargs)
186 return self.gather(outputs, self.output_device)
File ~/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py:200, in DataParallel.parallel_apply(self, replicas, inputs, kwargs)
199 def parallel_apply(self, replicas: Sequence[T], inputs: Sequence[Any], kwargs: Any) -> List[Any]:
--> 200     return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File ~/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/parallel/parallel_apply.py:108, in parallel_apply(modules, inputs, kwargs_tup, devices)
106     output = results[i]
107     if isinstance(output, ExceptionWrapper):
--> 108         output.reraise()
109     outputs.append(output)
110 return outputs
File ~/.conda/envs/llama/lib/python3.9/site-packages/torch/_utils.py:722, in ExceptionWrapper.reraise(self)
718 except TypeError:
719     # If the exception takes multiple arguments, don't try to
720     # instantiate since we don't know how to
721     raise RuntimeError(msg) from None
--> 722 raise exception
RuntimeError: Caught RuntimeError in replica 1 on device 1.
Original Traceback (most recent call last):
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in _worker
output = module(*input, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/models/llama.py", line 882, in PeftModelForCausalLM_fast_forward
return self.base_model(
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/peft/tuners/tuners_utils.py", line 161, in forward
return self.model.forward(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/models/llama.py", line 813, in _CausalLM_fast_forward
outputs = self.model(
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/models/llama.py", line 650, in LlamaModel_fast_forward
hidden_states = Unsloth_Offloaded_Gradient_Checkpointer.apply(
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/autograd/function.py", line 553, in apply
return super().apply(*args, **kwargs)  # type: ignore[misc]
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/cuda/amp/autocast_mode.py", line 115, in decorate_fwd
return fwd(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/models/_utils.py", line 333, in forward
(output,) = forward_function(hidden_states, *args)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/models/llama.py", line 433, in LlamaDecoderLayer_fast_forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/models/llama.py", line 308, in LlamaAttention_fast_forward
Q, K, V = self.apply_qkv(self, hidden_states)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/kernels/fast_lora.py", line 312, in apply_lora_qkv
Q, K, V = LoRA_QKV.apply(X,
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/autograd/function.py", line 553, in apply
return super().apply(*args, **kwargs)  # type: ignore[misc]
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/torch/cuda/amp/autocast_mode.py", line 115, in decorate_fwd
return fwd(*args, **kwargs)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/kernels/fast_lora.py", line 227, in forward
Q = matmul_lora(X, QW, QW_quant, QA, QB, QS)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/kernels/utils.py", line 225, in matmul_lora
W = fast_dequantize(W.t(), W_quant)
File "/home/yanai-lab/xiong-p/.conda/envs/llama/lib/python3.9/site-packages/unsloth/kernels/utils.py", line 104, in fast_dequantize
out_absmax += offset
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!

@Qinolion Are you using multiple GPUs?

yes

yes

@Qinolion  since you are using multiple GPUs, you will face this error.
Set the environment variable
by this shell command. (replace 1 by the number of GPU you want to use)
export CUDA_VISIBLE_DEVICES=1
@danielhanchen Just wanted to confirm can we run the free version of Unsloth on multiple GPUs?

@erwe324 The current OSS here won't exactly work for multi GPU, but Llama Factory's Unsloth integration does have a slow somewhat functional multi GPU offering.
Accelerate only partially works, and isn't really optimized


Poseidon truncates output sent to client

In the course of evaluating https://github.com/openHPI/dockerfiles/pull/37, I tested the new image locally. To do so, I've built the base image and one execution image (namely openhpi/co_execenv_java:8-antlr). Then, I switched to the execution environment shell and aimed to verify the environment variables by executing the env command.
This yielded surprising results, including the following (not edited in any way):

Actually, it turned out that "sometimes" I got the first variable only but not all. I've double checked the setup and everything, but couldn't find any issue in CodeOcean. Executing the same command over and over again in the Nomad UI shell didn't reveal the same behavior (but, of course, still listed all environment variables):

My last resort was to double-check Poseidon. To do so, I added the following log statement (in internal/nomad/sentry_debug_writer.go):
func (s *SentryDebugWriter) Write(debugData []byte) (n int, err error) {
	log.WithContext(s.Ctx).WithField("data", string(debugData[:])).Trace("SentryDebugWriter Write")
    # ... existing code

Surprisingly, for the two occurrences you see in the screenshot, my logs showed:
time="2024-09-04T17:04:46.094047Z" level=trace msg="SentryDebugWriter Write" data="\x1ePoseidon env<PHONE_NUMBER>085\x1eHOSTNAME=4bf0a8e5dbe4\r\nLANGUAGE=en_US.UTF-8\r\nJAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64\r\nJUNIT=/usr/java/lib/junit-4.13.2.jar\r\nPWD=/workspace\r\n_=/usr/bin/env\r\nHOME=/workspace\r\nLANG=en_US.UTF-8\r\nCOLUMNS=10000\r\nTERM=ansi\r\nUSER=user\r\nSHLVL=2\r\nANTLR=/usr/java/lib/antlr-4.5.3-complete.jar\r\nCODEOCEAN=true\r\nCLASSPATH=.:/usr/java/lib/hamcrest-3.0.jar:/usr/java/lib/junit-4.13.2.jar:/usr/java/lib/antlr-4.5.3-complete.jar:/usr/java/lib/antlr-java8.jar\r\nHAMCREST=/usr/java/lib/hamcrest-3.0.jar\r\nLC_ALL=en_US.UTF-8\r\nPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\r\nANTLR_JAVA8=/usr/java/lib/antlr-java8.jar\r\nUID=1001\r\nDEBIAN_FRONTEND=teletype\r\n\x1ePoseidon exit 0<PHONE_NUMBER>087\x1e" environment_id=10 package=nomad runner_id=10-84eac7c0-6aca-11ef-be51-2e62a1fdfe47

time="2024-09-04T17:04:46.094094Z" level=trace msg="SentryDebugWriter Write" data="HOSTNAME=4bf0a8e5dbe4\r" environment_id=10 package=nomad runner_id=10-84eac7c0-6aca-11ef-be51-2e62a1fdfe47

time="2024-09-04T17:04:46.094141Z" level=trace msg="Sending message to client" environment_id=10 message="&{stdout HOSTNAME=4bf0a8e5dbe4\r 0}" package=ws runner_id=10-84eac7c0-6aca-11ef-be51-2e62a1fdfe47

time="2024-09-04T17:06:09.570818Z" level=trace msg="SentryDebugWriter Write" data="HOSTNAME=4bf0a8e5dbe4\r\nLANGUAGE=en_US.UTF-8\r\nJAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64\r\nJUNIT=/usr/java/lib/junit-4.13.2.jar\r\nPWD=/workspace\r\n_=/usr/bin/env\r\nHOME=/workspace\r\nLANG=en_US.UTF-8\r\nCOLUMNS=10000\r\nTERM=ansi\r\nUSER=user\r\nSHLVL=2\r\nANTLR=/usr/java/lib/antlr-4.5.3-complete.jar\r\nCODEOCEAN=true\r\nCLASSPATH=.:/usr/java/lib/hamcrest-3.0.jar:/usr/java/lib/junit-4.13.2.jar:/usr/java/lib/antlr-4.5.3-complete.jar:/usr/java/lib/antlr-java8.jar\r\nHAMCREST=/usr/java/lib/hamcrest-3.0.jar\r\nLC_ALL=en_US.UTF-8\r\nPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\r\nANTLR_JAVA8=/usr/java/lib/antlr-java8.jar\r\nUID=1001\r\nDEBIAN_FRONTEND=teletype\r\n" environment_id=10 package=nomad runner_id=10-84eac7c0-6aca-11ef-be51-2e62a1fdfe47

time="2024-09-04T17:06:09.570897Z" level=trace msg="Sending message to client" environment_id=10 message="&{stdout HOSTNAME=4bf0a8e5dbe4\r\nLANGUAGE=en_US.UTF-8\r\nJAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64\r\nJUNIT=/usr/java/lib/junit-4.13.2.jar\r\nPWD=/workspace\r\n_=/usr/bin/env\r\nHOME=/workspace\r\nLANG=en_US.UTF-8\r\nCOLUMNS=10000\r\nTERM=ansi\r\nUSER=user\r\nSHLVL=2\r\nANTLR=/usr/java/lib/antlr-4.5.3-complete.jar\r\nCODEOCEAN=true\r\nCLASSPATH=.:/usr/java/lib/hamcrest-3.0.jar:/usr/java/lib/junit-4.13.2.jar:/usr/java/lib/antlr-4.5.3-complete.jar:/usr/java/lib/antlr-java8.jar\r\nHAMCREST=/usr/java/lib/hamcrest-3.0.jar\r\nLC_ALL=en_US.UTF-8\r\nPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\r\nANTLR_JAVA8=/usr/java/lib/antlr-java8.jar\r\nUID=1001\r\nDEBIAN_FRONTEND=teletype\r\n 0}" package=ws runner_id=10-84eac7c0-6aca-11ef-be51-2e62a1fdfe47

A full, unfiltered log is available here: poseidon.log
@mpass99 Do you have any indication what is going on here?
From my understanding ...

[ ] Poseidon is filtering the full output and rather forwarding a single line (in this case the HOSTNAME)
[ ] the SentryDebugWriter Write method is called "twice", and the call with the full output is somehow lost

Despite fixing this bug, I would appreciate if you could add the following trace options:

[x] Unprocessed data sent as stdin to the container, preferably as string (not byte array) if printable (otherwise byte array?)
[x] Unprocessed data received as stdout from the container, preferably as string (not byte array) if printable (otherwise byte array?)
[x] Unprocessed data received as stderr from the container, preferably as string (not byte array) if printable (otherwise byte array?)

For me, it was more difficult to identify the issue, since the first log line I saw from Poseidon already had the "filtered" output ("Sending message to client") which wasn't complete (obviously).

Another example would be to run ps aux, where the input my stop after the first line and then start again:

This is also forward wrongly to CodeOcean:
2024-09-04 16:35:01.918826 UTC: Receiving from ws://<IP_ADDRESS>:7200/api/v1/runners/10-110aa742-6adb-11ef-b496-2e62a1fdfe47/websocket?executionID=2440ac07-461b-4452-ae17-375cf3fc7e87: "{\"type\":\"start\"}"
2024-09-04 16:35:02.016129 UTC: Receiving from ws://<IP_ADDRESS>:7200/api/v1/runners/10-110aa742-6adb-11ef-b496-2e62a1fdfe47/websocket?executionID=2440ac07-461b-4452-ae17-375cf3fc7e87: "{\"type\":\"stdout\",\"data\":\"USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND\\r\"}"
2024-09-04 16:35:02.016459 UTC: Receiving from ws://<IP_ADDRESS>:7200/api/v1/runners/10-110aa742-6adb-11ef-b496-2e62a1fdfe47/websocket?executionID=2440ac07-461b-4452-ae17-375cf3fc7e87: "{\"type\":\"stdout\",\"data\":\"cho -ne \\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\x1EPoseidon exit \\\\\\\\\\\\$ec \\\\\\\\\\\\$(date +%s%3N)\\\\\\\\\\\\\\\\x1E\\\\\\\\\\\\\\\" \\u0026\\u0026 exit \\\\\\\\\\\\$ec) 2\\u003e /tmp/stderr_1725467701918565000.fifo\\\\\\\"\\\"\\r\\nuser      1408 25.0  0.0   9972  3476 pts/0    Ss+  16:35   0:00 /bin/bash -c echo -ne \\\"\\\\x1EPoseidon mkfifo $(date +%s%3N)\\\\x1E\\\"; mkfifo /tmp/stderr_1725467701918565000.fifo \\u0026\\u0026 (cat /tmp/stderr_1725467701918565000.fifo; rm /tmp/stderr_1725467701918565000.fifo); ec=$?; echo -ne \\\"\\\\x1EPoseidon exit $ec $(date +%s%3N)\\\\x1E\\\" \\u0026\\u0026 exit $ec\\r\\nuser      1420  0.0  0.0   9972  3476 pts/1    S+   16:35   0:00 /bin/bash -c echo -ne \\\"\\\\x1EPoseidon until $(date +%s%3N)\\\\x1E\\\"; until [ -e /tmp/stderr_1725467701918565000.fifo ]; do sleep 0.01; done; (echo -ne \\\"\\\\x1EPoseidon ps aux $(date +%s%3N)\\\\x1E\\\"; env CODEOCEAN=true /bin/bash -c \\\"ps aux\\\"; ec=$?; echo -ne \\\"\\\\x1EPoseidon exit $ec $(date +%s%3N)\\\\x1E\\\" \\u0026\\u0026 exit $ec) 2\\u003e /tmp/stderr_1725467701918565000.fifo\\r\\nuser      1426  0.0  0.0   9972  1568 pts/0    S+   16:35   0:00 /bin/bash -c echo -ne \\\"\\\\x1EPoseidon mkfifo $(date +%s%3N)\\\\x1E\\\"; mkfifo /tmp/stderr_1725467\"}"
2024-09-04 16:35:02.017006 UTC: Receiving from ws://<IP_ADDRESS>:7200/api/v1/runners/10-110aa742-6adb-11ef-b496-2e62a1fdfe47/websocket?executionID=2440ac07-461b-4452-ae17-375cf3fc7e87: "{\"type\":\"stdout\",\"data\":\"701918565000.fifo \\u0026\\u0026 (cat /tmp/stderr_1725467701918565000.fifo; rm /tmp/stderr_1725467701918565000.fifo); ec=$?; echo -ne \\\"\\\\x1EPoseidon exit $ec $(date +%s%3N)\\\\x1E\\\" \\u0026\\u0026 exit $ec\\r\\nuser      1427  0.0  0.0   8324  1556 pts/0    S+   16:35   0:00 cat /tmp/stderr_1725467701918565000.fifo\\r\\nuser      1428  0.0  0.0   9972  1820 pts/1    S+   16:35   0:00 /bin/bash -c echo -ne \\\"\\\\x1EPoseidon until $(date +%s%3N)\\\\x1E\\\"; until [ -e /tmp/stderr_1725467701918565000.fifo ]; do sleep 0.01; done; (echo -ne \\\"\\\\x1EPoseidon ps aux $(date +%s%3N)\\\\x1E\\\"; env CODEOCEAN=true /bin/bash -c \\\"ps aux\\\"; ec=$?; echo -ne \\\"\\\\x1EPoseidon exit $ec $(date +%s%3N)\\\\x1E\\\" \\u0026\\u0026 exit $ec) 2\\u003e /tmp/stderr_1725467701918565000.fifo\\r\\nuser      1430  0.0  0.0  13544  3996 pts/1    R+   16:35   0:00 ps aux\\r\\n\"}"
2024-09-04 16:35:02.070763 UTC: Receiving from ws://<IP_ADDRESS>:7200/api/v1/runners/10-110aa742-6adb-11ef-b496-2e62a1fdfe47/websocket?executionID=2440ac07-461b-4452-ae17-375cf3fc7e87: "{\"type\":\"exit\",\"data\":0}"
2024-09-04 16:35:02.071505 UTC: Closing connection to ws://<IP_ADDRESS>:7200/api/v1/runners/10-110aa742-6adb-11ef-b496-2e62a1fdfe47/websocket?executionID=2440ac07-461b-4452-ae17-375cf3fc7e87 with status: terminated_by_management

I was not able to reproduce the issue due to the "randomness" of how Docker splits its output. However, due to your great issue description, the cause could be traced down to the regex pattern matching the unprocessed data.

Poseidon is filtering the full output and rather forwarding a single line (in this case the HOSTNAME)

Yeah, we had an issue that the newline characters at the end terminated the match.

The SentryDebugWriter Write method is called "twice", and the call with the full output is somehow lost

Calling Write multiple times is intended, to be precise it should be called thrice - Once for the debug message in the beginning, then recursively for the payload data in the middle, and finally, recursively for the debug message in the end.

Great to hear that the issue allowed you to fix it despite not being able to reproduce it. In my tests, I first reproduced it locally (with Poseidon built from the main branch) and then you PR.
In a CodeOcean Rails console, I ran:
100.times do Runner.for(InternalUser.find(87), 10).execute_command("env", privileged_execution: true) end

This yielded 100 successful executions, but only 99 complete outputs (simple counted by performing a search using cmd + F). With your PR, 1000 out of 1000 executions had a correct output, which I assume is enough testing for the current issue.

Calling Write multiple times is intended, to be precise it should be called thrice - Once for the debug message in the beginning, then recursively for the payload data in the middle, and finally, recursively for the debug message in the end.

That makes sense and -- together with your PR -- answers my questions. For transparency: I just noticed that my original question was not well phrased. In the log, I saw:
time="2024-09-04T17:04:46.094047Z" level=trace msg="SentryDebugWriter Write" data="\x1ePoseidon env<PHONE_NUMBER>085\x1eHOSTNAME=4bf0a8e5dbe4\r\nLANGUAGE=en_US.UTF-8\r\nJAVA_HOME=/usr/lib/jvm/java-8-openjdk-arm64\r\nJUNIT=/usr/java/lib/junit-4.13.2.jar\r\nPWD=/workspace\r\n_=/usr/bin/env\r\nHOME=/workspace\r\nLANG=en_US.UTF-8\r\nCOLUMNS=10000\r\nTERM=ansi\r\nUSER=user\r\nSHLVL=2\r\nANTLR=/usr/java/lib/antlr-4.5.3-complete.jar\r\nCODEOCEAN=true\r\nCLASSPATH=.:/usr/java/lib/hamcrest-3.0.jar:/usr/java/lib/junit-4.13.2.jar:/usr/java/lib/antlr-4.5.3-complete.jar:/usr/java/lib/antlr-java8.jar\r\nHAMCREST=/usr/java/lib/hamcrest-3.0.jar\r\nLC_ALL=en_US.UTF-8\r\nPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\r\nANTLR_JAVA8=/usr/java/lib/antlr-java8.jar\r\nUID=1001\r\nDEBIAN_FRONTEND=teletype\r\n\x1ePoseidon exit 0<PHONE_NUMBER>087\x1e" environment_id=10 package=nomad runner_id=10-84eac7c0-6aca-11ef-be51-2e62a1fdfe47
time="2024-09-04T17:04:46.094094Z" level=trace msg="SentryDebugWriter Write" data="HOSTNAME=4bf0a8e5dbe4\r" environment_id=10 package=nomad runner_id=10-84eac7c0-6aca-11ef-be51-2e62a1fdfe47

Initially I thought that the second call to Write with the shorter output (which I was seeing) was somewhat "overwriting" or taking precedence over the previous one (where all lines are present). However, I now understood that the previous message incorrectly contained the debug log message and the stdout output due to the incorrect splitting. In my tests with your PR, the splitting, I saw a clear separation of the debug message and the user-generated output:
time="2024-09-11T01:03:25.899120Z" level=trace msg="SentryDebugWriter Write" data="\x1ePoseidon env<PHONE_NUMBER>897\x1e" environment_id=29 package=nomad runner_id=29-54c99636-6fc8-11ef-b6aa-2e62a1fdfe47
time="2024-09-11T01:03:25.899829Z" level=trace msg="SentryDebugWriter Write" data="HOSTNAME=cd5475b887f8\r\nLANGUAGE=en_US.UTF-8\r\nPWD=/workspace\r\n_=/usr/bin/env\r\nHOME=/workspace\r\nLANG=en_US.UTF-8\r\nCOLUMNS=10000\r\nPYTHONPATH=:/usr/lib/python3.8:/workspace\r\nTERM=ansi\r\nUSER=user\r\nSHLVL=2\r\nCODEOCEAN=true\r\nLC_ALL=en_US.UTF-8\r\nPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/python3.8\r\nUID=1001\r\nDEBIAN_FRONTEND=teletype\r\n" environment_id=29 package=nomad runner_id=29-54c99636-6fc8-11ef-b6aa-2e62a1fdfe47


toPrecision variant that does not add zeros

It would be nice to have an option to bypass adding zeros here.
What if we have dp = 1e6, and the value passed is just 0? We end up adding 1e6 zeros, which consumes a lot of time and may be pointless for most usage scenarios.
Also, I think that pushing constant value within cycle here (and in some other places in the code) is not the best way of doing this in sense of performance. You should consider rewriting it to something like
      // Append zeros?
      var d = k - x.c.length;
      if (d > 0) x.c = x.c.concat(new Array(d).fill(0));

Also2, why there are two nearly identical files, big.js and big.mjs? It is possible to translate es2015 modules to commonjs (e.g. with babel), so it may be useful to add the translation command to the build script and deal with just one file instead of two.

I am not clear what you mean here. Where is dp from

Uh, I've meant sd, not dp, sorry. sd as an argument of .toPrecision
I was wondering if there is a way to stringify a value with maximum possible precision, but without padding it with 0s. The resulting string should end with the last significant digit if it is possible, or be very long (1e6 decimal places) otherwise.
There are cases when it's known that some number does not contain repetends, but it's exact length in decimal places is not known.
I ended up with guessing the number of decimal places, like this:
function (big) {
    return big.toFixed(big.c.length + Math.abs(big.e))
            .replace(/\.?0+$/, '');
};

I was wondering if there is a way to stringify a value with maximum possible precision, but without padding it with 0s.

What's wrong with toString(), or toExponential() if you don't want trailing integer-part zeros?
If you want to be able to specifiy the maximum number of significant digits you could add the following method,
Big.prototype.precision = function (sd) {
  let r = new Big(this);
  let len = r.c.length;
  if (len > sd) {
    len = sd;
    while (r.c[len - 1] === 0 && len > 1) --len;
    r.c.length = len;
  }
  return r;
};

Then use as
let x = new Big('999.1234000056789');
x.precision(7).toString();    // '999.1234'
x.precision(10).toString();   // '999.1234'
x.precision(12).toString();   // '999.123400005'

Swap toString() for toFixed() if you never want exponential notation.

@MikeMcl thanks! This is exactly what I needed, toString() with configurable sd.

Added a prec method in v6.0.0, to round a Big number to a specified precision.


pgsql: remove unnecessary join

Can you mention in the commit message that it is about GetNotification? Other than that, LGTM.
This should be included in the next point-release.


topic queue consumer eventually receives message from RabbitMQ for incorrect OXP

Hi,
In a scenario with multiple OXPs, depending on the order they are started and also the order in which breakdown connections are created, we can ended up receiving different messages for different SDX-LC. For instance, the amlight SDX-LC can receive message from zaoxi breakdown, then sax receives messages for amlight, and finally zaoxi's SDX-LC receives messages destinated to sax.
It seems like the routing on RabbitMQ is being processed wrongly because of how instantiated the consumer is.
I was able to fix this by using the following change:
diff --git a/swagger_server/messaging/topic_queue_consumer.py b/swagger_server/messaging/topic_queue_consumer.py
index 138e4dd..a683f42 100644
--- a/swagger_server/messaging/topic_queue_consumer.py
+++ b/swagger_server/messaging/topic_queue_consumer.py
@@ -43,7 +43,7 @@ class TopicQueueConsumer(object):
         self.channel = self.connection.channel()
         self.exchange_name = exchange_name

-        self.result = self.channel.queue_declare(queue=SUB_QUEUE)
+        self.result = self.channel.queue_declare(queue="", exclusive=True)
         self._thread_queue = thread_queue

         self.binding_keys = []

Hi @congwang09 here is how to reproduce:

Using Docker create one container for RabbitMQ, another one for the SDX-Controller (producer), and three other ones for the SDX-LC (Consumer) -- see below I'm already configuring some environment variables that will be necessary:

docker run -d --name mq1 -e RABBITMQ_DEFAULT_USER=xpto -e RABBITMQ_DEFAULT_PASS=xpto rabbitmq:latest
docker run -d --name p1 -e MQ_HOST=mq1 -e  MQ_PORT=5672 -e MQ_USER=xpto -e MQ_PASS=xpto --link mq1 python:3.9 tail -f /dev/null
docker run -d --name c1 -e MQ_HOST=mq1 -e MQ_PORT=5672 -e MQ_USER=xpto -e MQ_PASS=xpto -e SUB_TOPIC=amlight -e SUB_QUEUE=connection -e SUB_EXCHANGE=connection --link mq1 python:3.9 tail -f /dev/null
docker run -d --name c2 -e MQ_HOST=mq1 -e MQ_PORT=5672 -e MQ_USER=xpto -e MQ_PASS=xpto -e SUB_TOPIC=sax -e SUB_QUEUE=connection -e SUB_EXCHANGE=connection --link mq1 python:3.9 tail -f /dev/null
docker run -d --name c3 -e MQ_HOST=mq1 -e MQ_PORT=5672 -e MQ_USER=xpto -e MQ_PASS=xpto -e SUB_TOPIC=tenet -e SUB_QUEUE=connection -e SUB_EXCHANGE=connection --link mq1 python:3.9 tail -f /dev/null

Configure the containers c1, c2 and c3 as the SDX-LC receiving MQ messages:

docker exec -it c1 bash
git clone https://github.com/atlanticwave-sdx/sdx-lc
cd /sdx-lc/swagger_server/messaging/
python3 -m pip install requests pika
# get rid of MongoDB stuff we dont need
sed -i 's/self.db_instance/#self.db_instance/g' topic_queue_consumer.py
sed -i 's/from swagger_server.utils.db_utils/#from swagger_server.utils.db_utils/g' topic_queue_consumer.py
# create a executor for the TopicConsumer object, monkey patching the handle_mq_msg() to just print out
cat >test.py <<EOF
import threading
from queue import Queue
from topic_queue_consumer import TopicQueueConsumer
import os

SUB_TOPIC = os.environ.get("SUB_TOPIC")

def my_handle_mq_msg(msg):
    print(f"consumer={SUB_TOPIC} msg={msg}")

thread_queue = Queue()
consumer = TopicQueueConsumer(thread_queue, "connection")
consumer.handle_mq_msg = my_handle_mq_msg
t1 = threading.Thread(target=consumer.start_consumer, args=())
t1.start()
t1.join()
EOF
exit

# TODO: repeat the same steps above for c2 and c3

Now let's configure the producer / SDX-Controller:

docker exec -it p1 bash
git clone https://github.com/atlanticwave-sdx/sdx-controller
cd sdx-controller/
# apply the fixes to allow rabbitmq authentication
curl -LO https://github.com/atlanticwave-sdx/sdx-controller/pull/223.diff
curl -LO https://github.com/atlanticwave-sdx/sdx-controller/pull/250.diff
patch -p1 < 223.diff
patch -p1 < 250.diff
# install dependencies
python3 -m pip install "connexion[swagger-ui] >= 2.14.1" pika
cd /sdx-controller/sdx_controller/messaging/
cat >test.py <<EOF
from topic_queue_producer import TopicQueueProducer

p1 = TopicQueueProducer(5, 'connection', 'amlight'); p1.call('create breakdown at amlight'); p1.stop_keep_alive()
p2 = TopicQueueProducer(5, 'connection', 'sax'); p2.call('create breakdown at sax'); p2.stop_keep_alive()
p3 = TopicQueueProducer(5, 'connection', 'tenet'); p3.call('create breakdown at tenet'); p3.stop_keep_alive()
EOF
exit

Now it is time to run the consumers and the producer to see how it will behave. You will need 4 terminals and we will run the consumers in different orders to see the results:

On Terminal 1: docker exec -it -w /sdx-lc/swagger_server/messaging c3 python3 test.py
On Terminal 2: docker exec -it -w /sdx-lc/swagger_server/messaging c1 python3 test.py
On Terminal 3: docker exec -it -w /sdx-lc/swagger_server/messaging c2 python3 test.py
On Terminal 4: docker exec -it -w /sdx-controller/sdx_controller/messaging p1 python3 test.py

Expected results: each SDX-LC won't print only the message directed to it
Actual result: we see messages directed to wrong containers
If we switch the order in which consumers are started, they result can be different.
Here is my view (in the picture below I executed three times the producer with consumers ordered in c3, c1, c2, then I stopped the consumers and executed in the order c2, c3, c1):

Applying the change suggested on the first message seems to fix this issue:
docker exec -it -w /sdx-lc/swagger_server/messaging c1 sed -i 's/result = self.channel.queue_declare(queue=SUB_QUEUE)/result = self.channel.queue_declare(queue="", exclusive=True)/' topic_queue_consumer.py

docker exec -it -w /sdx-lc/swagger_server/messaging c2 sed -i 's/result = self.channel.queue_declare(queue=SUB_QUEUE)/result = self.channel.queue_declare(queue="", exclusive=True)/' topic_queue_consumer.py

docker exec -it -w /sdx-lc/swagger_server/messaging c3 sed -i 's/result = self.channel.queue_declare(queue=SUB_QUEUE)/result = self.channel.queue_declare(queue="", exclusive=True)/' topic_queue_consumer.py

my test.py above was slightly improved to handle the KeyboarInterrupt exception from previous execution

Hi @congwang09 just to document here in the issue what I mentioned yesterday on slack:
just to add into the discussion,  the rabbitmq tutorial also recommends using the empty queue name and then later on bind to the unique queue when using Routing. My understanding is that the exchange plays the main role of separating the context here (the queue is interested in messages from this exchange), while the routing_key will separate individual consumers and their specific interest.
See that by using the empty queue name on queue_declare  call we will instruct rabbitmq server to create a unique name for that consumer, and then later on we bind the exchange, unique queue and (most important) routing key all together (this is already done here).


Terraform 0.14 upgrade

what

Upgrade to support Terraform 0.14 and bring up to current Cloud Posse standard

why

Support Terraform 0.14

/test all

/test all

Hi awesome people,
When are we merging this PR so that we can upgrade to terraform 0.14 :) ?

Hi awesome people,
When are we merging this PR so that we can upgrade to terraform 0.14 :) ?


Endless waiting for the consumer when trying to read all sent messages in the topic.

Hello! Thanks for your work.
I have a question.
I am using your example https://github.com/mostafa/xk6-kafka/blob/main/scripts/test_consumer_group.js
and ran into the problem that if you try to subtract all 300 messages, then the consumer waits indefinitely. And this happens somewhere in 3 out of 5 attempts.
Full code
/*
This is a k6 test script that imports the xk6-kafka and
tests Kafka with a 200 JSON messages per iteration.
*/

import { check } from "k6";
// import * as kafka from "k6/x/kafka";
import {
  Writer,
  Reader,
  Connection,
  SchemaRegistry,
  CODEC_SNAPPY,
  SCHEMA_TYPE_JSON,
} from "k6/x/kafka"; // import kafka extension

// Prints module-level constants
// console.log(kafka);

const brokers = ["localhost:9092"];
const topic = "test";
const groupID = "my-group";

const writer = new Writer({
  brokers: brokers,
  topic: topic,
  compression: CODEC_SNAPPY,
});
const reader = new Reader({
  brokers: brokers,
  groupID: groupID,
  groupTopics: [topic],
});
const connection = new Connection({
  address: brokers[0],
});
const schemaRegistry = new SchemaRegistry();

if (__VU == 0) {
  connection.createTopic({
    topic: topic,
    numPartitions: 3,
    replicationFactor: 1,
    configEntries: [
      {
        configName: "compression.type",
        configValue: CODEC_SNAPPY,
      },
    ],
  });
}

export const options = {
  thresholds: {
    // Base thresholds to see if the writer or reader is working
    kafka_writer_error_count: ["count == 0"],
    kafka_reader_error_count: ["count == 0"],
  },
};

export default function () {
  let messages = [];
  for (let i = 0; i < 100; i++) {
    for (let partition = 0; partition < 3; partition++) {
      messages.push({
        // The data type of the key is JSON
        key: schemaRegistry.serialize({
          data: {
            key: "value",
          },
          schemaType: SCHEMA_TYPE_JSON,
        }),
        // The data type of the value is JSON
        value: schemaRegistry.serialize({
          data: {
            key: "value",
          },
          schemaType: SCHEMA_TYPE_JSON,
        }),
        parition: partition,
      });
    }
  }

  writer.produce({ messages: messages });

  // Read one message only
  messages = reader.consume({ limit: 300 });

  check(messages, {
    "10 messages is received": (messages) => messages.length == 300,
  });

  check(messages[0], {
    "Topic equals to xk6_kafka_consumer_group_topic": (msg) =>
      msg["topic"] == topic,
    "Key contains key/value and is JSON": (msg) =>
      schemaRegistry.deserialize({
        data: msg.key,
        schemaType: SCHEMA_TYPE_JSON,
      }).key == "value",
    "Value contains key/value and is JSON": (msg) =>
      typeof schemaRegistry.deserialize({
        data: msg.value,
        schemaType: SCHEMA_TYPE_JSON,
      }) == "object" &&
      schemaRegistry.deserialize({
        data: msg.value,
        schemaType: SCHEMA_TYPE_JSON,
      }).key == "value",
  });
}

export function teardown(data) {
	  if (__VU == 0) {
    // Delete the topic
    connection.deleteTopic(topic);
  }
  writer.close();
  reader.close();
  connection.close();
}

logs
time="2023-03-13T09:02:59+03:00" level=info msg="entering loop for consumer group, my-group\n"
time="2023-03-13T09:02:59+03:00" level=info msg="entering loop for consumer group, my-group\n"
time="2023-03-13T09:03:04+03:00" level=info msg="joined group my-group as member k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-de501498-6407-4c63-8e28-15f52cd51a41 in generation 10"tions
time="2023-03-13T09:03:04+03:00" level=info msg="selected as leader for group, my-group\n" VU
time="2023-03-13T09:03:04+03:00" level=info msg="joined group my-group as member k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-87f19579-d19e-4e5c-aed6-b3b2bc286b5f in generation 10"
time="2023-03-13T09:03:04+03:00" level=info msg="joinGroup succeeded for response, my-group.  generationID=10, memberID=k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-87f19579-d19e-4e5c-aed6-b3b2bc286b5f"
time="2023-03-13T09:03:04+03:00" level=info msg="Joined group my-group as member k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-87f19579-d19e-4e5c-aed6-b3b2bc286b5f in generation 10"
time="2023-03-13T09:03:04+03:00" level=info msg="using 'range' balancer to assign group, my-group"
time="2023-03-13T09:03:04+03:00" level=info msg="found member: k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-de501498-6407-4c63-8e28-15f52cd51a41/[]byte(nil)"
time="2023-03-13T09:03:04+03:00" level=info msg="found member: k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-87f19579-d19e-4e5c-aed6-b3b2bc286b5f/[]byte(nil)"
time="2023-03-13T09:03:04+03:00" level=info msg="found topic/partition: test/0"
time="2023-03-13T09:03:04+03:00" level=info msg="assigned member/topic/partitions k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-de501498-6407-4c63-8e28-15f52cd51a41/test/[0]"
time="2023-03-13T09:03:04+03:00" level=info msg="joinGroup succeeded for response, my-group.  generationID=10, memberID=k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-de501498-6407-4c63-8e28-15f52cd51a41"
time="2023-03-13T09:03:04+03:00" level=info msg="Joined group my-group as member k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-de501498-6407-4c63-8e28-15f52cd51a41 in generation 10"
time="2023-03-13T09:03:04+03:00" level=info msg="Syncing 2 assignments for generation 10 as member k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-de501498-6407-4c63-8e28-15f52cd51a41"
time="2023-03-13T09:03:04+03:00" level=info msg="received empty assignments for group, my-group as member k6@QPRM-FVSC1F3 (github.com/segmentio/kafka-go)-87f19579-d19e-4e5c-aed6-b3b2bc286b5f for generation 10"
time="2023-03-13T09:03:04+03:00" level=info msg="sync group finished for group, my-group"
time="2023-03-13T09:03:04+03:00" level=info msg="sync group finished for group, my-group"

running (00m05.0s), 1/1 VUs, 0 complete and 0 interrupted iterations
default   [--------------------------------------time="2023-03-13T09:03:04+03:00" level=info msg="started heartbeat for group, my-group [3s]"
time="2023-03-13T09:03:04+03:00" level=info msg="subscribed to topics and partitions: map[]"
] 1 VUs  00m05.0s/10m0s  0/1 iters, 1 per VUtime="2023-03-13T09:03:04+03:00" level=info msg="started heartbeat for group, my-group [3s]"

time="2023-03-13T09:03:04+03:00" level=info msg="initializing kafka reader for partition 0 of test starting at offset first offset"
time="2023-03-13T09:03:04+03:00" level=info msg="subscribed to topics and partitions: map[{topic:test partition:0}:-2]"

I have tried the rebalanceTimeout and sessionTimeout settings but without success.
Could you suggest what I am doing wrong?

@bloodgang94
I ran your exact script (with no modifications) for a minute with 50 VUs and this is the result:

View results
./k6 run --vus 50 --duration 60s /mnt/c/Users/Mostafa\ Moradian/Desktop/test-kafka.js

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: /mnt/c/Users/Mostafa Moradian/Desktop/test-kafka.js
     output: engine

  scenarios: (100.00%) 1 scenario, 50 max VUs, 1m30s max duration (incl. graceful stop):
           * default: 50 looping VUs for 1m0s (gracefulStop: 30s)

ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."

     ✓ 10 messages is received
     ✓ Topic equals to xk6_kafka_consumer_group_topic
     ✓ Key contains key/value and is JSON
     ✓ Value contains key/value and is JSON

     █ teardown

     checks.........................: 100.00% ✓ 1764        ✗ 0
     data_received..................: 0 B     0 B/s
     data_sent......................: 0 B     0 B/s
     iteration_duration.............: avg=137.67ms min=122.3ms med=131.32ms max=941ms    p(90)=147.73ms p(95)=155.48ms
     iterations.....................: 441     4.857311/s
     kafka_reader_dial_count........: 1       0.011014/s
     kafka_reader_dial_seconds......: avg=4.13µs   min=0s      med=0s       max=1.82ms   p(90)=0s       p(95)=0s
   ✓ kafka_reader_error_count.......: 0       0/s
     kafka_reader_fetch_bytes.......: 7.5 MB  83 kB/s
     kafka_reader_fetch_bytes_max...: 1000000 min=1000000   max=1000000
     kafka_reader_fetch_bytes_min...: 1       min=1         max=1
     kafka_reader_fetch_size........: 124995  1376.733793/s
     kafka_reader_fetch_wait_max....: 10s     min=10s       max=10s
     kafka_reader_fetches_count.....: 16      0.176229/s
     kafka_reader_lag...............: 7400    min=6500      max=14600
     kafka_reader_message_bytes.....: 4.0 MB  44 kB/s
     kafka_reader_message_count.....: 132401  1458.30578/s
     kafka_reader_offset............: 132400  min=400       max=132400
     kafka_reader_queue_capacity....: 100     min=100       max=100
     kafka_reader_queue_length......: 100     min=99        max=100
     kafka_reader_read_seconds......: avg=127.24ms min=0s      med=0s       max=4.01s    p(90)=0s       p(95)=0s
     kafka_reader_rebalance_count...: 1       0.011014/s
     kafka_reader_timeouts_count....: 0       0/s
     kafka_reader_wait_seconds......: avg=32.96µs  min=0s      med=0s       max=1.54ms   p(90)=0s       p(95)=0s
     kafka_writer_acks_required.....: 0       min=0         max=0
     kafka_writer_async.............: 0.00%   ✓ 0           ✗ 490
     kafka_writer_attempts_max......: 0       min=0         max=0
     kafka_writer_batch_bytes.......: 26 kB   281 B/s
     kafka_writer_batch_max.........: 1       min=1         max=1
     kafka_writer_batch_size........: 490     5.397012/s
     kafka_writer_batch_timeout.....: 0s      min=0s        max=0s
   ✓ kafka_writer_error_count.......: 0       0/s
     kafka_writer_message_bytes.....: 7.6 MB  84 kB/s
     kafka_writer_message_count.....: 147000  1619.103705/s
     kafka_writer_read_timeout......: 0s      min=0s        max=0s
     kafka_writer_retries_count.....: 0       0/s
     kafka_writer_wait_seconds......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s
     kafka_writer_write_count.......: 147000  1619.103705/s
     kafka_writer_write_seconds.....: avg=30.37µs  min=12.5µs  med=23.17µs  max=163.86µs p(90)=50.43µs  p(95)=78.4µs
     kafka_writer_write_timeout.....: 0s      min=0s        max=0s
     vus............................: 0       min=0         max=50
     vus_max........................: 50      min=50        max=50

running (1m30.8s), 00/50 VUs, 441 complete and 49 interrupted iterations
default ✓ [======================================] 50 VUs  1m0s

I am aware of the reader errors, as you see in the logs. The writer generated 147k messages and the reader consumed 132k. The number of errors and your limit of 300 correlates to the difference between the messages produced vs. consumed. I tweaked the reader config and this is the best result I can get:
import {
  Writer,
  Reader,
  Connection,
  SchemaRegistry,
  CODEC_SNAPPY,
  SCHEMA_TYPE_JSON,
  SECOND,
} from "k6/x/kafka"

const reader = new Reader({
  brokers: brokers,
  groupID: groupID,
  groupTopics: [topic],
  queueCapacity: 300,
  readBatchTimeout: 20 * SECOND,
  maxBytes: 10000000, // 10 MB
  readLagInterval: 10 * SECOND,
})

View results
 ./k6 run --vus 50 --duration 60s /mnt/c/Users/Mostafa\ Moradian/Desktop/test-kafka.js

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: /mnt/c/Users/Mostafa Moradian/Desktop/test-kafka.js
     output: engine

  scenarios: (100.00%) 1 scenario, 50 max VUs, 1m30s max duration (incl. graceful stop):
           * default: 50 looping VUs for 1m0s (gracefulStop: 30s)

ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."
ERRO[0090] Unable to read messages.                      error="Unable to read messages."

     ✓ 10 messages is received
     ✓ Topic equals to xk6_kafka_consumer_group_topic
     ✓ Key contains key/value and is JSON
     ✓ Value contains key/value and is JSON

     █ teardown

     checks.........................: 100.00%  ✓ 1808        ✗ 0
     data_received..................: 0 B      0 B/s
     data_sent......................: 0 B      0 B/s
     iteration_duration.............: avg=134.43ms min=122.3ms med=128.82ms max=998.19ms p(90)=138.76ms p(95)=145.67ms
     iterations.....................: 452      4.971897/s
     kafka_reader_dial_count........: 1        0.011/s
     kafka_reader_dial_seconds......: avg=2.98µs   min=0s      med=0s       max=1.35ms   p(90)=0s       p(95)=0s
   ✓ kafka_reader_error_count.......: 0        0/s
     kafka_reader_fetch_bytes.......: 7.8 MB   86 kB/s
     kafka_reader_fetch_bytes_max...: 10000000 min=10000000  max=10000000
     kafka_reader_fetch_bytes_min...: 1        min=1         max=1
     kafka_reader_fetch_size........: 130200   1432.170354/s
     kafka_reader_fetch_wait_max....: 10s      min=10s       max=10s
     kafka_reader_fetches_count.....: 10       0.109998/s
     kafka_reader_lag...............: 8700     min=0         max=14400
     kafka_reader_message_bytes.....: 4.1 MB   45 kB/s
     kafka_reader_message_count.....: 135901   1494.880056/s
     kafka_reader_offset............: 135900   min=600       max=135900
     kafka_reader_queue_capacity....: 300      min=300       max=300
     kafka_reader_queue_length......: 300      min=300       max=300
     kafka_reader_read_seconds......: avg=125.21ms min=0s      med=0s       max=6.52s    p(90)=0s       p(95)=0s
     kafka_reader_rebalance_count...: 1        0.011/s
     kafka_reader_timeouts_count....: 0        0/s
     kafka_reader_wait_seconds......: avg=17.22µs  min=0s      med=0s       max=1.14ms   p(90)=0s       p(95)=0s
     kafka_writer_acks_required.....: 0        min=0         max=0
     kafka_writer_async.............: 0.00%    ✓ 0           ✗ 501
     kafka_writer_attempts_max......: 0        min=0         max=0
     kafka_writer_batch_bytes.......: 26 kB    287 B/s
     kafka_writer_batch_max.........: 1        min=1         max=1
     kafka_writer_batch_size........: 501      5.510886/s
     kafka_writer_batch_timeout.....: 0s       min=0s        max=0s
   ✓ kafka_writer_error_count.......: 0        0/s
     kafka_writer_message_bytes.....: 7.8 MB   86 kB/s
     kafka_writer_message_count.....: 150300   1653.265778/s
     kafka_writer_read_timeout......: 0s       min=0s        max=0s
     kafka_writer_retries_count.....: 0        0/s
     kafka_writer_wait_seconds......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s
     kafka_writer_write_count.......: 150300   1653.265778/s
     kafka_writer_write_seconds.....: avg=34.71µs  min=11.38µs med=21.61µs  max=406.52µs p(90)=45.03µs  p(95)=130.23µs
     kafka_writer_write_timeout.....: 0s       min=0s        max=0s
     vus............................: 0        min=0         max=50
     vus_max........................: 50       min=50        max=50

running (1m30.9s), 00/50 VUs, 452 complete and 49 interrupted iterations
default ✓ [======================================] 50 VUs  1m0s

I'll close this due to inactivity. Feel free to re-open it if you still have the issue.

@mostafa
After making changes also getting Unable to read messages error right
Then what does it mean?
How to avoid this error?

Even in your result also writer message count and reader message count are different


Disable Systemd handling of XF86PowerOff events

Adds a logind.conf.d folder with a config file that ignores power button pressing. Prerequisite for https://github.com/ChimeraOS/steamos-compositor-plus/pull/48 to be enabled.

Branch has some issues with previous PR. Reopening.


Sass css

included sass files for the styles and compiled them into css

Features approved!
*Insert gif here <------
Keep the great and hard work!
*Insert another motivating gif here <-------


adminusername-should-not-be-literal should give guidance on how to fix

It was a comment from @uchenkadicode.  Uchi, what did you want changed?
Example error:
When setting an adminUserName property, don't use a literal value. Found literal string value "adminUsername"bicep corehttps://aka.ms/bicep/linter/adminusername-should-not-be-literal

The docs are explicit about the solution (https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/linter-rule-admin-username-should-not-be-literal), so I'm not sure any changes are needed.

@StephenWeatherford, is there a place I can review the error code? I think this would be relatively simple code to check in the future.
I would like the message to say something like
"
Property "adminUserName" should not use a literal value. Use a parameter for adminUserName instead. bicep core(https://...)
param adminUsername string = defaultValue
"

Also as a note, I do not like using the phrase "don't do this" in an error message. It's a little harsh on a user that made a small mistake. I think we can still get the point across by saying "should not". This goes for any of our messages that say "don't do this" don't do that.

All the linter strings are here: https://github.com/Azure/bicep/blob/main/src/Bicep.Core/CoreResources.resx
Feel free to push a PR, but I'll need to help it through test updates.


[Request] Gtk::Application example

Would be nice have a example of usage Gtk::Application in rust, that shows what's recommended way to do this.

Here is a small example https://github.com/chergert/hello/blob/master/rust/src/main.rs

This is not really closed yet, an example in this repository would still be nice to have

Oh well, I filed the more general https://github.com/gtk-rs/examples/issues/140 for this now :)


Google CLA failure on all Pull Requests

Issue source:
https://github.com/GoogleCloudPlatform/generative-ai/commit/c2b2a75031646167fb04cec187504e41dec4e710
Commit message includes:
---------

Co-authored-by: Megha Agarwal
Co-authored-by: Polong Lin<EMAIL_ADDRESS>
Co-author Megha Agarwal doesn't have an email added (should be<EMAIL_ADDRESS>)
However, updating this commit message is a breaking change that requires significant effort to rebase all commits after this.

@meredithslota We had discussed this last year, is there anything that could be done to change the specific commit message, but not require a full rebasing?

Followed up on internal bug.


Graceful shutdown by properly handling SIGTERM

From https://github.com/uber/jaeger/pull/280 :

What I think should happen is you tell the health endpoint to start returning Unavailable and then sleep for a couple seconds, and the exit. The main tchannel / http servers will remain in operation during the sleep and allowed to finish service the in-flight requests, while the routing mesh might hopefully detect health check down and stop sending traffic.

#2076 still pending.