Add policies to validate GitHub certificates

https://issues.redhat.com/browse/HACBS-2083
To try it out, create the policy below.
policy.yaml
sources:
  - name: Default
    policy:
     # Using my fork while the change is not yet merged
      - github.com/lcarva/ec-policies//policy/lib?ref=HACBS-2083
      - github.com/lcarva/ec-policies//policy/release?ref=HACBS-2083
    data:
      - github.com/lcarva/ec-policies//data?ref=HACBS-2083
    ruleData:
      allowed_gh_workflow_repos: [lcarva/festoji]
      allowed_gh_workflow_refs: [refs/heads/master]
      allowed_gh_workflow_names: [Package]
      allowed_gh_workflow_triggers: [push]

configuration:
  include:
    - '@slsa1'
    - '@slsa2'
    - '@slsa3'
    - github_certificate

And run ec against an image built on GitHub:
EC_EXPERIMENTAL=1 ec validate image \
   --policy policy.yaml \
   --image quay.io/lucarval/festoji:latest \
   --output yaml \
   --certificate-identity-regexp '^https://github\.com' \
   --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
   --rekor-url 'https://rekor.sigstore.dev' \
   --info --show-successes

Click to see the output
components:
- containerImage: quay.io/lucarval/festoji@sha256:6a6b100a8b143683f654fc30c4f6e40fcc7d18cf7db050792de6ad8a25e6eb33
  name: Unnamed
  signatures:
  - certificate: |
      -----BEGIN CERTIFICATE-----
      MIIGgjCCBgigAwIBAgIUQNGRo7U3odD/NCO2AUOUZEHrrV4wCgYIKoZIzj0EAwMw
      NzEVMBMGA1UEChMMc2lnc3RvcmUuZGV2MR4wHAYDVQQDExVzaWdzdG9yZS1pbnRl
      cm1lZGlhdGUwHhcNMjMwNjIzMjAwODM4WhcNMjMwNjIzMjAxODM4WjAAMFkwEwYH
      KoZIzj0CAQYIKoZIzj0DAQcDQgAEF9tc9f4G+uPc23aEzS519jAjnzavr4wL0Cx5
      Zs4Khd9kcHONFZE1JFHmUICjP6BafRZ3cWz8yv35paQVSV+DVKOCBScwggUjMA4G
      A1UdDwEB/wQEAwIHgDATBgNVHSUEDDAKBggrBgEFBQcDAzAdBgNVHQ4EFgQUnDhg
      2f/e4XFEns+m+PltKUbHHf4wHwYDVR0jBBgwFoAU39Ppz1YkEZb5qNjpKFWixi4Y
      ZD8wYAYDVR0RAQH/BFYwVIZSaHR0cHM6Ly9naXRodWIuY29tL2xjYXJ2YS9mZXN0
      b2ppLy5naXRodWIvd29ya2Zsb3dzL3BhY2thZ2UueWFtbEByZWZzL2hlYWRzL21h
      c3RlcjA5BgorBgEEAYO/MAEBBCtodHRwczovL3Rva2VuLmFjdGlvbnMuZ2l0aHVi
      dXNlcmNvbnRlbnQuY29tMBIGCisGAQQBg78wAQIEBHB1c2gwNgYKKwYBBAGDvzAB
      AwQoODQ4ZWRjNDUyY2NiYzZkNDJlYzU2YzI4MDdlZWYyZjQ5ZTc1NGM1ZTAVBgor
      BgEEAYO/MAEEBAdQYWNrYWdlMBwGCisGAQQBg78wAQUEDmxjYXJ2YS9mZXN0b2pp
      MB8GCisGAQQBg78wAQYEEXJlZnMvaGVhZHMvbWFzdGVyMDsGCisGAQQBg78wAQgE
      LQwraHR0cHM6Ly90b2tlbi5hY3Rpb25zLmdpdGh1YnVzZXJjb250ZW50LmNvbTBi
      BgorBgEEAYO/MAEJBFQMUmh0dHBzOi8vZ2l0aHViLmNvbS9sY2FydmEvZmVzdG9q
      aS8uZ2l0aHViL3dvcmtmbG93cy9wYWNrYWdlLnlhbWxAcmVmcy9oZWFkcy9tYXN0
      ZXIwOAYKKwYBBAGDvzABCgQqDCg4NDhlZGM0NTJjY2JjNmQ0MmVjNTZjMjgwN2Vl
      ZjJmNDllNzU0YzVlMB0GCisGAQQBg78wAQsEDwwNZ2l0aHViLWhvc3RlZDAxBgor
      BgEEAYO/MAEMBCMMIWh0dHBzOi8vZ2l0aHViLmNvbS9sY2FydmEvZmVzdG9qaTA4
      BgorBgEEAYO/MAENBCoMKDg0OGVkYzQ1MmNjYmM2ZDQyZWM1NmMyODA3ZWVmMmY0
      OWU3NTRjNWUwIQYKKwYBBAGDvzABDgQTDBFyZWZzL2hlYWRzL21hc3RlcjAZBgor
      BgEEAYO/MAEPBAsMCTE1OTA2OTgzMjApBgorBgEEAYO/MAEQBBsMGWh0dHBzOi8v
      Z2l0aHViLmNvbS9sY2FydmEwFwYKKwYBBAGDvzABEQQJDAc1MjcyOTMxMGIGCisG
      AQQBg78wARIEVAxSaHR0cHM6Ly9naXRodWIuY29tL2xjYXJ2YS9mZXN0b2ppLy5n
      aXRodWIvd29ya2Zsb3dzL3BhY2thZ2UueWFtbEByZWZzL2hlYWRzL21hc3RlcjA4
      BgorBgEEAYO/MAETBCoMKDg0OGVkYzQ1MmNjYmM2ZDQyZWM1NmMyODA3ZWVmMmY0
      OWU3NTRjNWUwFAYKKwYBBAGDvzABFAQGDARwdXNoMFQGCisGAQQBg78wARUERgxE
      aHR0cHM6Ly9naXRodWIuY29tL2xjYXJ2YS9mZXN0b2ppL2FjdGlvbnMvcnVucy81
      MzYwMTI1NjEzL2F0dGVtcHRzLzEwgYkGCisGAQQB1nkCBAIEewR5AHcAdQDdPTBq
      xscRMmMZHhyZZzcCokpeuN48rf+HinKALynujgAAAYjp34D6AAAEAwBGMEQCIEDf
      e5O+p+0QdfRbRY4U5hJG+REG3Xxci78SBp8iuJEpAiAI6in8wxrfiC8reu0+EoFc
      wX2Ep4RzIYkAy+p2Ga6JvTAKBggqhkjOPQQDAwNoADBlAjB+CXmTANUemgjXL2/X
      nVIP9B+/02qr8N3kBIPV91VvuCbSMv0mqFImYX+cRxsuVtYCMQDd2NVxH0x5ErBU
      s/UT5EA4t34N1UcRRHfF3YPLPzIvgEYdg0sn3qmgABlPCr1BOkY=
      -----END CERTIFICATE-----
    chain:
    - |
      -----BEGIN CERTIFICATE-----
      MIICGjCCAaGgAwIBAgIUALnViVfnU0brJasmRkHrn/UnfaQwCgYIKoZIzj0EAwMw
      KjEVMBMGA1UEChMMc2lnc3RvcmUuZGV2MREwDwYDVQQDEwhzaWdzdG9yZTAeFw0y
      MjA0MTMyMDA2MTVaFw0zMTEwMDUxMzU2NThaMDcxFTATBgNVBAoTDHNpZ3N0b3Jl
      LmRldjEeMBwGA1UEAxMVc2lnc3RvcmUtaW50ZXJtZWRpYXRlMHYwEAYHKoZIzj0C
      AQYFK4EEACIDYgAE8RVS/ysH+NOvuDZyPIZtilgUF9NlarYpAd9HP1vBBH1U5CV7
      7LSS7s0ZiH4nE7Hv7ptS6LvvR/STk798LVgMzLlJ4HeIfF3tHSaexLcYpSASr1kS
      0N/RgBJz/9jWCiXno3sweTAOBgNVHQ8BAf8EBAMCAQYwEwYDVR0lBAwwCgYIKwYB
      BQUHAwMwEgYDVR0TAQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQU39Ppz1YkEZb5qNjp
      KFWixi4YZD8wHwYDVR0jBBgwFoAUWMAeX5FFpWapesyQoZMi0CrFxfowCgYIKoZI
      zj0EAwMDZwAwZAIwPCsQK4DYiZYDPIaDi5HFKnfxXx6ASSVmERfsynYBiX2X6SJR
      nZU84/9DZdnFvvxmAjBOt6QpBlc4J/0DxvkTCqpclvziL6BCCPnjdlIB3Pu3BxsP
      mygUY7Ii2zbdCdliiow=
      -----END CERTIFICATE-----
    - |
      -----BEGIN CERTIFICATE-----
      MIIB9zCCAXygAwIBAgIUALZNAPFdxHPwjeDloDwyYChAO/4wCgYIKoZIzj0EAwMw
      KjEVMBMGA1UEChMMc2lnc3RvcmUuZGV2MREwDwYDVQQDEwhzaWdzdG9yZTAeFw0y
      MTEwMDcxMzU2NTlaFw0zMTEwMDUxMzU2NThaMCoxFTATBgNVBAoTDHNpZ3N0b3Jl
      LmRldjERMA8GA1UEAxMIc2lnc3RvcmUwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAAT7
      XeFT4rb3PQGwS4IajtLk3/OlnpgangaBclYpsYBr5i+4ynB07ceb3LP0OIOZdxex
      X69c5iVuyJRQ+Hz05yi+UF3uBWAlHpiS5sh0+H2GHE7SXrk1EC5m1Tr19L9gg92j
      YzBhMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRY
      wB5fkUWlZql6zJChkyLQKsXF+jAfBgNVHSMEGDAWgBRYwB5fkUWlZql6zJChkyLQ
      KsXF+jAKBggqhkjOPQQDAwNpADBmAjEAj1nHeXZp+13NWBNa+EDsDP8G1WWg1tCM
      WP/WHPqpaVo0jhsweNFZgSs0eE7wYI4qAjEA2WB9ot98sIkoF3vZYdd3/VtWB5b9
      TNMea7Ix/stJ5TfcLLeABLE4BNJOsQ4vnBHJ
      -----END CERTIFICATE-----
    keyid: 9c3860d9ffdee171449ecfa6f8f96d2946c71dfe
    metadata:
      Fulcio Build Config Digest: 848edc452ccbc6d42ec56c2807eef2f49e754c5e
      Fulcio Build Config URI: https://github.com/lcarva/festoji/.github/workflows/package.yaml@refs/heads/master
      Fulcio Build Signer Digest: 848edc452ccbc6d42ec56c2807eef2f49e754c5e
      Fulcio Build Signer URI: https://github.com/lcarva/festoji/.github/workflows/package.yaml@refs/heads/master
      Fulcio Build Trigger: push
      Fulcio GitHub Workflow Name: Package
      Fulcio GitHub Workflow Ref: refs/heads/master
      Fulcio GitHub Workflow Repository: lcarva/festoji
      Fulcio GitHub Workflow SHA: 848edc452ccbc6d42ec56c2807eef2f49e754c5e
      Fulcio GitHub Workflow Trigger: push
      Fulcio Issuer: https://token.actions.githubusercontent.com
      Fulcio Issuer (V2): https://token.actions.githubusercontent.com
      Fulcio Run Invocation URI: https://github.com/lcarva/festoji/actions/runs/5360125613/attempts/1
      Fulcio Runner Environment: github-hosted
      Fulcio Source Repository Digest: 848edc452ccbc6d42ec56c2807eef2f49e754c5e
      Fulcio Source Repository Identifier: "159069832"
      Fulcio Source Repository Owner Identifier: "5272931"
      Fulcio Source Repository Owner URI: https://github.com/lcarva
      Fulcio Source Repository Ref: refs/heads/master
      Fulcio Source Repository URI: https://github.com/lcarva/festoji
      Issuer: CN=sigstore-intermediate,O=sigstore.dev
      Not After: "2023-06-23T20:18:38Z"
      Not Before: "2023-06-23T20:08:38Z"
      Serial Number: 40d191a3b537a1d0ff3423b60143946441ebad5e
      Subject Alternative Name: URIs:https://github.com/lcarva/festoji/.github/workflows/package.yaml@refs/heads/master
    sig: MEYCIQC4OxPKcp/1xNzKxb4e7klDW96N6GmSh2lySG/WUSuSeQIhAK6PhU8UeyEdLNUzBTH+Jw02QdRfwHvXpOVrN4oWEPJ3
  success: true
  successes:
  - metadata:
      code: builtin.attestation.signature_check
      title: Attestation signature check passed
    msg: Pass
  - metadata:
      code: builtin.attestation.syntax_check
      title: Attestation syntax check passed
    msg: Pass
  - metadata:
      code: builtin.image.signature_check
      title: Image signature check passed
    msg: Pass
  - metadata:
      code: github_certificate.gh_workflow_extensions
      description: Check if the image signature certificate contains the expected
        GitHub extensions. These are the extensions that represent the GitHub workflow
        trigger, sha, name, repository, and ref.
      title: GitHub Workflow Certificate Extensions
    msg: Pass
  - metadata:
      code: github_certificate.gh_workflow_name
      description: Check if the value of the GitHub Workflow Name extension in the
        image signature certificate matches one of the allowed values. Use the rule
        data key "allowed_gh_workflow_names" to specify the list of allowed values.
        An empty allow list, which is the default value, causes this check to succeeded.
      title: GitHub Workflow Name
    msg: Pass
  - metadata:
      code: github_certificate.gh_workflow_ref
      description: Check if the value of the GitHub Workflow Ref extension in the
        image signature certificate matches one of the allowed values. Use the rule
        data key "allowed_gh_workflow_refs" to specify the list of allowed values.
        An empty allow list, which is the default value, causes this check to succeeded.
      title: GitHub Workflow Repository
    msg: Pass
  - metadata:
      code: github_certificate.gh_workflow_repository
      description: Check if the value of the GitHub Workflow Repository extension
        in the image signature certificate matches one of the allowed values. Use
        the rule data key "allowed_gh_workflow_repos" to specify the list of allowed
        values. An empty allow list, which is the default value, causes this check
        to succeeded.
      title: GitHub Workflow Repository
    msg: Pass
  - metadata:
      code: github_certificate.gh_workflow_trigger
      description: Check if the value of the GitHub Workflow Trigger extension in
        the image signature certificate matches one of the allowed values. Use the
        rule data key "allowed_gh_workflow_triggers" to specify the list of allowed
        values. An empty allow list, which is the default value, causes this check
        to succeeded.
      title: GitHub Workflow Trigger
    msg: Pass
  - metadata:
      code: slsa_build_build_service.slsa_builder_id_accepted
      collections:
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: The attestation attribute predicate.builder.id is set to one of
        the values in the allowed_builder_ids rule data, e.g. "https://tekton.dev/chains/v2".
      title: SLSA Builder ID is known and accepted
    msg: Pass
  - metadata:
      code: slsa_build_build_service.slsa_builder_id_found
      collections:
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: The attestation attribute predicate.builder.id is set.
      title: SLSA Builder ID found
    msg: Pass
  - metadata:
      code: slsa_build_scripted_build.build_script_used
      collections:
      - slsa1
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: The attestation attribute predicate.buildConfig.tasks.steps is
        not empty of the pipeline task responsible for building the image.
      title: Build task contains steps
    msg: Pass
  - metadata:
      code: slsa_build_scripted_build.build_task_image_results_found
      collections:
      - slsa1
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: The attestations must contain a build task with the expected IMAGE_DIGEST
        and IMAGE_URL results.
      title: Build task set image digest and url task results
    msg: Pass
  - metadata:
      code: slsa_build_scripted_build.subject_build_task_matches
      collections:
      - slsa1
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: The subject of the attestations must match the IMAGE_DIGEST and
        IMAGE_URL values from the build task.
      title: Provenance subject matches build task image result
    msg: Pass
  - metadata:
      code: slsa_provenance_available.attestation_predicate_type_accepted
      collections:
      - minimal
      - slsa1
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: The predicateType field of the attestation must indicate the in-toto
        SLSA Provenance format was used to attest the PipelineRun.
      title: Expected attestation predicate type found
    msg: Pass
  - metadata:
      code: slsa_source_version_controlled.materials_format_okay
      collections:
      - minimal
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: 'At least one entry in the predicate.materials array of the attestation
        contains the expected attributes: uri and digest.sha1.'
      title: Materials have uri and digest
    msg: Pass
  - metadata:
      code: slsa_source_version_controlled.materials_include_git_sha
      collections:
      - minimal
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: Each entry in the predicate.materials array of the attestation
        includes a SHA1 digest which corresponds to a git commit.
      title: Materials include git commit shas
    msg: Pass
  - metadata:
      code: slsa_source_version_controlled.materials_uri_is_git_repo
      collections:
      - minimal
      - slsa2
      - slsa3
      depends_on:
      - attestation_type.known_attestation_type
      description: Each entry in the predicate.materials array of the attestation
        uses a git URI.
      title: Material uri is a git repo
    msg: Pass
ec-version: v0.1.1626-0058e37
effective-time: "2023-06-29T20:57:46.463658518Z"
key: ""
policy:
  configuration:
    include:
    - '@slsa1'
    - '@slsa2'
    - '@slsa3'
    - github_certificate
  rekorUrl: https://rekor.sigstore.dev
  sources:
  - data:
    - github.com/lcarva/ec-policies//data?ref=HACBS-2083
    name: Default
    policy:
    - github.com/lcarva/ec-policies//policy/lib?ref=HACBS-2083
    - github.com/lcarva/ec-policies//policy/release?ref=HACBS-2083
    ruleData:
      allowed_gh_workflow_names:
      - Package
      allowed_gh_workflow_refs:
      - refs/heads/master
      allowed_gh_workflow_repos:
      - lcarva/festoji
      allowed_gh_workflow_triggers:
      - push
success: true

When handling certificates all extensions marked as critical need to be checked, something like https://play.openpolicyagent.org/p/JMNo7VeKgE.
We have key usage neatly available as cert.KeyUsage and Subject Alternative Name, which for Fulcio is a bit tricky to validate in full, we get cert.DNSNames, cert.EmailAddresses, cert.IPAddresses and cert.URIs, I'm not sure if they hold all the values from SAN, i.e. if we need to encode the the expected URI and see if it matches the SAN wholly.
I've fiddled a bit with this and came up with: https://play.openpolicyagent.org/p/JehGR24uJy

I think validating that a cert is valid is outside the scope of the policy. As a policy author, I only care about checking explicit values on the certificate. Let's either have cosign do this, or do it ourselves in ec-cli.


Navbar

Adding Navbar to system

This pull request was superseded by PR #291 and is being closed without applying it.  Lissa (@Lissabella)  wrote the code for the NavBar that was incorporated into PR #291.


Add Python 3.10 support to _distutils_hack

Get the 'Loader' abstract class from importlib.abc rather than
importlib.util.abc (alias removed in Python 3.10).

Summary of changes

Closes
Pull Request Checklist

[ ] Changes have tests
[ ] News fragment added in changelog.d. See documentation for details

"import _distutils_hack" on Python 3.10 fails with:
Traceback (most recent call last):
  File "/home/vstinner/python/master/Doc/venv/bin/sphinx-build", line 5, in <module>
    from sphinx.cmd.build import main
  File "/home/vstinner/python/master/Doc/venv/lib/python3.10/site-packages/sphinx/cmd/build.py", line 25, in <module>
    from sphinx.application import Sphinx
  File "/home/vstinner/python/master/Doc/venv/lib/python3.10/site-packages/sphinx/application.py", line 31, in <module>
    from sphinx.config import Config
  File "/home/vstinner/python/master/Doc/venv/lib/python3.10/site-packages/sphinx/config.py", line 24, in <module>
    from sphinx.util import logging
  File "/home/vstinner/python/master/Doc/venv/lib/python3.10/site-packages/sphinx/util/__init__.py", line 39, in <module>
    from sphinx.util import smartypants  # noqa
  File "/home/vstinner/python/master/Doc/venv/lib/python3.10/site-packages/sphinx/util/smartypants.py", line 33, in <module>
    from sphinx.util.docutils import __version_info__ as docutils_version
  File "/home/vstinner/python/master/Doc/venv/lib/python3.10/site-packages/sphinx/util/docutils.py", line 15, in <module>
    from distutils.version import LooseVersion
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 982, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 925, in _find_spec
  File "/home/vstinner/python/master/Doc/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 74, in find_spec
    return method()
  File "/home/vstinner/python/master/Doc/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 79, in spec_for_distutils
    class DistutilsLoader(importlib.util.abc.Loader):
AttributeError: module 'importlib.util' has no attribute 'abc'

The "abc" submodule must be imported from importlib, not get from importlib.util.abc: https://docs.python.org/dev/library/importlib.html#module-importlib.abc

@vstinner This looks good to me, though it needs a news fragment.

@vstinner This looks good to me, though it needs a news fragment.

I added the changelog.d/2361.bugfix.rst file manually. Is it the expected filename and content?
Note: I failed to find a setuptools documentation explaining how to contribute to setuptools. I discovered that setuptools uses towncrier, but I failed to find a documentation explaining how to add a towncrier changelog entry :-( I'm used to blurb tool which opens a text editor and then creates the news entry.

https://setuptools.readthedocs.io/en/latest/developer-guide.html

https://setuptools.readthedocs.io/en/latest/developer-guide.html

Oh, thanks! It seems like "2361.bugfix.rst" is not an accepted filename, so I renamed it to "2361.change.rst". Is it better? I also squashed my commits.

I may be interesting to mention https://setuptools.readthedocs.io/ and/or https://setuptools.readthedocs.io/en/latest/developer-guide.html in the README file. I can write a second PR for that, once this PR is merged.  I prefer to wait until this PR is merged.)

I may be interesting to mention https://setuptools.readthedocs.io/ and/or https://setuptools.readthedocs.io/en/latest/developer-guide.html in the README file. I can write a second PR for that, once this PR is merged.  I prefer to wait until this PR is merged.)

Yeah, it would probably also be a good idea to rename the incredibly similar-seeming "Developer's Guide" and "Developer's Guide for Setuptools":

Probably that is best addressed as part of the #2093 revamp (though adding something to the README would be good too). It seems slightly easier in the latest iteration (though the confusing headings are still there, just more hidden).

I'll probably rename the changelog from change to misc so it implies an 0.0.1 bump, but good enough. Thanks Victor.

Thanks Victor.

You're welcome. Good luck with fixing setuptools 50.0 regressions. I'm a strong supporter of burying distutils! So I will try to help to make it possible.


Generates wrong bcrypt package, and version

you can see of my repo nutra-server @ 7380410
the following issue:
it installs python_bcrypt==0.3.2 instead of bcrypt==3.1.7

I am facing this issue too. Any luck? I still cannot confidently rely on pipreqs

for now, i am just using
pur -r requirements.txt

I've encountered this too - it seems like quite a significant bug considering how frequently used bcrypt is for password encryption.
It generated python_bcrypt==0.3.2 instead of bcrypt==3.2.0 for me.

Same happened to me! Thank God I noticed before deploying.

This is happening to me too. Generating requirements.txt with pipeqs always pulls in python_bcrypt==0.3.2, even though I  manually removed that and installed the desired bcrypt==4.1.3. I can't figure out what is causing the wrong bcrypt to be pulled in. Could it be something in my project code?
I am using:
Alma Linux 9, 5.14.0-427.13.1.el9_4.x86_64
pip 24.0
pipreqs   0.5.0

Same here, currently I manually replaced python_bcrypt to bcrypt in requirements.


Updated image release schedule is misleading

This document states "AKS provides one new image per week with the latest updates"
This is not accurate for Windows nodes.
Support has advised there is no defined release schedule for Windows node images, it can take up to several weeks after Microsoft updates have been released for an updated Windows node image to be available.
The wording should be corrected as to not set the expectation that an updated Windows node image should be available within 1-2 weeks of Microsoft Updates being released.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8092825b-b867-d947-9842-f44eba91db00
Version Independent ID: 5b9b865c-0d51-9955-564c-1c898d3150a7
Content: Upgrade Azure Kubernetes Service (AKS) node images - Azure Kubernetes Service
Content Source: articles/aks/node-image-upgrade.md
Service: container-service
GitHub Login: @mlearned
Microsoft Alias: jpalma

@mikenri Thanks for reaching out. We will investigate and update the thread accordingly.

@mlearned Can you please check and add your comments on this doc update request as applicable

It seems this statement isn't accurate for Linux either.
It's been 12 days since an updated Linux image has been released.

@mlearned @psmanhas  any update on the issue ?

Thank you for the feedback.
We've updated the article to address your feedback and the changes should be available soon.
#please-close


[SMALLFIX] add CreateFile option to RunOperation

@aaudiber whenever you have the time PTAL, thanks!

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/866/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/881/
Test PASSed.


Performance Suggestions?

Hello,
I've been utilizing brozzler-easy for testing and brozzler looks to be working wonderfully. I have a very large website I am trying to archive and unsure of a few things that I can't figure out through the job-conf.rst.
I'm running a local version of the website on my local machine. So that site is not running from it's public domain. Is there way to get brozzler to replace my local host domain with the actual public domain?
Another question I have, is there any way to boost the performance? Possibly configure it to use more threads? Currently when I setup a brozzler job and monitor it in Brozzler Dashboard, it shows two sites being actively crawled. Is that an example of Brozzler running two threads to crawl the site?
Maybe there's a writeup somewhere explaining optimal ways to use brozzler on a local machine?
greatly appreciate any insights. Sorry to post this here, not sure how else to get in touch with people on this project.
Thank you.

I'm running a local version of the website on my local machine. So that site is not running from it's public domain. Is there way to get brozzler to replace my local host domain with the actual public domain?

Neither brozzler nor warcprox have that functionality built in. But it sounds doable with /etc/hosts.

Another question I have, is there any way to boost the performance? Possibly configure it to use more threads?

You can configure the number of browsers running simultaneously with the -n,--max-browsers option. But only one browser at a time will work on a single site. You might need to reorganize your crawl if you want more parallelization (depending on what you're doing).

Maybe there's a writeup somewhere explaining optimal ways to use brozzler on a local machine?

I'm afraid not, as far as I'm aware :-\


Occasionally Ag.update() returns nan velocity and position

Hi Tom,
Please see this example of error I encountered:
Ag = Agent(Env)
Ag.dt = 0.1
print(Ag.pos, Ag.velocity)
# [0.79958237 0.76560547]  [0.51603493 0.49723318])

random_drift_velocity = np.random.random(2)
print(random_drift_velocity)
# [0.51777053 0.88738508]

Ag.update(drift_velocity=random_drift_velocity,
          drift_to_random_strength_ratio=2.0)
print(Ag.pos, Ag.velocity)
# [nan nan]  [nan nan]

We discovered tis bug originates in the utils.rayleigh_to_normal() function. Will be fixed at next update. Current work around is to either only pass in small drift velocities (of order Ag.speed_std) or increase the mean speed of the Agent Ag.speed_std.

Hi Daniel, thanks for raising this issue. It should be fixed now. Any chance you could confirm this then I'll close it

Thanks Tom, can confirm this has been fixed. Merry Christmas!


Add Repository::getType().

For GitRepository, this will return "git".

:+1: Name seems fine w/ me.

:ship:


Improve API doc content

Current documentation of the API is a bit "poor", it doesn't use all potential of Swagger/OpenAPI spec.
What I suggest is to update Nelmio to its newly released v4 (https://github.com/nelmio/NelmioApiDocBundle/releases/tag/v4.0.0-BETA1).
I already have an almost ready PR for this, but it's based on https://github.com/jorge07/symfony-5-es-cqrs-boilerplate/pull/160 for serialization groups.
Depending on your thoughts, I may rebase to remove this dependency and add a serialization based documentation on another PR

I really would like to migrate to openApi + have a proper library to control that response are openApi complient

So, here is what I've got so far https://imgur.com/s1NThhf. I think there is a good basis for describing API input/output.
That being said, wouldn't it worth to add API Platform as a "middleware" ? It could expose commands/queries via autogenerated API doc and validate inputs. Plus, pagination is already handled in their core, same for serializarion and elasticsearch (didn't had time to test yet)

It can be a good replacement for the api controllers. I've never used but I'm open to give a try and learn about it

Hi @jorge07 !
Sorry to not came back before, I was working on a draft for API Platform proposal. You can find my demo repo here https://github.com/jon-ht/api-platform-cqrs-es
I've started from scratch with Symfony 5.1, not all files have been reported yet and I'm still missing tests (will work on it soon)
When this project will be mature enough, I propose to open a PR here and close mine if you want (it's your project after all 😉))
Would be nice if you could take a look to give me your feedback. Don't hesitate to tell me if I'm doing something wrong

Nice, will do!

I left an issue in the repo with my initial feedback.

Will be fixed in #194

Thanks @jon-ht for all the feedback received here. I just moved to V4, released a week ago, and specs now follow openAPI and have outputs and inputs in doc


Question about training process

Thank you for your work. After reading the paper, I have a question regarding the training process that I would like to ask.
You had mentioned in your paper “For supervision, ground-truth intensity values can be obtained from the ground-truth CT image based on the coordinates of points by trilinear interpolation.” This implies that 3D CT data is needed as supervision during network training. Doesn’t this contradict the concept of "sparse view 3D reconstruction"?
I’m not sure if I misunderstood something, but I hope to get clarification. Thank you!

3D supervision is necessary during the training as the method is data-driven. However, the 3D is not needed during the inference, and the network can reconstruct the 3D from sparse-view inputs.

Why is a CT reconstruction performed when there is already ground-truth 3DCT data? The reconstructed CT data is certainly not as good as the ground-truth one, so why do you have to do extra work？

Thanks for the reply, I would also like to know where the ground-truth CT data is obtained in the code?

Thanks for the reply, I would also like to know where the ground-truth CT data is obtained in the code?

The data was preprocessed and the CT data was saved in blocks for faster data loading. See ./data/knee_cbct/3_block_save.py#L50
In the data loading, the CT values are loaded from a randomly selected block. See ./code/dataset.py#L198

Thanks for your explanation.
On Mon, Dec 30, 2024 at 9:05 PM Yiqun Lin @.***> wrote:

Thanks for the reply, I would also like to know where the ground-truth CT
data is obtained in the code?
The data was preprocessed and the CT data was saved in blocks for faster
data loading. See ./data/knee_cbct/3_block_save.py#L50
In the data loading, the CT values are loaded from a randomly selected
block. See ./code/dataset.py#L198
—
Reply to this email directly, view it on GitHub
https://github.com/xmed-lab/DIF-Net/issues/8#issuecomment-2565458951,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BMPEQKA2UKBZR6BG2IGK5CT2IFAJHAVCNFSM6AAAAABQF4S66GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRVGQ2TQOJVGE
.
You are receiving this because you commented.Message ID:
@.***>


Image tab not full width

Hi guys,
I'm using newest XLPagerTabStrip 8.1.0, XCode 10.0 and testing on iOS 12.0 and I have a problem.
I create tab with item info like this:
let itemInfo : IndicatorInfo = IndicatorInfo.init(title : "               ",image: UIImage(named: "myImage"))
And the result:

(other tabs with title and empty image are normal).
How can I fix this?

@tiendaik did you manage to solve this?


how to show a node label without hover a node

I'd like to see a node label without hover a node. Any way to do this. Many thanks.

@KweiLi
Reply might be really late. But you could use
drawLabels: true
setting for this.
Reference: https://github.com/jacomyal/sigma.js/wiki/Settings

Thanks very much

On Aug 13, 2018, at 5:08 PM, arpitahuddar<EMAIL_ADDRESS>wrote:
@KweiLi https://github.com/KweiLi
Reply might be really late. But you could use
drawLabels: true
setting for this.
Reference: https://github.com/jacomyal/sigma.js/wiki/Settings https://github.com/jacomyal/sigma.js/wiki/Settings
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/jacomyal/sigma.js/issues/922#issuecomment-412665030, or mute the thread https://github.com/notifications/unsubscribe-auth/AdF7zbjV2EpjzXLW1M1st6da357mumT0ks5uQermgaJpZM4RKQ-4.


Add markdown preview feature

Create html temp file with pandoc and open in default web browser

Done: notes view
HTML temp file management to improve


debug

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[ ] Does not have any user-facing changes.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@bianpengyuan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test release-notes_istio

integ-pilot-k8s-tests_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test release-notes_istio

integ-pilot-k8s-tests_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test integ-pilot-k8s-tests_istio

integ-pilot-multicluster-tests_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test all

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test integ-pilot-k8s-tests_istio

integ-pilot-multicluster-tests_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test integ-pilot-multicluster-tests_istio

release-notes_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-multicluster-tests_istio
9b34e03a29a7db334b41ec294a7a2cff644872cf
link
/test integ-pilot-multicluster-tests_istio

release-notes_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test release-notes_istio

integ-pilot-k8s-tests_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test release-notes_istio

integ-pilot-k8s-tests_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test integ-pilot-k8s-tests_istio

integ-pilot-multicluster-tests_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test all

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test integ-pilot-k8s-tests_istio

integ-pilot-multicluster-tests_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test integ-pilot-multicluster-tests_istio

release-notes_istio
c3308e68c6ef8feeba732f29081fbbe238b4c79f
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
3e040efcca35eaa034770fb4f0c93a734091e837
link
/test integ-pilot-k8s-tests_istio

release-notes_istio
c3308e68c6ef8feeba732f29081fbbe238b4c79f
link
/test release-notes_istio

integ-pilot-multicluster-tests_istio
c3308e68c6ef8feeba732f29081fbbe238b4c79f
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
c3308e68c6ef8feeba732f29081fbbe238b4c79f
link
/test release-notes_istio

integ-pilot-multicluster-tests_istio
c3308e68c6ef8feeba732f29081fbbe238b4c79f
link
/test integ-pilot-multicluster-tests_istio

integ-pilot-k8s-tests_istio
c3308e68c6ef8feeba732f29081fbbe238b4c79f
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@bianpengyuan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-multicluster-tests_istio
c3308e68c6ef8feeba732f29081fbbe238b4c79f
link
/test integ-pilot-multicluster-tests_istio

integ-pilot-k8s-tests_istio
c3308e68c6ef8feeba732f29081fbbe238b4c79f
link
/test integ-pilot-k8s-tests_istio

release-notes_istio
7d461d6a023c732ae9da6f9672084985a9d99392
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest


Allow variable substitution in any part of Header value

Currently it's limited to the beginning of the header value.
In feign.RequestTemplate#resolve():
public RequestTemplate resolve(Map<String, ?> unencoded) {

    for (String field : headers.keySet()) {
      Collection<String> resolvedValues = new ArrayList<String>();
      for (String value : valuesOrEmpty(headers, field)) {
        String resolved;
        if (value.indexOf('{') == 0) {  <<<<<<<<<<<<<<< unnecessary limitation?
          resolved = expand(value, unencoded);
        } else {
          resolved = value;
        }

  }

value.indexOf('{') == 0 unnecessarily limits the variable substitution.
So in the following awesomeVariable will get replaced:
@Headers({"My-Awesome-Header:  {awesomeVariable} something"})
but not in:
@Headers({"My-Awesome-Header:  something {awesomeVariable}"})

Is there a particular standard header you are looking to support? Or is
this a custom one?

Yes, this is a custom header, where I have to pass a user id dynamically.
For example,
@Headers({"X-Jive-Run-As:  userid 100001"})
Documentation around this custom header "X-Jive-Run-As" is at https://community.jivesoftware.com/docs/DOC-110343

Ahh interesting. Yeah I thought about it, and don't thing this is a big
deal. Feel free to raise a PR to allow this!

FWIW, the auth0 v2 api expects a query string like ?q=field:{fieldValue} (ie /api/v2/users?search_engine=v2&q=email:"me@you.com"). This is troublesome to for feign because it is looking for { at position 0.

That being said, it is possible to encapsulate weird api's like this by writing a wrapper class around your feign client. That's what i did. Keeps feign clean.

I ran into this issue when trying to send an Authorization-Header with a Bearer token.
@Headers("Authorization: Bearer {access_token}")
Actually I'm using an Expander to fill in "Bearer " as prefix to the token. Would be more smarter when placeholders can be at any index.

With #778 merged, it is now possible to declare headers as templates and have them resolved using method parameters.  I'm going to close this issue for now.  If others on this thread find that support is still lacking, please feel free to create a new issue.


LibriTTS vocoders

Hi @kan-bayashi
A general question regarding the LibriTTS vocoders -- I am using them (melgan/hifigan/parallel_wavegan) to decode the spectrograms generated from LibriTTS Tacotron2/Transformer-TTS. When the setup was in single-speaker LJspeech, the synthesis sounds good -- the audio was crisp. However, moving to LibriTTS, the same synthesis now sounds mechanic (some of them are still decent). The attention plots from Tacotron2/Transformer-TTS are diagonal. I am guessing the issue lies in LibriTTS vocoders, as they are not as good as LJspeech vocoders? Or is the issue pertains to speaker modeling over, that the speaker encoder (speechbrain's ECAPA-TDNN) does not capture LibriTTS' speaker distribution sufficiently, which affects the synthesized spectrograms from Tacotron2/Transformer-TTS?

You should clarify your problem is came from text2mel part or vocoder part.

When the setup was in single-speaker LJspeech, the synthesis sounds good -- the audio was crisp.

You mean both text2mel and vocoder are trained on LJSpeech?

However, moving to LibriTTS, the same synthesis now sounds mechanic (some of them are still decent).

You mean both text2mel and vocoder are trained on LibriTTS?

I am guessing the issue lies in LibriTTS vocoders, as they are not as good as LJspeech vocoders?

You can check LJSpeech text2mel + LibriTTS vocoder for the comparison.

In my experiments, multi-speaker (x-vector-based) taco 2 / transformer trained on LibriTTS is not so good.
Basically, LibriTTS includes a lot of long pause and the pause symbol is not appeared in the text.
In ESPnet recipe, I tried to remove them as possible as I can using force alignment results, but that is not perfect.
There, in terms of quality, single speaker model is definitely better than multi-speaker model when using taco2 or Transformer.

My experience with Tacotron2 so far has been that this is a very important step. Not only framing silence but also pauses between the words can cause Tacotron2 to fall apart. VAD is not a reliable way of detecting those.

the synthesis depends heavily on the speaker encoder in tacotron2/transformer-TTS

Right. The quality largely depends on text2mel part. Vocoder is trained on natural speech so it is not good at recovering the missing components in generated melspectrogram. It tends to synthesis it as is.

How important is this particular pre-processing? The silence trimming is based on force alignment to identify silence segments.

As @oytunturk, this trimming is very important especially for AR S2S models.
You can find my experiments with libritts here (https://github.com/espnet/espnet/pull/2807).
NAR models are more robust but it is definitely better to remove as possible as we can.
Most of the training problems will be solved by cleaning data.

hi @kan-bayashi
Thanks for the response. Some updates on my side: I have identified the issue I am facing to be speaker modeling. What I did is to pick a train set speaker that sounded good, use that speaker embedding for all the test utterances, and found the test utterances sound good. I believe the issue is that there are (many) LibriTTS speakers in (both) train/test sets that the pre-trained speaker encoder does not predict well.
What I plan to do is to take my current xvector + TTS  and finetune with one-hot speaker label. I hope this will improve speaker modeling.
One additional question -- for LibriTTS pre-trained vocoders, are they only trained on the training set? Since LibriTTS train & test sets do not have any speaker overlap, could there be speaker modeling issue in vocoder as well?

Thank you for sharing your insight, @jefflai108.

Since LibriTTS train & test sets do not have any speaker overlap, could there be speaker modeling issue in vocoder as well?

Good point. I have read some papers about the vocoder using speaker embedding to enhance the quality of unknown speaker voice. There is a room for the improvement for unknown voice. However, in my experiment, the problem is more serious in text2mel part rather than vocoder (Vocoder works well by using just multi-speaker training data without any modification of the architecture).

Thanks for your insight @kan-bayashi
Please allow a few days for me to do more experiments.

hi @kan-bayashi
I have some updates for the multi-speaker TTS. Some speakers are really good, and some are sub-optimal. My co-author and I are still deciding whether to submit it to Interspeech. Since you helped answer many of my questions, I would like to invite you to the author list. If you are interested in it, I can tell you more about what the submission is about and send you the demo page. Please let me know! and perhaps what's the best way to contact you?

@jefflai108 Thank you for sharing your status and inviting me as a co-author :)
But I have just answered a few question, I think it is not enough to be added as a co-author.
So if you add me in acknowledgment in paper or demo hp, I would be grateful.

sure, sounds good @kan-bayashi


Import ground markers

No longer using ground marker tiles.
Added ability to convert ground markers into tileman tiles

I'm not seeing the panel when I test locally :(


Authenticating with S3 from local machine

I am trying to authenticate to a vector store on S3. It works from inside AWS Lambda but not when running locally. Looking at the source code I'm not sure if it is using AWS credentials. Is it possible to pass credentials? I have confirmed my authenticated user can access the folder on s3 using boto3 so they do have permissions but I guess they are not being used. Here's what I was trying:
import lancedb

S3_BUCKET_NAME = "my-bucket-name"
db = lancedb.connect(f's3://{S3_BUCKET_NAME}')

table_name = "my-table
tbl = db.open_table(table_name)

Here's the error msg:
OSError: When getting information for key 'my-table.lance' in bucket 'my-bucket-name': AWS Error ACCESS_DENIED during HeadObject operation: No response body.

how have you setup the credentials on the host where you're running that code? Authentication is done in Rust, which should pick up the whole credential chain. Just to double check, are you able to try setting the credentials in the standard envvars and see if that changes things?

This code is running on my laptop. I have not done anything explicitly to set credentials in the code as I couldn't see any docs on it. I am writing Python and have never used any Rust. I normally authenticate using AWS SSO and if I use boto3 to query s3 it will pick up my creds. When you say set the creds in standard environment variables do you mean access key and secret, i.e. not SSO I guess?

@wjones127 does object store work well with aws sso? (And does the lance credential provider pick that up)?

Hello,
Same issue here, to access a public bucket on S3 compatible storage (Minio)
The fllowing code works fine:
import boto3
client = boto3.client("s3", endpoint_url="https://<my_minio_url>",)
client.list_objects_v2(Bucket="my_public_bucket")

and this code too works fine:
import boto3
from s3path import S3Path, register_configuration_parameter
minio_ressource = boto3.resource('s3', endpoint_url="https://<my_minio_url>")
minio_bucket_path = S3Path("/my_public_bucket")
register_configuration_parameter(minio_bucket_path, resource=minio_ressource)
print([d.as_uri() for d in minio_bucket_path.glob("*/*.lance")])

but the following:
import lancedb
ds = lancedb.connect("s3://my_public_bucket/my_lance_tables_dir")
ds.table_names()

raises
OSError: When listing objects under key 'my_lance_tables_dir' in bucket 'my_public_bucket': AWS Error INVALID_ACCESS_KEY_ID during ListObjectsV2 operation: The AWS Access Key Id you provided does not exist in our records.
I also tried with credentials set in env. vars (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY,...) or in ~/.aws/credentials (it's a public bucket but tried anyway)
What is weird about this, it's this code worked a couple of mounths ago, but even by reverting to older lancedb version I now always have this issue. So maybe it's not really related to lancedb, but it works with direct boto3 or S3Path access

@BertrandRenault For connecting to minio, have you set both AWS_ENDPOINT and AWS_DEFAULT_REGION?
export AWS_ENDPOINT=https://<my_minio_url>
export AWS_DEFAULT_REGION=us-east-1

https://lancedb.github.io/lance/read_and_write.html#s3-compatible-stores

Thans a lots, I had this set before but it was changed from AWS_ENDPOINT to ENDPOINT_URL...
It now work perfectly, thanks again

Why was this closed? Does it work with SSO? What was the solution?

@wjones127 why was this issue closed? I am still facing the problem and cannot see what the solution is. How can I connect to a LanceDB table in S3 from my local machine for when using AWS SSO?


DotNetCoreBuildSettings MSBuildSettings AddFileLogger() does not produce msbuild.log

With dotnet --version == 2.0.0
Task("Build")
    .IsDependentOn("Restore-NuGet-Packages")
    .Does(() =>
{
    DotNetCoreBuild("./TheSolution.sln", new DotNetCoreBuildSettings
    {
        Configuration = configuration,
        MSBuildSettings = new DotNetCoreMSBuildSettings
        {
            TreatAllWarningsAs = MSBuildTreatAllWarningsAs.Error,
            Verbosity = DotNetCoreVerbosity.Minimal
        }.AddFileLogger()
    });
});

does not produce the expected msbuild.log file.
Running dotnet build /fl successfully creates the file, though.

@kendaleiv what command line do you get if you run Cake with diagnostic logging?
Does it work if you specify the file name using the AddFileLogger(this DotNetCoreMSBuildSettings settings, MSBuildFileLoggerSettings fileLoggerParameters) extension method and specify a LogFile?

Executing: "C:/Program Files/dotnet/dotnet.exe" build "./TheSolution.sln" --configuration Release /warnaserror

With .AddFileLogger(new MSBuildFileLoggerSettings { LogFile = "msbuild.log" }) specified it creates the file.
Executing: "C:/Program Files/dotnet/dotnet.exe" build "./TheSolution.sln" --configuration Release /fileLogger /fileloggerparameters:LogFile="C:/Users/ken.dale/Documents/Projects/the-repository-folder/msbuild.log" /warnaserror


Use github actions to release

I want to use github actions to build macOS application.
I'm using the image of macOS-10.14 provided by github actions, but it seems have problem.
https://github.com/CaiJingLong/JsonToDart/releases/tag/v0.0.20
I get the result when I double click to open it.

And if I use the cli to open the binary of flutter, it is normal.
$JsonToDart.app/Contents/MacOS/json_to_dart
After opening via cli, the app is also restored to normal, I can open it by double-clicking.
Zip file is my screen recorder to reproducing the problem.
Kapture 2019-11-18 at 13.31.14.mp4.zip

hover doctor
hover doctor
hover: Running on darwin
hover: Docker installed: true
hover: Sharing flutter version
Flutter 1.9.1+hotfix.4 • channel beta • https://github.com/flutter/flutter.git
Framework • revision cc949a8e8b (7 weeks ago) • 2019-09-27 15:04:59 -0700
Engine • revision b863200c37
Tools • Dart 2.5.0
hover: Flutter engine commit: https://github.com/flutter/engine/commit/b863200c37
hover: Finding out the C compiler version
Apple clang version 11.0.0 (clang-1<IP_ADDRESS>)
Target: x86_64-apple-darwin19.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
hover: Sharing the content of go.mod
module github.com/fluttercandies/JsonToDart/tree/master/Flutter/desktop/desktop

go 1.12

require (
        github.com/go-flutter-desktop/go-flutter v0.33.0
        github.com/pkg/errors v0.8.1
        github.com/stretchr/objx v0.2.0 // indirect
        gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
        gopkg.in/yaml.v2 v2.2.5 // indirect
)
hover: Warning: No hover.yaml file found: open go/hover.yaml: no such file or directory
hover: Sharing the content of go/cmd
go/cmd/compatibility_darwin.go  go/cmd/main.go  go/cmd/options.go

Duplicate of #313
Sorry for making you create another issue, I didn't realize it was the cause of 313.

Without additional information, we are unfortunately not sure how to resolve
this issue. Please follow #310 as the issue has more comments.
Thanks for your contribution.


Port Extension to Firefox

[ ] Modify the extension api calls to work for Chrome or Firefox
[ ] Modify the build process to build:

[ ] an unpacked chrome extension, for dev
[ ] a packed firefox extension, for prod and dev
[ ] a packed chrome extension, for prod

Resources
Building a Cross Browser Extension

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/user_interface/Popups
https://extensionworkshop.com/documentation/develop/temporary-installation-in-firefox/
https://extensionworkshop.com/documentation/publish/package-your-extension/
https://extensionworkshop.com/documentation/develop/web-ext-command-reference/#web-ext-run
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/tabs
about:debugging#/runtime/this-firefox


Upgraded dataservices api extension version

Upgrade the dataservices API version to 0.2.0 that has the isolines feature

retest this please


Worked around misspelled BadZipFile class in Python 2

The zipfile module in Python 2 has the BadZipFile exception class misspelled as BadZipfile.
Closes #652

/morph test


Required resources and time for training

Hi
Thanks for your inspiring work.
Is the current code complete and ready for running? and how long does it generally take with  one 3090?

I haven't tried it on the 3090, but on the V100 it takes less than a day if you don't use Landmark information.


Allow setting copybutton_prompt_text for specific codeblocks only

Description / Summary
We're using sphinx-copybutton for our documentation and love it. Thank you very much.
We'd like to allow stripping the prompt text for specific blocks of code but not all of them.
You can see an example here:
https://docs.robusta.dev/master/getting-started/installation.html
I'd like to strip $ from one code block on that page (and only copy the line with $). The other code blocks don't contain $ so I'd like to avoid doing any tricks there.
Value / benefit
It would make the copybutton_prompt_text feature usable for us.
Implementation details
No response
Tasks to complete
No response

To clarify, the stripping isn't the main issue. The main issue is determining which lines to copy and that should sometimes only be lines with $ and sometimes all lines.

Now that a new version is released, I would say: don't use copybutton_prompt_text, use copybutton_exclude.  copybutton_prompt_text hard codes patterns to exclude . copybutton_exclude excludes text by CSS classes... and pygments (the highlighter) can magically label the text for you!
copybutton_exclude = '.linenos, .gp'

For example, console session, it excludes the $ sign:
$ echo hi
hi

If it semantically isn't a console session where prompts are not something to copy but some complete text to be copied:
$ echo hi
hi

This doesn't just work for console but any pygments language that tags prompts as .gp.


AllowClickThrough + DisableURLRewrite = ?

Hello,
If we enable "Let users click through to the original URL" (AllowClickThrough), when "Do not rewrite URLs, do checks via Safe Links API only" (DisableURLRewrite) is also selected, does the former still apply?  In other words, with "Do not rewrite URLs, do checks via Safe Links API" enabled, is there even a webpage where a "Proceed anyway" link could be shown?
Thanks

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0c77e1cb-1509-83cc-7c2a-564d5e3e91ea
Version Independent ID: 9f07278c-37ab-beac-a5dd-fa4f4825b3a4
Content: Microsoft recommendations for EOP and Defender for Office 365 security settings
Content Source: microsoft-365/security/office-365-security/recommended-settings-for-eop-and-office365.md
Service: microsoft-365-security
Sub-service: mdo
GitHub Login: @chrisda
Microsoft Alias: chrisda

Checking

@JeremyTBradshaw, I'm told that the Do not rewrite URLs, do checks via Safe Links API only doesn't affect the availability of the block page if a user clicks on a bad link; the setting just skips wrapping the link. Whether or not the user can click through to the original link is still controlled by the Let users click through to the original URL setting.
OK?

Hi @chrisda that's perfect, thank you very much.  I will refer to this issue for the current client and may do a PR later to update the explanation of the API approach.

To make it easier for you to submit feedback on articles on learn.microsoft.com, we're transitioning our feedback system from GitHub Issues to a new experience.
As part of the transition, this GitHub Issue will be moved to a private repository. We're moving Issues to another repository so we can continue working on Issues that were open at the time of the transition. When this Issue is moved, you'll no longer be able to access it.
If you want to provide additional information before this Issue is moved, please update this Issue before December 15th, 2023.
With the new experience, you no longer need to sign in to GitHub to enter and submit your feedback. Instead, you can choose directly on each article's page whether the article was helpful. Then you can then choose one or more reasons for your feedback and optionally provide additional context before you select Submit.
Here's what the new experience looks like.
Note: The new experience is being rolled out across learn.microsoft.com in phases. If you don't see the new experience on an article, please check back later.
First, select whether the article was helpful:

Then, choose at least one reason for your feedback and optionally provide additional details about your feedback:

Article was helpful
Article was unhelpful

Finally, select Submit and you're done!

Meant to close it upon @chrisda 's last update.


Xamarin.Forms 5 + Android = Images often fail to load when loading image is used

Hi.
This happens on Androids only, and with all image types I've tested with (and all permutations of caching), including SVG images and downloaded images.
The problem is that if I use a loading image (an animated spinner gif in my case), when the image has finished downloading/loading, the loading image never disappears. So the spinner stops but doesn't go away, completely obscuring the image beneath. This happens very very often, maybe 75% of the time.
I would love a fix for this as it's a blocker atm. I'll be happy to provide additional information.
Thanks.
(I never tested this with XF 4 so I can't say if it's an XF 5 issue only)

Hi. Isn't anyone else having this problem? FFImageLoading images just won't load on Android for me. They fail at least three out of four times.
Is this project supported anymore?
Thanks.

Nobody uses FFImageLoading with Android at all?
Or is FFImageLoading dead?
Thank you.

This is also happening for me

Ping. Any updates on this?
Images fail to load on Android when a loading image is used, with Xamarin.Forms 5.
Thank you!

Ping. Any updates on this?
Images fail to load on Android when a loading image is used, with Xamarin.Forms 5.
Thank you!


JAVA-1125 Change way remove(query) on gridfs is performed to improve performances (branch 3.0.x)

Problem
While using a query to remove data in the GridFS (both [bucket].files and [bucket].chunks collections) the current driver first issues a select, and then loops over the results to run 2 removes (files and chunks) on each iteration.
On large resultsets, this behavior can results in thousands of requests x2 (files and chunks).
I can understand that performing files and chunks removal, one after the other, is a way to limit data inconsistency. But there still is a risk.
Thus, as long as the linked removal between files and chunks isn't managed by the server itself, the client side is responsible for checking whether both files and chunks are consistent.
Solution
I updated my previous PR by adding a parameter to keep the legacy behavior but allowing to force the "bulk removal".
A remove(query) = remove(query,true) as the default existing behavior.
A remove(query, false) only issues 3 requests :

one select on "files",
and then a remove using the query on files
and a remove with a $in clause on chunks.
Fields ids are remembered using a list.
On a single remove this won't be a great improvement, but on large sets of files it'll be worthwhile.

"Legacy" remove(query) = 2 * n requests for remove on gridfs.
"Bulk" remove(query, false) = 3 requests for remove on gridfs.
where n = number of files matched by the query.
[Fixes : https://jira.mongodb.org/browse/JAVA-1125]
[Updates/Improves of PR https://github.com/mongodb/mongo-java-driver/pull/171 against branch 3.0.x]

My sincere apologies.  We recently merged the 3.0.x into master, and on deletion of the 3.0.x branch all the pull requests were summarily closed.


OpenAPi codegen for processes should use Process Documentation and Start Event documentation to populate descriptions in openapi and swagger ui

Description
for a process there is the element doc and the "Process instances description" fields:

and on a start event we have:

This documentation would be very helpful add into the openapi spec / swagger ui.
if multiple Start Events are supported, then having descriptions on what each event does in the swagger would be very valuable.
Implementation ideas
No response

Hi @StephenOTT! This is reasonable, opened https://issues.redhat.com/browse/KOGITO-6573 to track it.


ci: add GitHub token permissions for workflow

This PR adds minimum token permissions for the GITHUB_TOKEN using https://github.com/step-security/secure-workflows.
GitHub recommends defining minimum GITHUB_TOKEN permissions for securing GitHub Actions workflows

https://github.blog/changelog/2021-04-20-github-actions-control-permissions-for-github_token/
https://docs.github.com/en/actions/security-guides/automatic-token-authentication#modifying-the-permissions-for-the-github_token
The Open Source Security Foundation (OpenSSF) Scorecards treats not setting token permissions as a high-risk issue

This project is part of the top 100 critical projects as per OpenSSF (https://github.com/ossf/wg-securing-critical-projects), so fixing the token permissions to improve security.
Signed-off-by: Varun Sharma<EMAIL_ADDRESS>
What kind of change does this PR introduce?
Build related change

Did you add tests for your changes?
New tests not needed for the change

Does this PR introduce a breaking change?
No

What needs to be documented once your changes are merged?
Not applicable

Thanks


Let's join our Russian brothers in liberating capitalism

Let's join our Russian brothers in liberating capitalism。
We need peace and respect

Closing - please read about our response https://github.com/facebook/react/issues/23402#issuecomment-1058227670


Guard x86 specific codes with "#if defined(i386) || defined(x86_64)"

Basically this patch set allows to cross compile xbyak and returns zero feature detected from xbyak, it is part of the pull request to mkl-dnn (a client of xbyak) at https://github.com/intel/mkl-dnn/pull/393.
mkl-dnn detects the CPU features provided by xbyak before jitting, if there is no corresponding feature such as SSE4.2 or AVX512, it will fall back to a C++ implementation for the AI operation. In order to cross compile and use mkl-dnn and dldt (a client of mkl-dnn), we have to use the C++ implementation inside mkl-dnn. This patch set gets it done by guarding the x86 specific codes.

Thank you for the patch. I merged and modified it a little.


Does ist need CakePHP?

Would it be possible to add the Tab also to any other web application?

I guess you could do this quite easy. It only uses the Cake conventions to wrap it as a plugin. The functionality could be applied to any (php) web application.

What would the steps to be done? I guess "just" setting up a fresh CakePHP FeedbackIt app for the backend and then put the Feedback widget in any app?


Use of node loader extensions for valid file types of parsing definition sources

Hello, @adrai!
Here is a proposed solution regarding of initializing domain files in other than *.js file extensions. For resolving valid file types here being used node loader extensions by control of domain init param useLoaderExtensions which is a boolean. On option default value it uses default valid file types.
Issue #89

Can you extend the readme? and if possible also a little "integration" test?

Have no idea how to add integration test for this. But about extending readme I'm ok to do so.

Ready, readme is done.

v2.10.0


planner-api: Split the main into commands

This PR splits the main file into commands to have another command migrate along with the run command.
The main.go functionality was moved into run command and a new command migrate is added that is only migrating the db. The migrate command is needed to migrate db during unit tests.
Signed-off-by: Cosmin Tupangiu<EMAIL_ADDRESS>

@machacekondra ptal


DeprecationWarning: 'originalKeywordKind' has been deprecated since v5.0.0 and will no longer be usable after v5.2.0. Use 'identifierToKeywordKind(identifier)' instead

When using TSTL on the latest version of TypeScript, it produces this warning:

DeprecationWarning: 'originalKeywordKind' has been deprecated since v5.0.0 and will no longer be usable after v5.2.0. Use 'identifierToKeywordKind(identifier)' instead

I tried it and it didn‘t work

What version of TSTL are you using?

Sorry ,I used umi. thank you. I'll think of something else
------------------ 原始邮件 ------------------
发件人:                                                                                "TypeScriptToLua/TypeScriptToLua"                                                                                @.>;
发送时间: 2023年8月26日(星期六) 晚上9:51
@.>;
<EMAIL_ADDRESS>主题: Re: [TypeScriptToLua/TypeScriptToLua] DeprecationWarning: 'originalKeywordKind' has been deprecated since v5.0.0 and will no longer be usable after v5.2.0. Use 'identifierToKeywordKind(identifier)' instead (Issue #1450)
What version of TSTL are you using?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>


[RN][iOS] Fix static linking for Bridgeless mode

Summary:
Working with @gabrieldonadel, we realized that static frameworks of the React-RendererRuntime are not following the proper folder structure.
When a user tries to import ReactCommon/RCTHost in the app delegate, for example, the user ends up with an error and they can't find the files.
These changes fixes this by establishing the right folder structure in the static frameworks
Changelog:
[Internal] - Make sure that React-RuntimeCore and JSErrorHandler are created with the proper structure for static frameworks
Test Plan:
Tested locally on an app with 0.74.
Before: it failed to build.
After: it build successfully.

@gabrieldonadel I tested this locally and it works. Do you mind test it yourself as well?


[5.3] Query builder in the wrong link database query

When your database is configured to read and write
// If you do it first.
DB::table('test')->get();
// some other operation
DB::connection ('mysql::write')->table('detail')->get();
// The database link you are using in your query is still read-only.

Please paste your database config file to recreate the bug.

@taylorotwell  Here is the database config file.

// database.php
'default' => 'mysql',

...

'mysql' => [
    'read' => [
        'host' => '<IP_ADDRESS>',
    ],
    'write' => [
        'host' => '<IP_ADDRESS>'
    ],
    'driver'    => 'mysql',
    'database'  => 'database',
    'username'  => 'root',
    'password'  => '',
    'charset'   => 'utf8',
    'collation' => 'utf8_unicode_ci',
    'prefix'    => '',
],

I've made this change on 5.4.


Update specs2-core, specs2-scalacheck to 4.9.3

Updates

org.specs2:specs2-core
org.specs2:specs2-scalacheck

from 4.3.6 to 4.9.3.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.specs2" } ]

labels: test-library-update, semver-minor

Superseded by #103.

Superseded by #103.


Bug in backtest

@lppkarl I believe I found a bug.  NullReferenceException in backtesting.  Maybe I'm doing something wrong?
If I am doing something wrong, maybe we need validation.  Otherwise I think this is valid scenario, no?
Here is my sample.
     public class MyClass
{
      static int RunBackTest()
        {
            var importer = new AlphaVantageImporter("[YourAPIKey]", OutputSize.full);
            var fb = importer.ImportAsync("fb", startTime: new DateTime(2018,1,1), period: PeriodOption.PerMinute).Result;

            var importer2 = new AlphaVantageImporter("[YourAPIKey]", OutputSize.full);
            var wba = importer2.ImportAsync("xom", startTime: new DateTime(2018, 1, 1), period: PeriodOption.PerMinute).Result;

            var importer3 = new AlphaVantageImporter("[YourAPIKey]", OutputSize.full);
            var att = importer3.ImportAsync("t", startTime: new DateTime(2018, 1, 1), period: PeriodOption.PerMinute).Result;

            var buyRule = Rule.Create(c => c.IsMacdBullishCross(12, 26, 9))
                            .And(c => c.IsRsiOversold2());

            var sellRule = Rule.Create(c => c.IsMacdBearishCross(12, 26, 9))
                            .And(c => c.IsRsiOverbought2());

            var runner = new Trady.Analysis.Backtest.Builder()
                .Add(att, 33).Add(wba, 33).Add(fb, 33)
                .Buy(buyRule)
                .Sell(sellRule)
                .Build();

            var result = runner.RunAsync(10000, 1, new DateTime(2018,1,1)).Result;

            Console.WriteLine($"Transaction count: {result.Transactions.Count():#.##}, P/L ratio: {result.TotalCorrectedProfitLossRatio * 100}%, Principal: {result.TotalPrincipal}, Total: {result.TotalCorrectedBalance}");

            return 0;
        }
}

public static class Signals
    {
        public static bool IsRsiOverbought2(this IIndexedOhlcv ic, int periodCount = 14)
            => ic.Get<RelativeStrengthIndex>(periodCount)[ic.Index].Tick.IsTrue(t => t >= 60);

        public static bool IsRsiOversold2(this IIndexedOhlcv ic, int periodCount = 14)
            => ic.Get<RelativeStrengthIndex>(periodCount)[ic.Index].Tick.IsTrue(t => t <= 40);
    }

The null reference happens because a sell signal is discovered before a buy signal, getting the open of the last transaction will throw null reference exception

@lppkarl so did I do something wrong here?  How do I prevent this from occurring? Or do we need to do a null check before processing the sell signals?

Nope, you're not doing wrong. I've commited code to fix the issue this morning.


Lottery 10 Spin Location

I know this is more like a joke, but if the click location can be a little bit to the left...

Seriously speaking, a bit to the left might go outside the 10x summon button.
It's clicking at the perfect place for NA right now. YKWIM.

Ah. If it's clicking in the right place for NA then it's perfect ;)


Frost warning

Would be nice to have the "frost warning".
I switched from "https://github.com/wijnandtop/home_assistant_gardena" repository where I could get this property.

Hello Bjornta,
Unfortunately, this information is not available in the official API. Wijnandtop was using an unofficial reverse-engineered API.
We cannot do it until Gardena decides to add it in the official API, but when it will  be available, we will add it.
Thanks, Grm


chore: Upgrade Go dependencies to address CVEs (#76)

elazarl/goproxy [CVE-2023-37788] Denial of Service (DoS)
emicklei/go-restful [CVE-2022-1996] Authorization Bypass Through User-Controlled Key
prometheus/client_golang [CWE-400] Denial of Service (DoS)

Reference commit: https://github.com/opendatahub-io/modelmesh-runtime-adapter/commit/014856cf19f535b66b8f2d63a5dd96e1462b810d

CI issue (python), fixed by https://github.com/opendatahub-io/modelmesh-runtime-adapter/pull/43


Check for Redundant braces in Given Expression.

💥 Proposal
Given a string denoting a valid expression which contains the operators ’+’, ‘-‘, ‘*’, ‘/’.
Check whether the given string has redundant braces or not.
Return 1, if it has redundant braces, else return 0.
Have you read the Contributing Guidelines on Pull Requests?
Yes.

/assign


Connect Arduino sketch to MQTT broker

As a user I want the smart car to be able to connect to my UI and follow commands.
Acceptance criteria: Smart car performs commands given by user input through the UI.

Done and just need to put it in the master branch.


Adds IT and PT languages

Self descriptive.

Thanks for the approval @FelipeFMA, I can't merge it though, no access.

grazie & obrigado @gmarconi ! sorry it took a few days, busy times over here.


Adding sample yaml files for mounting secrets as file

Purpose

Add documentation for new feature that allows secrets to be mounted as files

Does this introduce a breaking change?
[ ] Yes
[x] No

Pull Request Type
What kind of change does this Pull Request introduce?
[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[x] Documentation content changes
[ ] Other... Please describe:

@sajayantony @northtyphoon @SteveLasker


Docs: Improve Font awesome SVG example.

Bug, feature request, or proposal:
Proposal
What is the expected behavior?
Font awesome SVG documentation should be improved.
What is the current behavior?
Ng Material FA docs are limited and don't demonstrate SVG best practices. Font awesome imports of SVGs use their own custom object which breaks down the SVG/XML and represents it as an object, Ng Material expects the complete XML to be added to its registry.
This native SVG usage is preferred because Ng Material has very specific CSS styles that make embedding icons as children of mat-icon element not work as expected, this documentation can be improved so 3rd party icons work seamlessly with material.
i.e. embedding the font awesome icon can be done as:
<button mat-menu-item>       <mat-icon><fa-icon [icon]="faGoogle"></fa-icon></mat-icon>       Google </button>
... this is the 'default SVG way' if you follow FA docs, it produces an additional FA wrapper element which is undesirable as it also has it's own CSS embedded which does not always play nicely with materials.
It works best as:
<button mat-menu-item > <mat-icon svgIcon="fab:google"></mat-icon> </button>
What is the use-case or motivation for changing an existing behavior?
Seamless FA + ng Material Icons integration example.
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
Latest
Suggested Doc
Suggestion Example code for using FA+ng Material Icons:
Add tree shakable Imports:
import { faGoogle } from '@fortawesome/free-brands-svg-icons';
`iconService.addSvg(faGoogle); // add icon to library`

HTML usage:
<mat-icon svgIcon="fab:google"></mat-icon> or <mat-icon svgIcon="google"></mat-icon>
Add helper service:
`
@Injectable()
export class IconService {
constructor(private iconRegistry: MatIconRegistry, private sanitizer: DomSanitizer) { }
addSvg(icon: IconDefinition) {
// need to re-assemble the SVG XML
const svg =
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 ${icon.icon[0]} ${icon.icon[1]}"><path d="${icon.icon[4]}" /></svg>; // use template strings
// consider adding a duplicate check here so you don't do this twice
this.iconRegistry.addSvgIconLiteralInNamespace(
icon.prefix, // prefix here is optional, implement as needed
icon.iconName,
this.sanitizer.bypassSecurityTrustHtml(svg)
);
}
}
`

@tomgruszowski wow...it did work and so simple to implement, thanks. This needs to be in the docs.

@tomgruszowski
I just wanted to say thanks for sharing your solution.  All information out there is as a result of your effort.  I tried your method, and it seemed fine, but I also stumbled on this icon method.
import { faHome } from '@fortawesome/free-solid-svg-icons';
import { icon } from '@fortawesome/fontawesome-svg-core';

constructor(iconRegistry: MatIconRegistry){
   [faPlay].forEach((iconDefinition) => {
          const i = icon(iconDefinition);
          iconRegistry.addSvgIconLiteralInNamespace(iconDefinition.prefix, iconDefinition.iconName, sanitizer.bypassSecurityTrustHtml(i.html[0]));
   });
}

Seems to work the same, but I don't like the [0] bit.

Finally just to share for anyone else.
I ended up with something like this icon.service.ts
import { Injectable } from '@angular/core';
import { MatIconRegistry } from '@angular/material';
import { DomSanitizer } from '@angular/platform-browser';

import { faStackOverflow, faGithubSquare, faGoodreads } from '@fortawesome/free-brands-svg-icons';
import { faCog, faBars } from '@fortawesome/free-solid-svg-icons';
import { icon } from '@fortawesome/fontawesome-svg-core';

@Injectable({
  providedIn: 'root'
})
export class IconService {

  constructor(iconRegistry: MatIconRegistry, sanitizer: DomSanitizer) {

    // make a list of all supported icons
    const iconDefinitions = [faStackOverflow, faGithubSquare, faGoodreads, faCog, faBars];

    // process
    for (const iconDefinition of iconDefinitions) {

      const icn = icon(iconDefinition, {
        transform: {
          size: 24
        }
      });

      const { prefix, iconName, html } = icn;
      const svg = sanitizer.bypassSecurityTrustHtml(html[0]);

      // this string is used in the `svgIcon` property of a `<mat-icon>`.
      console.log(`${prefix}:${iconName}`);

      iconRegistry.addSvgIconLiteralInNamespace(prefix, iconName, svg);
    }
  }
}


Feature Pyramid Network code bug

https://github.com/pytorch/vision/blob/c2e8a00885e68ae1200eb6440f540e181d9125de/torchvision/ops/feature_pyramid_network.py#L60
this line does not work, I think it should be modified as self.modules() instead of self.children()

Hi @cl2227619761, what do you mean by

this line does not work

?
Please post a minimal example where you encounter an issue with this line.

Hi @cl2227619761
That is a very good catch!
@pmeier the issue is that self.children() returns the immediate modules, so in this case it would be inner_blocks and layer_blocks, none of which are nn.Conv2d.
The fix is what @cl2227619761 mentioned: replace self.children() with self.modules().
But this brings the question on the impact of the initialization. If I remember correctly, we needed the custom initialization in order to get best results, so I'm surprised that this was left out.
A PR fixing this would be great, but it would also be great if we could measure the impact of this incorrect initialization in the model performance.
@mthrok if you find the time, it would be great to assess how much this mistake affects performance.

My bad, I misunderstood the comment above the loop:
https://github.com/pytorch/vision/blob/c2e8a00885e68ae1200eb6440f540e181d9125de/torchvision/ops/feature_pyramid_network.py#L59-L63

Can I help with this? If you point me to what model/scripts you used to train I can train with the correct init and the current one.

@gan3sh500 You can find the training scripts here.

I’ll have to train on single 2080ti. I’ve seen smaller batch size create worse convergence but is it fine if I compare with both the kaiming init and otherwise trained with same conditions?

@gan3sh500 sorry for the delay in replying.
If you change the number of GPUs, you'll need to adapt the learning rate to follow the linear scaling rule -- if you divide the global batch size by 8x, you should divide the learning rate by 8x


Remove RELEASE_TAG, which has been removed from downstream jobs

Related with https://github.com/adoptium/temurin-build/pull/2166/
https://github.com/adoptium/aqa-tests/pull/2035
Signed-off-by: Sophia Guo<EMAIL_ADDRESS>

Hmm, is this not used by OpenJ9 / Semeru builds or even temurin builds with the _adopt suffix tag?

I believe it's not used by Openj9 / Semeru builds @llxia could you confirm that?
What is the builds with _adopt suffix tag? Are we still running them?

I believe it's not used by Openj9 / Semeru builds @llxia could you confirm that?
What is the builds with _adopt suffix tag? Are we still running them?

I think we can ignore the _adopt suffix tags as that's silently handled now CC @gdams

I believe it's not used by Openj9 / Semeru builds

RELEASE_TAG is removed in aqa-tests repo. I do not think it is used by Openj9 / Semeru builds. FYI @AdamBrousseau

@smlambert could you please take a review? Thanks


terraform provider: update to v3.47.0

Description of your changes
Updates TF provider to v3.47.0 to get the fix for https://github.com/upbound/provider-azure/issues/411
The latest version is v3.49.0 but it contains breaking changes such as required field removal. If we have a process to address those, I'm happy to do it. cc @ulucinar
Fixes https://github.com/upbound/provider-azure/issues/411
I have:

[x] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested
Uptest as part of GH Actions.

/test-examples="examples/dbformysql/database.yaml"

Just chatted with @ulucinar and he suggested that we keep this update for the next release since we'll be cutting the current release today and this is too big of a change last minute.

@muvaf @ulucinar any updates on this? It's becoming a rather urgent problem on our side.

@yhrn we are planning to upgrade the underlying terraform providers with the next release of the Upbound official providers.

I can't commit to a date yet, but likely in the next week or two.


refactor(naming): Consistent names and folder structure

update src/styles folder/files structure and naming to match the adopted methodology (Atomic Web Design). We use this kind of "naming" also for the [documentation] (http://zalando.github.io/dress-code).

close as duplicate of #86


Added Microsoft Edge Beta version 92.0.902.30.

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

/azp run


[Bug]: When a user who is invited to the workspace via a group, tries to leave the workspace, we see Internal server error toast message

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When a user who is invited to the workspace via a group, tries to leave the workspace, we see Internal server error toast message. We need to update the error message in this case that will notify the user he/she wouldn't be able to leave the workspace since he/she hasn't been added to the workspace directly but gets access to it via the group.
Also, update the cypress tests for the same.

Tests failing links:
https://cloud.cypress.io/projects/n14p1y/runs/4454/test-results/5dfb44a6-9e5b-4476-ba23-d544625b5e21/video
https://github.com/appsmithorg/appsmith-ee/actions/runs/3738015503/jobs/6344075950
Steps To Reproduce

Go to Group edit page on Admin settings page
Add user1 to that group
Assign Developer role for workspace1 to this Group
Now logout and login as user1
You will start seeing workspace1 on the homepage for user1
Click the context menu for this workspace1 and click Leave workspace option
Notice that we see a toast message saying "Internal server error"

Public Sample App
No response
Issue video log
No response
Version
GAC DP

Todays date the leave workspace action can be performed only if the user has a appsmith default roles


Videos sometimes not working on Chromium

Seems to be a problem with an outdated chromium version on dnf.


L

Issue description

Target framework

[ ] .NET Core
[ ] .NET Framework
[ ] .NET Standard

dotnet --info output or About VS info
<replace>

Requesting to add more details

Closing as spam


TYPE DCGM_FI_PROF_ metrics value issue

im testing dcgm exporter version [2.3.6-2.6.6] with mig enabled on k8s
everything seems to work fine except for the profiling metrics.
profiling metritcs also seem to work fine at the beginning but when the dcgm-exporter pod has been  up for several hours the profiling metrics show values that just doesn't make sense.
below is the value dcgm exporter gives when running a tensorflow benchmark on a mig instance.
the dcgm exporter pod has been up for 18 hours during which the mig instance was sitting idle for the first 17 hours and the TF benchmark was running for the last hour. And still running.
# TYPE DCGM_FI_PROF_GR_ENGINE_ACTIVE gauge
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-8cqdc",container="container-testest-1",namespace="gi-user1-giops-ai",pod="testest-9sxmd"} 0.117468

and if i restart the dcgm exporter while keeping the TF job running i get the following value.
# TYPE DCGM_FI_PROF_GR_ENGINE_ACTIVE gauge
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="container-testest-1",namespace="gi-user1-giops-ai",pod="testest-9sxmd"} 0.959228

the difference in the two values are too large to neglect. it seems like the values are not getting flushed for some reason.

@Omoong,
Thank you for the report!
Could you try to do a small experiment - run dcgm-exporter and generate some GPU load for a minute or two. After that keep GPU idle and watch for the DCGM_FI_PROF* metrics for another couple of minutes.
I'm wondering if dcgm-exporter reports the same values it reported in the first minute or two again and again when the GPU is idle.

@nikkon-dev sure thing.
just few more comments on this current issue.
we tested these valuses on dcgmi instead of the exporter about a year ago and the same issue was present back then.
when we monitored the DCGM_FI_PROF* metrics for a long time the change in values were very slow and meaningless like how its is now with the DCGM-exporter.
and we had to restart the dcgmi command to get the instantaneous value. So im hypothesizing that the issue with method  dcgm uses to gather the value for these metrics.
Im still running few more tests as you have requested.
I ran another test (not the test you requested, that i will do after i post this comment), where I left both the dcgm exporter and the job running for about 5 hours. (the initial state is same as my initial post in this issue)
a. job ran first.
b. dcgm exporter showed
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="container-testest-1",namespace="gi-user1-giops-ai",pod="testest-9sxmd"} 0.959228
when it first came up.
so like i mentioned before i left both pods running for 5 hours. and after 5 hours i checked the same metric again and received follwing.
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="container-testest-1",namespace="gi-user1-giops-ai",pod="testest-9sxmd"} 0.955586
which seems to be about right.
next, I stopped the job and read the values again. and received the following
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="",namespace="",pod=""} 0.955019
which shouldn't be right.... and since there could be late responses i waited about 10~15 minutes and read the values again and received. the following
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="",namespace="",pod=""} 0.814772
since no jobs were running for the past 10-15 minutes that cannot be right.
at the same point in time as the last value all the DCGM_FI_PROF are wrong for that instance. as shown in the following.
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="",namespace="",pod=""} 0.133374
DCGM_FI_PROF_DRAM_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="",namespace="",pod=""} 0.580126
but other values are just fine as shown in the following
DCGM_FI_DEV_FB_USED{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="",namespace="",pod=""} 13
DCGM_FI_DEV_FB_FREE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-4flp2",container="",namespace="",pod=""} 11954
and then the values as soon as i restart the dcgm pods
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.000000
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.000000
DCGM_FI_PROF_DRAM_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.000000
now i will try the test you requested

@nikkon-dev
this is the test you requested.

kubectl rollout restart ds dcgm-exporter
curl <IP_ADDRESS>:9400/metrics

DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.000000
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.000000
DCGM_FI_PROF_DRAM_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.000000

started a tf-benchmark job on the same instance.
curl <IP_ADDRESS>:9400/metrics (immediately, notice how the dcgm didnt pick up the tf-benchmark pod)

DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.381124
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.022137
DCGM_FI_PROF_DRAM_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.098690

curl <IP_ADDRESS>:9400/metrics (its after a few minutes but it didnt pick up the k8s pod for some reason...)

DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.966660
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.159025
DCGM_FI_PROF_DRAM_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.549615

stop the tf-benchmark job
curl <IP_ADDRESS>:9400/metrics (values immediately after killing the job)

DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.938835
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.156037
DCGM_FI_PROF_DRAM_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.549615

nvidia-smi (immediately after quiting the tf-benchmark job)

|==================+======================+===========+=======================|
|  0    1   0   0  |     13MiB / 11968MiB | 28      0 |  2   0    2    0    0 |
|                  |      0MiB / 16383MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+

curl <IP_ADDRESS>:9400/metrics (after letting it sit idle for few minutes)

DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.000000
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.000000
DCGM_FI_PROF_DRAM_ACTIVE{gpu="0",UUID="GPU-603a8873-0edd-c157-ce48-2078e8b1a935",device="nvidia0",modelName="NVIDIA A30",GPU_I_PROFILE="2g.12gb",GPU_I_ID="1",Hostname="dcgm-exporter-sn5rp",container="",namespace="",pod=""} 0.549615

it seemed like the some values were flushed out and some weren;t flushed out.

@Omoong,
Thanks for the tests. Unfortunately, it's challenging to spot a pattern if there is just a single point.
Do you have graphs that could help see the pattern in time?
Another option would be to directly run dcgmi dmon -d 500 -e 1001,1004,1005 to collect values every half second (you may change to -d 1000 to collect every second). Continous values output when the GPU is idle will help see the pattern because that's not just one stuck, not flushed value, but the whole series +- some delta.

I almost forgot an important question - what is your driver version and GPU name? nvidia-smi output should be enough.

@nikkon-dev
im tyring to run dcgmi on the base os. and the profiling module is failing to load...
libdcgmmoduleprofiling.so
libdcgmmoduleprofiling.so.2
libdcgmmoduleprofiling.so.2.3.6
are all  under /usr/lib/x86_64-linux-gnu
any ideas?

@nikkon-dev
and im using
NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6
for nvidia driver on an asus server with two A30's

@Omoong,
Regarding the initial issue - please check the just-released DCGM 2.4.5 - it should have a fix for "ghost" MIG metrics.

As for BaseOS - the profiling module needs nv-hostengine run as root.
If it fails to load even with the root permissions, please collect the debug logs from the nv-hostegine:

Stop nvidia-dcgm service (if running)
As root, run nv-hostengine -f /tmp/nv-hostengine.debug.log --log-level debug
Run dcgmi dmon -e 1001
That should be enough to trigger the profiling module loading.
We will need to see the /tmp/nv-hostengine.debug.log after that.

@nikkon-dev Thx,, I will try to get some graphs using grafana and see if i get the same values with dcgm on baseOS
Also, we had the same issue on BaseOS when the dcgm dmon command was running for a long time. I will also check if that also happens

@nikkon-dev
hi im finally back with the test results.

okay so there were quite a bit of changes.

we used the updated version of dcgm exporter in this test.
a, before we used 2.3.6-2.6.6
b. in this test we used 2.4.5-2.6.7
instead of running dcgm with the hostengine running inside the container, we installed the dcgm on the baseOS and used -r option inside the container to connect the exporter to the nv-hostengine on the baseOS.

So as you can see in the picture. GRACT(1001) seems to work fine now.
but DRAMA(1005). still seems to give wrong values.

the second image is the result of dcgmi dmon and curl on the exporter consecutively.
i know the profile is wrong on the dcgmi dmon but stilll it should show some values on gpu0 if the dcgmi dmon picked up values for gpu0 instance1 but it shows 0

I have the same problem. I have a mig partition of 1g 1g 2g 3g for one A100 gpu, and using a gpu_burn to make all of them full load, and here is the result of DCGM_FI_PROF_GR_ENGINE_ACTIVE:

DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="3",device="nvidia3",modelName="NVIDIA A100 80GB PCIe",GPU_I_PROFILE="1g.10gb",GPU_I_ID="9",DCGM_FI_DRIVER_VERSION="515.65.01"} 0.999980
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="3",device="nvidia3",modelName="NVIDIA A100 80GB PCIe",GPU_I_PROFILE="1g.10gb",GPU_I_ID="10",DCGM_FI_DRIVER_VERSION="515.65.01"} 0.000000
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="3",device="nvidia3",modelName="NVIDIA A100 80GB PCIe",GPU_I_PROFILE="2g.20gb",GPU_I_ID="3",DCGM_FI_DRIVER_VERSION="515.65.01"} 0.404050
DCGM_FI_PROF_GR_ENGINE_ACTIVE{gpu="3",device="nvidia3",modelName="NVIDIA A100 80GB PCIe",GPU_I_PROFILE="3g.40gb",GPU_I_ID="2",DCGM_FI_DRIVER_VERSION="515.65.01"} 0.333328


Introduce mac os for installer

fix #107

I add @jeppojeps as reviewer for the brew lines, I haven't worked with it, but it looks good and simple. Anyway Antonio check it out if you can.

The PR does not have brew commands the issue is that we need to install grossomodo the same packages we do on linux instead that via apt, via brew. So we expect the user to have brew up and running. And very likely the XCode CommandLineTools