Nuxt Layers - VSCode auto-imports

Environment
Monorepo root
------------------------------
- Operating System: `Linux`
- Node Version:     `v18.15.0`
- Nuxt Version:     `3.3.2`
- Nitro Version:    `2.3.2`
- Package Manager<EMAIL_ADDRESS>- Builder:          `vite`
- User Config:      `-`
- Runtime Modules:  `-`
- Build Modules:    `-`
------------------------------

App and Layer
------------------------------
- Operating System: `Linux`
- Node Version:     `v18.15.0`
- Nuxt Version:     `3.3.2`
- Nitro Version:    `2.3.2`
- Package Manager<EMAIL_ADDRESS>- Builder:          `vite`
- User Config:      `extends`
- Runtime Modules:  `-`
- Build Modules:    `-`
------------------------------

Reproduction

clone this example https://github.com/kleinpetr/nuxt-layers
run pnpm i in the root (it runs nuxt prepare at the end)
open layers/my-layer/nuxt.config.ts (you should see the error as below)

Describe the bug
I am playing with Nuxt Layers and facing some troubles, especially with auto imports inside layers
when I run pnpm build or even nuxi prepare the error message disappears for a while but then returns back as you can see on the gif below

I am using Volar TS Exntension with Takeover mode (I also tried enable builtin TS Language Extension and Disable Volar, but the result is still the same)
Do you have any suggestion what should be the problem?  (in the /app folder everything works fine, but inside /layers it is not)
Thanks for help :pray:
Additional context
Discussion: https://github.com/nuxt/nuxt/discussions/19963
OS: Linux Fedora 37
VSCode: version 1.76.2
Logs
No response

You probably need to enable includeWorkspace for your types:
export default defineNuxtConfig({
  typescript: {
    includeWorkspace: true
  }
})

Thank you @kleinpetr for the pointers!

if you use typecheck, do not enable includeWorkspace , It will check all packages and not use their own tsconfig files


novacustom: improve recovery guides

Reattach the bottom cover (without screwing it in) and flip the laptop over

In order to do the next step (removing the keyboard), the bottom cover should remain unattached. Otherwise, it's hard to push the screwdriver into the hole.

pre-commit is failing :\


haskellPackages.ghcide: Fix build

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

btw. I want to try out to not force push a single commit again and again in a PR. I think it is much better to only rebase on merge conflicts and otherwise make a squash commit on merge.


Remove unnecessary Equatable constraint.

Removes the unnecessary Equatable constraint on stored items as we can make use of the cacheIdentifier instead, which is already supposed to uniquely identify the object.

Hmm... This test is passing for me locally. 🤔

I figured out what's going on.
It looks like your reset function dispatches work using an async Task. this results in a race condition as you are calling reset during your unit tests setup.

Perhaps a change to .reset() is needed? 🤔
Alternatively a change could be made to the tests. You could put the @StoredValue inside each test.

Thank you so much for the help @connor-ricks! As we discussed elsewhere I've gone ahead and made some changes today to address both the Equatable conformance and the @MainActor test flakiness. I think the change to remove Equatable conformance is a good one, and going forward @StoredValue operate on @MainActor which means we shouldn't run into this test flakiness.
Thanks again, sorry it took me a bit to get to, but I'm really grateful for the contribution!


fix: adding mode for graphql syntax with dynamic bindings

Description
Adding mode for dynamic binding with quotes for graphql query syntax.
Fixes #16702
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Manually
Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

/ok-to-test sha=34c11dd

/ok-to-test sha=db6db2c

/ok-to-test sha=1d88366

Adding Test Plan Approved label as the issue is moved to Done pipeline.


feat(mirror): add mirroring between database and files

Which Problems Are Solved

Mirror command does not support migrations between a database and files

How the Problems Are Solved

Add global variables:

isSrcFile, isDestFile: bool to determine if src/dest is file
filePath: string describing path to directory

Add error checking for mirroring to/from files in PersistentPreRun of mirror command
Add functions in system.go, auth.go, event_store.go, and verify.go to handle mirroring to/from files

Additional Changes

Extend mirror documentation to include file mirroring
Extend defaults.yaml to include file field
Update mirror commands' descriptions to include file mirroring
Use database.Statement for SQL statements instead of strings
Made variable names, error messages, and line spacing consistent

Additional Context

Closes #7966
Follow-up for PR #7004

Hi @adlerhurst ,
No worries! Thank you for taking the time to review our changes :)

hi @andvvo during the latest review of the PR I found an issue that it is not possible to mirror the data from crdb to files to postgres. I will spend some time during this week to propose how we could continue on here. Thanks for your patience

Hi @adlerhurst ,
Your proposal PR looks good to me. Would you like me to start working on your findings?

Hi @adlerhurst ,
Your proposal PR looks good to me. Would you like me to start working on your findings?

If you have the time would be great 😊


Add double gauss lens assembly

Object that groups together lens primitives to form a double gauss lens assembly.
The spacing of elements is determined by operations on the group rather than operations on individual elements, e.g. zoom, focal length, etc.

This could be implemented as a Node where sub lens elements are simply added to self by the init. Users may wish to manipulate transforms of the sub-elements directly so we won't restrict this. However access is intended to be done via the api.

This doesn't seem urgent for V0.4. Prefer not to hold up the next release. Moving back to v1.0.


Request: add bikehike.org + others to the filter

All sites:

https://bikehike.org/what-muscles-does-cycling-tone/
https://allfamousbirthday.com/faqs/does-cycling-give-you-a-big-bum/
https://iron-set.com/us/how-do-you-build-your-outer-thigh-muscle/
https://wikipedikia.org/is-20-minutes-of-cycling-a-day-enough/

Original: https://www.welovecycling.com/wide/2018/06/21/embracing-strong-legs-how-cycling-does-doesnt-change-a-womans-body/

Thanks a lot for your contribution.
You might suggest these websites to :

uBlock filters – Annoyances (https://github.com/uBlockOrigin/uAssets/blob/master/filters/annoyances.txt)
Fanboy’s Annoyance (https://easylist.to)
AdGuard Annoyance (https://github.com/AdguardTeam/AdguardFilters#adguard-filters)

Ah, my bad. I missed this was mostly for Github/SO copycat sites. I was just looking for a way to ban sites like these from my own searches, but I can edit that in myself.
Thanks for your work!

Also, you can use https://letsblock.it/filters/search-results to generate filters for website you want. (You can also use my tool in /src/, but it is not as easy)

I actually did find that on your readme just now! I should have read it more closely the first time around, haha. I just got excited about this tool. I really appreciate it and am certainly putting it to good use.


fix required flags not being uptodate

Signed-off-by: Alexander Trost<EMAIL_ADDRESS>Resolves #12

Merging to fix the current unable to run the exporter issues.


Regenerate all .NET VCS path curations

Please ensure that your pull request adheres to our contribution guidelines. Thank you!

Closing this for now, as the tasks have to be adapted.


Doctors model

In this PR, the following were implemented as the project requirements:

[x] Create the doctor model
[x] Add validations
[x] Add the associations
[x] has many users
[x] has many reservations

Hi @SiansandaTamara, Thank you for your approval.


Search box report a lot of CustomConfig and not asset

PROVISIONING NOTE: If you are having issues with the SharePoint look book while trying to provision M365 learning pathways please do not post that issue to this repo as we cannot help you here. Please post all questions and issues for the SharePoint look book to SharePoint/sp-provisioning-service GitHub Issues List.

WEB PART NOT LOADING: If you are getting the error: "Microsoft 365 learning pathways has a configuration issue. Ask your administrator for assistance". Please review the Diagnosing Pre-Check guidance before submitting your bug.

Describe the bug
When searching content, the result returned is a lot of CustomConfig and not pages of actual assets
To Reproduce

Click on the search "Search this site"
type a keyword
this will display a lot of CustomConfig instead of the content.

Expected behavior
The result should return the assets
Screenshots

Learning Pathways version number
<IP_ADDRESS>
Additional context
Learning Pathway was deployed on 23/03/2021

Hi @wackou72
This is expected as you are using SharePoint Search in your screenshot and the Learning Pathways Web Part loads the majority of its pre-packaged content from other locations such as the Office Support Site. The content stored locally are the configuration lists you mentioned and the Site Pages from the navigation menu and hero web parts. Additionally, if you have created your own custom content and stored it on the site, it should also be searchable.
SharePoint Search Overview
The Learning Pathways Web Part does have its own search function which can be accessed via the magnifying glass icon in the upper left corner of the web part.

Hello,
Thanks for your help.
I understand the technical part, but its not very user friendly.
Regards

Unfortunately, we have absolutely no way to effect search in the way you want as the content is not part of your tenant. You can however configure the custom lists to not be shown in search results if you feel that would be beneficial in your environment.
Option 2 in this article explains how to do so.

User provided feedback on the information given
No further actions at this time
Close ticket


Link props pathname is double encoded with reserved characters

There is no way to set pathname which has encoded reserved character (; , / ? : @ & = + $)
Version
4.1.1
Test Case
https://codepen.io/wilfrem/pen/MmapNM
Steps to reproduce

Set Link to.pathname encoded value. for example /tag/foo%2Fbar (%2F is encoded /)
render it and check link url

Expected Behavior
link url is /tag/foo%2Fbar
Actual Behavior
link url is /tag/foo%25%2Fbar

Don't encode the values up-front. The history library does this for you. See: ReactTraining/history#442

Don't encode the values up-front?
So, What if the user input is included in the path?
<Link to={"/tag/" +userInputValue}> is correct usage?
if userInputValue is foo/bar, It's not match <Route exact path="/tag/:tagValue" />.
It's match <Route path="/tag/:tagValue" /> but match.params.tagValue is foo, not foo/bar.
Should I post this question to https://github.com/ReactTraining/history/issues ?

If you have a path segment that might include a forward slash, I would use the one or more + symbol.
<Route path="/tag/:tagValue+" />

Path segment is ok.
So, how about question mark?
if userInputValue is foo?bar=baz, <Link to={"/tag/" + userInputValue}> is match to <Route path="/tag/:tagValue+" />. but match.params.tagValue is foo.
<Link to={{pathname: "/tag/" + userInputValue}}> is well match to <Route path="/tag/:tagValue+" /> and  match.params.tagValueisfoo?bar=baz. but if user reload this page, match.params.tagValuebecome tofoo`
It seems necessary to escape only reserved characters (; , / ? : @ & = + $ #).

There is actually some discussion about this in https://github.com/ReactTraining/history/issues/461
I think that I jumped the gun a bit in encoding in history's createPath function. I'm working on a comment about this there, but right now I'm fairly certain that that can be removed without breaking the decoding functionality.

Ok. I watch this discussion.
Thanks.


Busy loop recreating state.json on invalid .pcap files

When uploading an empty or otherwise invalid .pcap file, pkappa2 will try to parse it over and over, creating and deleting a state.json file in the process.

The problem appears to be that the problematic file isn't removed from the mgr.importJobs list on failure.
Here it returns that 0 files were processed when one of the pcaps in the batch fail to parse.
https://github.com/spq/pkappa2/blob/73e39cd5aa79286bcd53d57e04a40b721a6713c4/internal/index/builder/builder.go#L103-L106
The caller only logs the error and continues anyway
https://github.com/spq/pkappa2/blob/73e39cd5aa79286bcd53d57e04a40b721a6713c4/internal/index/manager/manager.go#L362-L365
And eventually tries to remove processed files from the job queue, but since processedFiles is 0 this will go on forever.
https://github.com/spq/pkappa2/blob/73e39cd5aa79286bcd53d57e04a40b721a6713c4/internal/index/manager/manager.go#L404-L410
The error handling of the Builder.FromPcap function seems bogous. Are there any errors where it's useful to retry? Otherwise I'd just ignore the failing files and continue instead of return. All the other early returns need to be looked at as well.

pkappa2 refuses to start after it received an invalid pcap once as well. So the problematic pcaps need to be ignored during startup as well. I don't know if we want to delete them upon receival, but maybe moving into an "invalid" folder for later analysis might be good?
https://github.com/spq/pkappa2/blob/73e39cd5aa79286bcd53d57e04a40b721a6713c4/internal/index/builder/builder.go#L53-L56

i have used some of your patches but left out some that I don't think are needed to fix this and modified some others to have as small an impact as possible.
could you confirm that this is fixed?

It appears to be solved now. Thank you!


add delete icon on measurement item

Can you remove the yarn lock file from the changes?

We should keep yarn.lock changes as we have changes in package.json in sub packages(for now, nl-pacs), do you have any issues from this yarn.lock?

Can you remove the yarn lock file from the changes?

We should keep yarn.lock changes as we have changes in package.json in sub packages(for now, nl-pacs), do you have any issues from this yarn.lock?

Nevermind I was under the impression that any package.json changes for sub packages (nl-pacs) and the associated yarn.lock file wouldn't affect the top-level yarn.lock file for nl-ohif, but apparently it does.

Should we add these changes to nl-ohif-modules?

@shivasuri , @Blackglade , please take a look


Update reference-powershell.md

Clarify use of AADCloudSyncTools when agent is not yet running/configured
For consistency, used the same parameters described in https://docs.microsoft.com/en-us/azure/active-directory/cloud-sync/how-to-troubleshoot
Added link for agent troubleshooting page at https://docs.microsoft.com/en-us/azure/active-directory/cloud-sync/how-to-troubleshoot

@NuAlex : Thanks for your contribution! The author(s) have been notified to review your proposed change.

IMPORTANT: When this content is ready to merge, the reviewer must add #sign-off in a comment or the approval may get overlooked.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


Update Facebook to v3.1 endpoints

#92 Facebook has changed its API version from v2.12 to v3.1. It has also removed the public_profile scope, this permission is available by default. See https://developers.facebook.com/docs/graph-api/changelog/version3.0#login
None of the other providers have made any relevant API changes. I did add some doc links for next time.

Looks fine, why is the VSTS CI failing tho?

Same cause: the tests rely on global machine state. This is always a bad thing for tests. The fix is to change the test project to persist dataprotection keys to a temporary directory. https://docs.microsoft.com/en-us/aspnet/core/security/data-protection/configuration/overview?view=aspnetcore-2.1&tabs=aspnetcore2x#persistkeystofilesystem

I'm am dubious of that explanation. DataProtection was designed to be be shared between multiple instances of an application and should be handling concurrency issues like this. We've used it this way for  years without issue, why the sudden regression?


Style spellchecking

This may be resolved in #257?
(issue in which I'm trying to fix #236 and #224)

I see this is already assigned. I sincerely apologize, but I just discovered this issue, and the weird part is that I actually searched before hand and somehow completely missed this. :(

Closing this in favor of #257.


Cannot connect to nrepl

I Found about Clojure Sublimed today, I think the ideas it implements are fantastic, however I'm not able to connect to nrepl. I have Sublime Text set in Clojure (Sublimed) mode. I tried multiple ports (All these ports are locally accessible) but the plugin keeps failing to connect.
My specs:
Sublime Text 3 build 4143
Leiningen 2.9.5 on Java 15.0.1 OpenJDK 64-Bit Server VM

I wasn't, I completely missed the instruction in the docs.  Thank you


Feat: Provide a Faker Provider interface / Open up make_random_id

For various unit testing it's beneficial to generate some mock ids (and/or arns). There is already a lot of make_random_<name>_id in the code that can be re-used for this.
By providing a Faker Provider (https://faker.readthedocs.io/en/master/), that code can be re-used.
I would like to know if this has some change of being incorporated if I bring a PR for that (before I start the work).

Hi @hans-d! You can seed Moto, that way all ID's/ARN's should be deterministic. See the documentation here:
http://docs.getmoto.org/en/latest/docs/configuration/recorder/index.html#deterministic-identifiers
Note that the docs are talking about using this in the context of the Recorder-feature, but seeding can be done on it's own.
Does that work for your usecase?

It's more about using those ID in other code (not the moto code), that want to use "an" account id, and not needing to setup a full moto for that part of the testing.

Getting something like this in moto:
from moto.organizations.utils import (
    make_random_account_id,
    make_random_org_id,
    ORGANIZATION_ARN_FORMAT,
)

from faker.providers import BaseProvider

class AwsProvider(BaseProvider):
    def aws_account_id(self) -> str:
        return self.aws_organizations_account_id()

    def aws_organizations_organization_arn(self) -> str:
        master_account_id = self.aws_organizations_account_id()
        organization_id = self.aws_organizations_organization_id()

        return ORGANIZATION_ARN_FORMAT.format(master_account_id, organization_id)

    def aws_organizations_account_id(self) -> str:
        return make_random_account_id()

    def aws_organizations_organization_id(self) -> str:
        return make_random_org_id()

and can used somewhere else like:
def test_something(self, _faker: Faker):
        data = call_my_real_fn(account=_faker.aws_account_id())

        assert len(data)==1

Gotcha, I understand.
I would not want that to be part of Moto though, I understand the usefulness, but I don't think it's used by enough people to offset the maintenance cost.

Just what I wanted to check. Will make something separately, building on top of moto.


File sensor not performing file validity checks correctly

Home Assistant release (hass --version):
0.63
Component/platform:
File sensor
Description of problem:
This sensor is not performing file validity check with cv.isfile or using hass.config.is_allowed_path
Expected:
Expect validation.
Additional info:
I am currently performing these validity checks here

Added a PR for it! :smile:

Fast work, cheers!


Update version of IaaS clients

What this PR does / why we need it:
Updates versions of aws, aliyun, azure, openstack, gcp
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
@petersutter I wonder if we should also update the ubuntu image to 20.04?
Release note:

Updated versions of IaaS clients:
- aws client update to 1.23.6
- aliyun client updated to 3.0.117
- azure client updated to 2.36.0
- google cloud sdk updated to 384.0.1
- python-novaclient updated to 17.7.0
- python-glanceclient updated to 3.6.0
- python-cinderclient updated to 8.3.0
- python-swiftclient updated to 3.13.1
- python-openstackclient updated to 5.8.0

I wonder if we should also update the ubuntu image to 20.04

yes makes sense 👍

I wonder if we should also update the ubuntu image to 20.04

yes makes sense 👍

done


[backend]making  connect request api

when brands post collab it will be listed in collab page then logged in influcners can go and apply on that collab and when inlfucners click on apply then a connect request will be sent to brand and brand can view request and either accept or delete it once it gets accepted a hello message will be sent to influcner and brand similar system liek its for influcner model

@Rohit can you please assign me.


Still an issue. It causes a debug exception on debug now.


Collect metrics on webhook calls.

Proposed Changes
Collect metrics per webhook type.
Types of Changes
What types of changes does your code introduce to this project?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes issue #XXXX)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation (correction or otherwise)
[ ] Cosmetics (whitespace, styles...)
[ ] DevOps (Build scripts, pipelines...)

Checklist
Put an x in the boxes that apply. You can also fill these out after creating
the PR. If you're unsure about any of them, don't hesitate to ask on the
mailing list. We're here to help! This is simply a reminder of what we are
going to look for before merging your code.

[x] I have read the CODE_OF_CONDUCT.md document
[x] All tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[x] I have added necessary documentation (if needed)
[x] Any dependent changes have been merged and published in related repositories
[x] I have updated changelog (At the bottom of the release version)
[x] I have squashed all my commits into one before merging

Further Comments
I ran the webhooks suite and dialyzer. They pass. I haven't run the full VMQ suite because of clique issues.

This isn't ready to merge yet. However, I wanted to post what I have to see if this is the right approach. It addresses https://github.com/vernemq/vernemq/issues/1915

@ioolkos I think I should change this, and just add the webhook metrics here: https://github.com/vernemq/vernemq/blob/master/apps/vmq_server/src/vmq_metrics.erl#L681
It kind of conflates the vmq_server and vmq_webhooks model. I could also initialize all webhook metrics on VMQ web hooks startup. My initial strategy was to initialize the metrics on demand, because I wanted to a metric per webhook name (vs webhook "type"). The metrics system isn't really built for dynamically named metrics. So, I think the best route is to statically initialize the webhooks counters in the counters definition function in vmc_metrics. I'll have to leak the webhooks module in because of met2idx.

@varnerac thanks, one thing unclear to me: did you try to use a vmq_metrics_mfa (as, for example, in the bridge plugin or SWC plugin) and found specific reasons against it?
The simplicity of the vmq_metrics_mfa approach let's you just export a metrics/0 function from any custom metrics module, and it'd only be enabled when the plugin is active.
Have you found this too rigid for the Webhooks case?

@ioolkos I think the vmq_metrics_mfa is probably a better approach. I'll work on that.

@ioolkos This one use the metrics MFA callback

@varnerac allright, thanks, looks good.

This broke tests apparently:
rebar3 ct
[...]
===> Compiling apps/vmq_server/test/vmq_publish_SUITE.erl failed
apps/vmq_server/test/vmq_publish_SUITE.erl:8:10: can't find include file "../src/vmq_metrics.hrl"

apps/vmq_server/test/vmq_publish_SUITE.erl:742:19: record metric_def undefined
``

I'll patch


color support for hex

I added the option to set the stroke color using color from the Logo spec
You can use hex for example #ffffff and ffffff both will work as the code will append "#" to the hex if its not at char 0

Forgot to add setx and sety when i did the PR for setxy

Cool, thank you!


Ignore manifest creation

Hi, I generate manifest.json dynamically hence I don't need it in the build. Is there an option to ignore it? Thanks.
A snippet of manifest generation for understanding:
// fetch data, then fetch manifest
const link = document.createElement('link')
link.rel = 'manifest'
link.href = `${import.meta.env.VITE_API_URL}/wb/manifest/json/${store.state.guideToken}/`
document.querySelector('head').appendChild(link)

@Fleeck just configure manifest: false

Thanks, it works!


Overflow Menu Accessibility support

Detailed description

Describe in detail the issue you're having.
Looking at using the overflow menu as we are having accessibility issues. Are there plans to add the keyboard accessible function to the menu?

Per the guideline: (https://www.w3.org/TR/wai-aria-practices/#menubutton and https://www.w3.org/TR/wai-aria-practices/#menu)
We need support for the enter, space, arrows, end and home. The enter/space open the menu and then you follow the menubar guidelines for arrows.
Also I noticed while using the Storybook i cannot tab to get focus on the menu to able to use the menu with keyboard only.

Is this a feature request (new component, new icon), a bug, or a general issue?
General accessibility issue with overflow menus.

Is this issue related to a specific component?
Overflow menu

What offering/product do you work on? Any pressing ship or release dates we should be aware of?
IBM Watson Recruitment, Accessibility deadline of Dec 10, 2018.

Looks like the menu does not focus on the first item as well

Can we get an ETA on fix?

DAP Analysis

Level
Description

V
Elements that use 'role' must reference a valid WAI-ARIA role.

V
The WAI-ARIA role(s) and/or attribute(s) aria-expanded are not valid for the element div. See rule help for guidance.

V
All content must reside within a WAI-ARIA landmark or labelled region role.

V
Widget with role of button must have at least one tabbable element.

FYI, I just merged some a11y improvements to the vanilla Overflow Menu that may help in resolving these DAP issues: https://github.com/IBM/carbon-components/commit/c7bdb3105e128d6e7f3253d25769979e6aa15c37

Thank you @tw15egan !!


📺 PYVISTA_VIRTUAL_DISPLAY #41

See #41

Close by https://github.com/pyvista/pyvista/pull/848#issuecomment-723535988

Close by https://github.com/pyvista/pyvista/pull/848#issuecomment-723535988


sh deadlocking in child fork process due to logging

I recently had an issue where some tests I have which were invoking sh with _bg=True were deadlocking.
After investigating, I noticed this was happening inside the sh library, and related to logging inside of an at-fork handler. This is a minimal reproducible example of the issue:
# Run using python3.9 -mpytest --full-trace -o log_cli=true -s -vv --log-cli-level=DEBUG repro.py
import logging
import os

import sh

def test_sh_log_in_child_fork():
    logger = logging.getLogger()

    os.register_at_fork(after_in_child=lambda: logger.debug("in child"))

    procs = [sh.Command("true")(_bg=True) for _ in range(10)]
    for proc in procs:
        proc.wait()

Hitting this issue does require a bit of scheduling bad luck, I think the background thread needs to be interrupted while the lock is held. My guess is that the log line is being formatted. Then fork() needs to be called from the main thread.
In practice though this seems to not be that rare. In my testing the above reproduction hits it every time, with only 10 iterations. I'm guessing some of the objects being logged take some time to format? I also think instead of using os.register_at_fork() you could shoot yourself in the foot via the preexec_fn argument.
I am not an expert on python internals - but generally calling fork() from a multi-threaded application has some pretty concerning risks. It's realistically not going to be possible to avoid the issue entirely since users will always be able to cause this themselves, but I think the sh library's background thread is making this problem far more likely than it needs to be.
Can this thread avoid logging / other lock-holding-operations?
I think it's also worth noting that at-fork handlers were only added in python3.7 - so they may not be that common yet

Thanks for the detailed report!
The docs you linked to state that:

These calls are only made if control is expected to return to the Python interpreter.

However, that's not actually what happens in this case, since the after_in_child is called even though we start true as a background process. I failed reproduce the issue with sleep 1 instead of true, so it's possible that the issue surfaces when the child process in question exits very fast.
Out of curiosity, what leads you to think that the sh library's background thread causes this problem?

The docs you linked to state that:

These calls are only made if control is expected to return to the Python interpreter.

However, that's not actually what happens in this case, since the after_in_child is called even though we start true as a background process. I failed to reproduce the issue with sleep 1 instead of true, so it's possible that the issue only surfaces when the child process in question exits very fast.

These pre/post fork calls are made because os.fork is used instead of subprocess - subprocess does skip these calls because it 'knows' that after fork no python code is executed before exec (unless you specify a preexec_fn - see below for the warning associated with that) - and it only knows this because it has the fork/exec logic implemented in C. In sh it does need to return to the python interpreter between fork and exec.
https://github.com/python/cpython/blob/22b75d9bef1bffe82bfa1adfcbec0243c9202041/Modules/_posixsubprocess.c#L768-L774
I can't follow how os.fork is registered in cpython, but I'm guessing everything invoking plain os.fork is assumed to return back to python. I think using subprocess, or directly invoking this _posixsubprocess.fork_exec method is the only way to start a new process without risking deadlocks

Out of curiosity, what leads you to think that the sh library's background thread causes this problem?

I had a stacktrace which showed the child process stuck inside a logging related lock, and the background thread is also locking. I was able to reproduce this without sh only by having a background thread also logging.

Thanks for the repro script and report, it was very helpful.

I am not an expert on python internals - but generally calling fork() from a multi-threaded application has some pretty concerning risks.

We can't control if the application using sh is multi-threaded or not, so sh must assume multi-threading in any case. I believe the main risk that sh needs to be concerned about is holding a lock before fork, and then expecting the lock to be opened in the child. The script that you posted demonstrates this problem. Most of what happens in the child between fork and exec is very minimal and mostly about closing down file descriptors, setting process ids, and setting signals. There is very little risk for the child to mess with anything threading or lock related, unless of course a user uses an escape hatch like os.register_at_fork or _preexec_fn. Using those hatches means you're doing pretty advanced stuff and should be prepared for dragons.
I'm inclined to say that the fix here is "don't do that". Although I can see how expecting to use logging within the child, before exec, could be a reasonable thing to expect. But it's not sh's responsibility to let users know that the logging module uses locks. I'm not even sure how we could communicate that risk back to the user, outside of a documentation change?
The only alternative that I see is to disable sh's logging altogether by default, unless a _debug_logging argument is passed explicitly to a command. That way, sh doesn't do any logging, and this deadlock-in-child risk goes away. It's not a bad idea imo, since I've suspected that all the extra sh logging has a performance impact.

In Python 3.12, a DeprecationWarning is issued in os.fork() when the forking process has multiple threads.
There is a well-referenced discussion linked in the docs, but the gist is: if a process that has multiple threads is forking, only the C API functions that are "async-signal-safe" can be used in the child[^1]. CPython does not guarantee it will only use those functions (e.g. it can call malloc), therefore forking while having multiple threads is unsafe.
I don't know what should be done about this (can sh be rewritten to not fork?), but I felt that this was relevant to this issue.
Now, does CPython actually call any unsafe functions in the child specifically while sh is forking but the child has not yet execed? It very much does:
strace -o threaded_fork.strace -ff python -c "import sh; sh.true()"
cat $(grep -l 'execve("/usr/bin/true' threaded_fork.strace.*)
# rm threaded_fork.strace.*

On Linux running CPython 3.11.6 I get the following trace:

trace
set_robust_list(0x74314b494a20, 24)     = 0
gettid()                                = 2805932
clock_gettime(CLOCK_MONOTONIC, {tv_sec=503190, tv_nsec=874804781}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=503190, tv_nsec=875002427}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=503190, tv_nsec=875625044}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=503190, tv_nsec=876523667}) = 0
getrandom("\xf2\x4b\x3c\xd4\x7d\xa1\x3f\xa2\xa0\x35\x69\xa0\x95\x28\x56\x45\x7d\x3d\x52\xa9\x8a\x1f\x0a\x49\x4c\xc1\x97\x41\x23\xe5\x54\x23"..., 2496, GRND_NONBLOCK) = 2496
fcntl(12, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
fcntl(12, F_SETFD, FD_CLOEXEC)          = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK, sa_restorer=0x74314ac5c710}, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK, sa_restorer=0x74314ac5c710}, 8) = 0
getsid(0)                               = 5767
getpgid(0)                              = 2805923
write(10, "5767,2805923", 12)           = 12
ioctl(6, TCGETS, {c_iflag=ICRNL|IXON, c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR, c_cflag=B38400|CS8|CREAD, c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|IEXTEN|ECHOCTL|ECHOKE, ...}) = 0
ioctl(6, TCGETS, {c_iflag=ICRNL|IXON, c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR, c_cflag=B38400|CS8|CREAD, c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|IEXTEN|ECHOCTL|ECHOKE, ...}) = 0
ioctl(6, TCSETSF, {c_iflag=, c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|ONLCR, c_cflag=B38400|CS8|CREAD, c_lflag=ECHOE|ECHOK|ECHOCTL|ECHOKE, ...}) = 0
close(4)                                = 0
close(5)                                = 0
close(7)                                = 0
close(9)                                = 0
close(11)                               = 0
dup2(3, 0)                              = 0
dup2(6, 1)                              = 1
dup2(8, 2)                              = 2
ioctl(1, TIOCSWINSZ, {ws_row=24, ws_col=80, ws_xpixel=0, ws_ypixel=0}) = 0
openat(AT_FDCWD, "/dev/fd", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
newfstatat(4, "", {st_mode=S_IFDIR|0500, st_size=9, ...}, AT_EMPTY_PATH) = 0
brk(0x5a1c358cb000)                     = 0x5a1c358cb000
getdents64(4, 0x5a1c358a2200 /* 11 entries */, 32768) = 264
getdents64(4, 0x5a1c358a2200 /* 0 entries */, 32768) = 0
close(4)                                = 0
close(3)                                = 0
close(4)                                = -1 EBADF (Bad file descriptor)
close(6)                                = 0
close(8)                                = 0
close(10)                               = 0
execve("/usr/bin/true", ["/usr/bin/true"], 0x7fff2b157738 /* 84 vars */) = 0
(rest omitted)

Out of those calls:

gettid is not POSIX, but getpid is async-signal-safe, so I don't see why it wouldn't be also;
getrandom is not POSIX, could be unsafe but I don't know;
rt_sigaction -- sigaction is async-signal-safe, rt_sigaction is the syscall used for it when the kernel provides it;
getsid is not async-signal-safe but we're calling it on ourselves so maybe it's fine?;
getpgid is not async-signal-safe but getpgrp is and it's probably the same syscall;
ioctl is not async-signal-safe;
newfstatat is fsstatat which is async-signal-safe;
brk is no longer POSIX (removed in POSIX.1-2001), probably not async-signal-safe since it's memory stuff, likely this part of code;
getdents64 is listdir which is not async-signal-safe, it's this part of code;
... and the rest are async-signal-safe.

Basically, this was a long way of saying that while the child code after the fork is indeed minimal, it's still unsafe according to POSIX. Whether that's something that needs addressing is another matter -- it seems to work fine in practice and that's what really matters, but perhaps it's something you might want to look into.
Cheers, and thank you for this amazing library!
[^1]: After a fork() in a multithreaded program, the child can safely call only async-signal-safe functions [...] until such time as it calls execve(). -- fork(2) - Linux manual page

@micha030201 Thank you for the deep dive and the excellent summary!

Curious if this is related to the issue I just saw, after upgrading to the new PyCharm 2024.1.
Code using sh and _async=True, which previously worked without issue suddenly fails when the debugger is active, with the output:
  File "<redacted>/lib/python3.12/site-packages/sh.py", line 2160, in __init__
    sid, pgid = (
    ^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)

Appears to be reading the sid/pgid immediately after a os.fork call, and the read() call returns an empty string in this environment, so sh explodes.
I can workaround this by downgrading PyCharm or not activating the debugger. But it seems like there is something unhappy in the intersection of async/fork/sh.

The deprecation warning is displayed when using sh from within pytest:
===== warnings summary =====
tests/foo/test_repo.py::test_repo_is_signed
  /home/jesusa/.local/share/hatch/env/pip-compile/toolkit/EyrTdOhe/test/lib/python3.12/site-packages/sh.py:1980: DeprecationWarning: This process (pid=700399) is multi-threaded, use of fork() may lead to deadlocks in the child.
    self.pid = os.fork()
============================

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

It's possible to disable the warning, but i'm not sure I want to do that just yet. Hopefully someday forking can be configured off in the sh module.

Has anyone used os.posix_spawn as a replacement to fork+exec? It might work as a replacement with minimal changes.

For the record, everything here would need to be converted somehow to posix_spawn


Running as non-root in Kubernetes

This PR defines the USER in the Dockerfile as 10000. When running the image as nonroot in Kubernetes, the resulting pod(s) will have a CreateContainerConfigError. This is because the USER in the image is not defined and assumed to be root.
The work around for this in Kubernetes is to define the user to be some non-zero user. However, as the image does not require any elevated privileges, it would be nice to make it work with the non-root user specification. This is relevant to the Kubernetes Pod Security Admission changes, which require a number of changes to be compliant with the restricted policy.
ref: https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted

Just as a note, it doesn't matter what number the user is as long as it isn't 0. I don't know if there is some specific value that would like to be used instead.


[Issue]: build fails to non-existing llvm path

Problem Description
Tried building hipamd in clr but failing because LLVM_DIR is apparently wrongly set or not set:
From clr/hipamd
mkdir build
cd build
cmake ..
./..'

/usr/lib64/cmake/llvm/../../../bin/clang -O3 --hip-path=/home/nonroot/build-install-scripts/rocm/ROCm-6.1/clr/hipamd/build/include/.. -std=c++17 -nogpulib -isystem /home/nonroot/build-install-scripts/rocm/ROCm-6.1/clr/hipamd/build/include -isystem /include -isystem /home/nonroot/build-install-scripts/rocm/ROCm-6.1/clr/hipamd/include --cuda-device-only --cuda-gpu-arch=gfx1030 -x hip /tmp/hip_pch.3610/hip_pch.h -E
/home/nonroot/build-install-scripts/rocm/ROCm-6.1/clr/hipamd/src/hip_embed_pch.sh: line 148: /usr/lib64/cmake/llvm/../../../bin/clang: No such file or directory

in that script file called hip_embed_pch.sh, the path is set in $LLVM_DIR/bin/clang which itself seems to be called from main CMakeLists.txt.
hip_embed_pch.sh USAGE:
echo "Usage: $(basename "$0") HIP_BUILD_INC_DIR HIP_INC_DIR HIP_AMD_INC_DIR LLVM_DIR [option] [RTC_LIB_OUTPUT]"
This is called from cmake as:
183:  execute_process(COMMAND sh -c "${CMAKE_CURRENT_SOURCE_DIR}/hip_embed_pch.sh
${HIP_COMMON_INCLUDE_DIR}
${PROJECT_BINARY_DIR}/include
${PROJECT_SOURCE_DIR}/include
${HIP_LLVM_ROOT}"
COMMAND_ECHO STDERR RESULT_VARIABLE EMBED_PCH_RC WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
set(HIP_LLVM_ROOT "${LLVM_DIR}/../../..")
HIP_LLVM_ROOT is defined here in same cmake:
179:  if(NOT DEFINED HIP_LLVM_ROOT)
180:    set(HIP_LLVM_ROOT "${LLVM_DIR}/../../..")
but it is unclear what LLVM_DIR should be, by default path of rocm installation, it seems LLVM_DIR=/opt/rocm-6.1.0/llvm which I could pass on to cmake as env param but going back three folders up will land in / which is not going to work.
OS:
NAME="CentOS Stream"
VERSION="9"
CPU:
model name      : Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
GPU:
Name:                    Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
Marketing Name:          Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
Name:                    Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
Marketing Name:          Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
Name:                    gfx908
Marketing Name:          AMD Instinct MI100
Name:                    amdgcn-amd-amdhsa--gfx908:sramecc+:xnack-
Name:                    gfx908
Marketing Name:          AMD Instinct MI100
Name:                    amdgcn-amd-amdhsa--gfx908:sramecc+:xnack-
Operating System
Centos 9
CPU
Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
GPU
AMD Instinct MI100
ROCm Version
ROCm 6.1.0
ROCm Component
clr
Steps to Reproduce
from clr/hipamd/ ; mkdir build ; cd build
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
No response
Additional Information
No response

Hi @jdgh000 I think there should be a /opt/rocm/llvm/lib/cmake/llvm and find_package normally returns that as LLVM_DIR.
Then HIP_CLANG_ROOT is set to  /opt/rocm/llvm.
But it looks like that folder is missing from your installation and find_package returns the system one /usr/lib64/cmake/llvm instead.
I think you probably need to additionally install the rocm-llvm-dev package. This will add the missing directories under /opt/rocm/llvm

i am not understanding your evaluation at all and makes no sense at all!?. I already mentioned where it is having error in my original description.
It is looking for clang nothing else which is installed as evidently seen in :
/home/nonroot/build-install-scripts/rocm/ROCm-6.1/clr/hipamd/src/hip_embed_pch.sh: line 148: /usr/lib64/cmake/llvm/../../../bin/clang: No such file or directory
Regardless, llvm-devel installed with no expectation it will solve and after installation, you were wrong, and I was right, it made no difference.

Hi @jdgh000, LLVM_DIR is a  cache variable set by cmake find_package call here:
https://github.com/ROCm/clr/blob/develop/hipamd/src/CMakeLists.txt#L182
In the find_package call, the PATHS is set to ${ROCM_PATH}/llvm. That means it is expected that the config file is found under /opt/rocm. In that case LLVM_DIR would be set to /opt/rocm/llvm/lib64/cmake/llvm and HIP_LLVM_ROOT,  (the argument passed to the script hip_embed_pch.sh) points to  /opt/rocm/llvm which would make sense.
A note that the above LLVM_DIR cmake variable is not to be confused with the 4th parameter LLVM_DIR that is used in hip_embed_pch.sh and that is actually the HIP_LLVM_ROOT.
In your case find_package(LLVM) instead of returning the llvm configuration under /opt/rocm it returns one found in /usr
/usr/lib64/cmake/llvm. I think that is not expected and is likely the root cause of the problem as it leads to invalid paths later on.
After installing rocm-llvm-dev can you please confirm that  /opt/rocm/llvm/lib/cmake/llvm or (/opt/rocm/llvm/lib64/cmake/llvm) exists on your machine. If it exists please try again with a clean cmake build (LLVM_DIR is a cache variable)
PS. I think the find_package call here https://github.com/ROCm/clr/blob/develop/hipamd/src/CMakeLists.txt#L182
should have set the NO_DEFAULT_PATH to ignore config files found in system paths like /usr and return only what is found under /opt/rocm.

I actually refer back to build instruction for clr. instead of trying to build from hipamd. But same error occurs in cmake stage.
I dont believe it is my job to go into deep debugging session, i think it should be yours to investigate if users can not build following the instruction published:
Stuff you are talking about LLVM_ROOT and other dirs are unfortunately not mentioned in the instruction:
Linux
Clone this repository
cd clr && mkdir build && cd build
For HIP : cmake .. -DCLR_BUILD_HIP=ON -DHIP_COMMON_DIR=$HIP_COMMON_DIR
HIP_COMMON_DIR points to HIP
HIPCC_BIN_DIR points to HIPCC's bin folder. If not provided, it defaults to /opt/rocm/bin.
For OpenCL™ : cmake .. -DCLR_BUILD_OCL=ON
make : to build
make install : to install

do we have update here?

Hi @jdgh000  can you modify your cmake command to this and confirm if it works:
cmake .. -DCMAKE_PREFIX_PATH="/opt/rocm/" -DCLR_BUILD_HIP=ON -DHIP_COMMON_DIR=/home/nonroot/git/HIP
In that case I will follow-up with a fix as setting the CMAKE_PREFIX_PATH should not be required, as per the published instructions.

this appears working however this time, getting another error about max vs. fmax in header:
To erase compaibility issues, i reinstalled rocm6.0 and checkout rocm-6.0.x branch of both HIP and clr and still occurs.
-- Found OpenGL: /usr/lib64/libOpenGL.so
-- HIPCC_BIN_DIR found at /opt/rocm/bin
-- HIP_COMMON_DIR found at /home/nonroot/git/HIP
-- HIPNV_DIR found at
-- Found Perl: /usr/bin/perl (found version "5.32.1")
CMake Error at hipamd/CMakeLists.txt:87 (file):
file STRINGS file "/home/nonroot/git/HIP/VERSION" cannot be read.
-- Found Git: /usr/bin/git (found version "2.43.0")
-- Using CPACK_DEBIAN_PACKAGE_RELEASE local
-- CPACK_RPM_PACKAGE_RELEASE: local%{?dist}
-- HIP Platform: amd
-- HIP Runtime: rocclr
-- HIP Compiler: clang
-- ROCM Installation path(ROCM_PATH): /opt/rocm
-- HIP will be installed in: /opt/rocm
-- Could NOT find Terminfo (missing: Terminfo_LIBRARIES Terminfo_LINKABLE)
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11")
-- Found zstd: /usr/lib64/libzstd.so
-- Found LibXml2: /usr/lib64/libxml2.so (found version "2.9.13")
'sh' '-c' '/root/extdir/gg/git/clr/hipamd/src/hip_embed_pch.sh /home/nonroot/git/HIP/include /root/extdir/gg/git/clr/build/hipamd/include /root/extdir/gg/git/clr/hipamd/include /opt/rocm/llvm/lib/cmake/llvm/../../..'

/opt/rocm/llvm/lib/cmake/llvm/../../../bin/clang -O3 --rocm-path=/root/extdir/gg/git/clr/build/hipamd/include/.. -std=c++17 -nogpulib -isystem /root/extdir/gg/git/clr/build/hipamd/include -isystem /root/extdir/gg/git/HIP/include -isystem /root/extdir/gg/git/clr/hipamd/include --cuda-device-only --cuda-gpu-arch=gfx1030 -x hip /tmp/hip_pch.237/hip_pch.h -E
cat /tmp/hip_pch.237/hip_macros.h
/opt/rocm/llvm/lib/cmake/llvm/../../../bin/clang -cc1 -O3 -emit-pch -triple amdgcn-amd-amdhsa -aux-triple x86_64-unknown-linux-gnu -fcuda-is-device -std=c++17 -fgnuc-version=4.2.1 -o /tmp/hip_pch.237/hip_wave32.pch -x hip-cpp-output -
In file included from /tmp/hip_pch.237/hip_pch.h:1:
In file included from /root/extdir/gg/git/HIP/include/hip/hip_runtime.h:62:
In file included from /root/extdir/gg/git/clr/hipamd/include/hip/amd_detail/amd_hip_runtime.h:432:
/opt/rocm-6.0.0/lib/llvm/lib/clang/17.0.0/include/__clang_cuda_complex_builtins.h:194:30: error: use of undeclared identifier 'max'; did you mean 'fmax'?
double __logbw = std::logb(max(std::abs(__c), std::abs(__d)));
^
/opt/rocm-6.0.0/lib/llvm/lib/clang/17.0.0/include/__clang_cuda_math_forward_declares.h:73:81: note: 'fmax' declared here
static inline attribute((always_inline)) attribute((device)) double fmax(double, double);
^
In file included from /tmp/hip_pch.237/hip_pch.h:1:
In file included from /root/extdir/gg/git/HIP/include/hip/hip_runtime.h:62:
In file included from /root/extdir/gg/git/clr/hipamd/include/hip/amd_detail/amd_hip_runtime.h:432:
/opt/rocm-6.0.0/lib/llvm/lib/clang/17.0.0/include/__clang_cuda_complex_builtins.h:227:29: error: use of undeclared identifier 'max'; did you mean 'fmax'?
float __logbw = std::logb(max(std::abs(__c), std::abs(__d)));
^
/opt/rocm-6.0.0/lib/llvm/lib/clang/17.0.0/include/__clang_cuda_math_forward_declares.h:74:80: note: 'fmax' declared here
static inline attribute((always_inline)) attribute((device)) float fmax(float, float);
^
2 errors generated when compiling for .
CMake Error at hipamd/src/CMakeLists.txt:182 (message):
Failed to embed PCH

I think that the cause for the max vs. fmax error is this:
CMake Error at hipamd/CMakeLists.txt:87 (file):
file STRINGS file "/home/nonroot/git/HIP/VERSION" cannot be read.

Please double check that the path exists and the file permissions are correct.

the file is there, it is unclear how it is resulting, i checkoud HIP as is, no modification done whatsoever but it can not process it:
cat ../../HIP/VERSION
#HIP_VERSION_MAJOR
6
#HIP_VERSION_MINOR
0
#HIP_VERSION_PATCH
32831
[root@localhost build]# cat ../hipamd/CMakeLists.txt  | grep -n VERSION | grep ^87:
87:file(STRINGS ${HIP_COMMON_DIR}/VERSION VERSION_LIST REGEX "^[0-9]+")

are you checking on your end? I am just wondering/surprised why kept thinking this is my environment and check this. This is clearly the issue with your end.
For example, you mention "Please double check that the path to the VERSION file exists and the permissions are correct." but this how it is checkout from your repository snapshot and build fails.

If the file is checkout as it is from the repo, with no modifications, it should work. I can't reproduce the issue on my end.
cmake --version
cmake version 3.22.1

I have hard time relating what you are saying. I already provided logs and version and you still arguing against facts that it should work. What do you mean by that?
"I can't reproduce the issue on my end". This is soooo simple, you just checkout and build using instruction and easily reproducible. What steps did you take? What O/S and rocm version did you use?

Another surpirse is I saw the version file error which I you led me to believe it is causing disrepancy of max vs. fmax. YOu did not provide any rationale response about why it occurs. For me, it makes no sense the failure to read VERSION causing max/fmax error.

Hi @jdgh000,

Another surprise is I saw the version file error which I you led me to believe it is causing disrepancy of max vs. fmax. YOu did not provide any rationale response about why it occurs. For me, it makes no sense the failure to read VERSION causing max/fmax error.

cmake generates a file hip_version.h under the clr build directory that adds definitions based on the information read from the VERSION file. For example:
#define HIP_VERSION_MAJOR 6
#define HIP_VERSION_MINOR 0
#define HIP_VERSION_PATCH 32831

hip_version.h is then used by clang  to setup the right include files for the hip version you are trying to build, among other places this file is being used.
If the version information is missing or is wrong, many compatibility issues can come up and the max vs fmax error we are seeing is one of them.
I am using Ubuntu 22.04.3, rocm 6.0, cmake version 3.22.1
git clone -b rocm-6.0.x https://github.com/ROCm/clr.git
git clone -b rocm-6.0.x https://github.com/ROCm/HIP.git
cd clr && mkdir build && cd build
cmake .. -DCMAKE_PREFIX_PATH="/opt/rocm/" -DCLR_BUILD_HIP=ON -DHIP_COMMON_DIR="/home/iassiour/HIP"
make

Following the steps above, I do not get the file VERSION cannot be read error and the configuration plus build runs successfully.
A note that I am checking out the rocm-6.0.x branch of clr/HIP but makes no difference for me if I use the develop branches and the latest rocm version.

i installed rocm6.0 on ubntu 22.04 but it says missing file make.in but it is  there:
CMake Error at hipamd/CMakeLists.txt:87 (file):
  file STRINGS file
  "~/extdir/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP//VERSION"
  cannot be read.

-- Using CPACK_DEBIAN_PACKAGE_RELEASE local
-- CPACK_RPM_PACKAGE_RELEASE: local
-- HIP Platform: amd
-- HIP Runtime: rocclr
-- HIP Compiler: clang
-- ROCM Installation path(ROCM_PATH): /opt/rocm
-- HIP will be installed in: /opt/rocm
'sh' '-c' '/home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/hipamd/src/hip_embed_pch.sh ~/extdir/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP//include /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/build/hipamd/include /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/hipamd/include /opt/rocm/llvm/lib/cmake/llvm/../../..'
+ /opt/rocm/llvm/lib/cmake/llvm/../../../bin/clang -O3 --rocm-path=/home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/build/hipamd/include/.. -std=c++17 -nogpulib -isystem /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/build/hipamd/include -isystem /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP//include -isystem /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/hipamd/include --cuda-device-only --cuda-gpu-arch=gfx1030 -x hip /tmp/hip_pch.2530/hip_pch.h -E
+ cat /tmp/hip_pch.2530/hip_macros.h
+ /opt/rocm/llvm/lib/cmake/llvm/../../../bin/clang -cc1 -O3 -emit-pch -triple amdgcn-amd-amdhsa -aux-triple x86_64-unknown-linux-gnu -fcuda-is-device -std=c++17 -fgnuc-version=4.2.1 -o /tmp/hip_pch.2530/hip_wave32.pch -x hip-cpp-output -
In file included from /tmp/hip_pch.2530/hip_pch.h:1:
In file included from /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP//include/hip/hip_runtime.h:62:
In file included from /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/hipamd/include/hip/amd_detail/amd_hip_runtime.h:432:
/opt/rocm-6.0.0/lib/llvm/lib/clang/17.0.0/include/__clang_cuda_complex_builtins.h:194:30: error: use of undeclared identifier 'max'; did you mean 'fmax'?
  double __logbw = std::logb(max(std::abs(__c), std::abs(__d)));
                             ^
/opt/rocm-6.0.0/lib/llvm/lib/clang/17.0.0/include/__clang_cuda_math_forward_declares.h:73:81: note: 'fmax' declared here
static __inline__ __attribute__((always_inline)) __attribute__((device)) double fmax(double, double);
                                                                                ^
In file included from /tmp/hip_pch.2530/hip_pch.h:1:
In file included from /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP//include/hip/hip_runtime.h:62:
In file included from /home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/hipamd/include/hip/amd_detail/amd_hip_runtime.h:432:
/opt/rocm-6.0.0/lib/llvm/lib/clang/17.0.0/include/__clang_cuda_complex_builtins.h:227:29: error: use of undeclared identifier 'max'; did you mean 'fmax'?
  float __logbw = std::logb(max(std::abs(__c), std::abs(__d)));
                            ^
/opt/rocm-6.0.0/lib/llvm/lib/clang/17.0.0/include/__clang_cuda_math_forward_declares.h:74:80: note: 'fmax' declared here
static __inline__ __attribute__((always_inline)) __attribute__((device)) float fmax(float, float);
                                                                               ^
2 errors generated when compiling for .
CMake Error at hipamd/src/CMakeLists.txt:182 (message):
  Failed to embed PCH

-- Configuring incomplete, errors occurred!
See also "/home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/build/CMakeFiles/CMakeOutput.log".
root@localhost:~/extdir/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/build# find ~/extdir/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/ -name hip-lang-config.cmake.in
/home/nonroot/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP/hip-lang-config.cmake.in
root@localhost:~/extdir/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/clr/build# cmake .. -DCLR_BUILD_HIP=ON -DHIP_COMMON_DIR=~/extdir/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP/

Also seeing VERSION read fail randomly, the file is  there but why it is not reading? what should be the permission?
Again, i checked out as is in the repo and did not modify anything.

root@localhost:/home/nonroot/git//build-install-scripts/rocm/ROCm-6.0/clr/build# ls -l /home/nonroot/gitcodelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP//VERSION                                                                                -rw-r--r--. 1 root root 67 Jun 25 05:38 /root/extdir/git/codelab-scripts/build-install-scripts/rocm/ROCm-6.0/HIP//VERSION
root@localhost:/home/nonroot/git//build-install-scripts/rocm/ROCm-6.0/clr/build# cmake --version
cmake version 3.22.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).

let me look at it and come back!

@jdgh000 Any updates? Thanks!

I am not seeing clearing the build directory fixes anything, I just retried:

sudo cmake -DCLR_BUILD_HIP=ON -DHIP_COMMON_DIR=$HIP_COMMON_DIR=../../HIP  ..
-- The C compiler identification is GNU 11.5.0
-- The CXX compiler identification is GNU 11.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- HIPCC Binary Directory: /opt/rocm/bin
-- HIP Common Directory: =../../HIP
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found AMD_OPENCL: /root/rocm/ROCm-6.2/clr/opencl/khronos/headers/opencl2.2/CL
CMake Error at rocclr/cmake/ROCclrLC.cmake:37 (file):
  file STRINGS file
  "/root/rocm/ROCm-6.2/clr/rocclr/=../../HIP/VERSION"
  cannot be read.
Call Stack (most recent call first):
  rocclr/cmake/ROCclr.cmake:139 (include)
  rocclr/CMakeLists.txt:30 (include)

-- Found NUMA: /usr/lib64/libnuma.so
-- Found OpenGL: /usr/lib64/libOpenGL.so
-- HIPCC_BIN_DIR found at /opt/rocm/bin
-- HIP_COMMON_DIR found at =../../HIP
-- HIPNV_DIR found at
-- Found Perl: /bin/perl (found version "5.32.1")
CMake Error at hipamd/CMakeLists.txt:88 (file):
  file STRINGS file
  "/root/rocm/ROCm-6.2/clr/hipamd/=../../HIP/VERSION"
  cannot be read.

-- Found Git: /bin/git (found version "2.43.5")
-- Using CPACK_DEBIAN_PACKAGE_RELEASE local
-- CPACK_RPM_PACKAGE_RELEASE: local%{?dist}
-- HIP Platform: amd
-- HIP Runtime: rocclr
-- HIP Compiler: clang
-- ROCM Installation path(ROCM_PATH): /opt/rocm
-- HIP will be installed in: /opt/rocm
-- Performing Test Terminfo_LINKABLE
-- Performing Test Terminfo_LINKABLE - Success
-- Found Terminfo: /usr/lib64/libtinfo.so
-- Found ZLIB: /usr/local/lib/libz.so (found version "1.3.#define ZLIB_VERSION "1.3"")
-- Found zstd: /usr/local/lib64/libzstd.so
-- Found LibXml2: /usr/lib64/libxml2.so (found version "2.9.13")
'sh' '-c' '/root/rocm/ROCm-6.2/clr/hipamd/src/hip_embed_pch.sh =../../HIP/include /root/rocm/ROCm-6.2/clr/build/hipamd/include /root/rocm/ROCm-6.2/clr/hipamd/include /opt/rocm/llvm/lib/cmake/llvm/../../..'
+ /opt/rocm/llvm/lib/cmake/llvm/../../../bin/clang -O3 --hip-path=/root/rocm/ROCm-6.2/clr/build/hipamd/include/.. -std=c++17 -nogpulib -isystem /root/rocm/ROCm-6.2/clr/build/hipamd/include -isystem =../../HIP/include -isystem /root/rocm/ROCm-6.2/clr/hipamd/include --cuda-device-only --cuda-gpu-arch=gfx1030 -x hip /tmp/hip_pch.80988/hip_pch.h -E
/tmp/hip_pch.80988/hip_pch.h:1:10: fatal error: 'hip/hip_runtime.h' file not found
    1 | #include "hip/hip_runtime.h"
      |          ^~~~~~~~~~~~~~~~~~~
1 error generated when compiling for gfx1030.
CMake Error at hipamd/src/CMakeLists.txt:200 (message):
  Failed to embed PCH

-- Configuring incomplete, errors occurred!
[nonroot@SMC-DH144-DC16-U27 build]$  sudo rm -rf ./* ; sudo cmake -DCLR_BUILD_HIP=ON -DHIP_COMMON_DIR=$HIP_COMMON_DIR=../../HIP  ..^C
[nonroot@SMC-DH144-DC16-U27 build]$ find ../../HIP -name hip_runtime.h
../../HIP/include/hip/hip_runtime.h
[nonroot@SMC-DH144-DC16-U27 build]$ ls -l ../../HIP
total 84
drwxr-xr-x.  2 root root    73 Nov  1 15:34 bin
drwxr-xr-x.  3 root root    54 Nov  1 15:34 cmake
-rw-r--r--.  1 root root    51 Nov  1 15:34 CODEOWNERS
-rw-r--r--.  1 root root     0 Nov  1 15:34 configure
-rw-r--r--.  1 root root  8992 Nov  1 15:34 CONTRIBUTING.md
drwxr-xr-x.  2 root root    90 Nov  1 15:34 docker
drwxr-xr-x. 10 root root  4096 Nov  1 15:34 docs
-rw-r--r--.  1 root root  5271 Nov  1 15:34 hip-lang-config.cmake.in
drwxr-xr-x.  3 root root    25 Nov  1 15:34 include
-rwxr-xr-x.  1 root root  1208 Nov  1 15:34 install.sh
-rw-r--r--.  1 root root 16645 Nov  1 15:34 Jenkinsfile
-rw-r--r--.  1 root root  1080 Nov  1 15:34 LICENSE.txt
-rw-r--r--.  1 root root 11115 Nov  1 15:34 README.md
-rw-r--r--.  1 root root  9833 Nov  1 15:34 RELEASE.md
drwxr-xr-x.  4 root root    42 Nov  1 15:34 util
-rw-r--r--.  1 root root    67 Nov  1 15:34 VERSION

i am just following exactly what the clr page says:
https://github.com/ROCm/clr
For HIP:

cmake .. -DCLR_BUILD_HIP=ON -DHIP_COMMON_DIR=$HIP_COMMON_DIR

Hi @jdgh000, thanks for the update! I noticed in the log you provided you had

-DHIP_COMMON_DIR=$HIP_COMMON_DIR=../../HIP

Which seems to be a typo?
It shows in the subsequent log
CMake Error at rocclr/cmake/ROCclrLC.cmake:37 (file):
  file STRINGS file
  "/root/rocm/ROCm-6.2/clr/rocclr/=../../HIP/VERSION

Can you try with
-DHIP_COMMON_DIR=../../HIP
and let us know the output? Thanks!

Hi @jdgh000 thanks for the prompt update! Seems like the issue might with using relative path ../../HIP as your HIP common dir. Can you try using the absolute path to your HIP folder? Thanks!

after setting abs path, appears to be working


Allow Master Records To Be Filtered During Calculate

Add textarea to Calculate screen where a WHERE clause can optionally be
entered, allowing the master records to be filtered.

Very nice, great idea, i know some users have said this can take a while in large orgs. @wes1278 another candidate for the WHERE clause UI! :+1:

I seriously need to find time to work on that again. So many applications for it. The route I was taking before wasn't great and kind of hacky. I've got a new idea that I'll try out and see what comes of it.

Thanks for reviewing and merging so quickly.


runtime: stuck at stopTheWorldWithSema during k8s operations

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.8.3 linux/amd64
Does this issue reproduce with the latest release?
I do not know how to reproduce it. But it occurs triple on different hosts.
What operating system and processor architecture are you using (go env)?
Linux VM-2-19-ubuntu 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
KVM virtual machine , NUMA
GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ubuntu/go"
GOROOT="/opt/go"
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build765569602=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
What did you do?
Run Kubelet(Kubernetes agent) ,Create and Delete Container Repeatly
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
What did you expect to see?
The process runs normally.
What did you see instead?
The process is still there but does not respond anymore(and no more logging). When I use curl http://ip:port/debug/pprof/goroutine?debug=1 to check the stack, but it halts. There is nothing useful in stderr or dmesg.（I have core dump but it too big to upload）
i tried gdb it and got following result:

thread,goroutines stack

(Thread 52 (LWP 4987) )sched.stopwait == 1 and stopTheWorldWithSema stuck on
https://github.com/golang/go/blob/357c9141369361101345f3048a6b2b3e149299d5/src/runtime/proc.go#L1029

Thread 6 (LWP 4622) very strange function address

#0  runtime.futex () at /opt/go/src/runtime/sys_linux_amd64.s:426
#1  0x000000000042acd7 in runtime.futexsleep (addr=0x543e7e8 <runtime.sched+200>, val=0, ns=100000) at /opt/go/src/runtime/os_linux.go:62
#2  0x0000000000410930 in runtime.notetsleep_internal (n=0x543e7e8 <runtime.sched+200>, ns=100000, ~r2=false) at /opt/go/src/runtime/lock_futex.go:174
#3  0x00000000004109f6 in runtime.notetsleep (n=0x543e7e8 <runtime.sched+200>, ns=100000, ~r2=false) at /opt/go/src/runtime/lock_futex.go:194
#4  0x0000000000431704 in runtime.stopTheWorldWithSema () at /opt/go/src/runtime/proc.go:1029
#5  0x00000000004599d9 in runtime.systemstack () at /opt/go/src/runtime/asm_amd64.s:327
#6  0x00000000004319f0 in ?? () at /opt/go/src/runtime/proc.go:1067
#7  0x000000c4218e81e0 in ?? ()
#8  0x00007ff1d67fbb7f in ?? ()
#9  0x000000c4218e81a0 in ?? ()
#10 0x00007ff16dffaef8 in ?? ()
#11 0x0000000000431a54 in runtime.mstart () at /opt/go/src/runtime/proc.go:1149
#12 0x0000000002c13e43 in crosscall_amd64 () at /opt/go/src/runtime/cgo/gcc_amd64.S:35
#13 0x00007ff1d67fbc00 in ?? ()
#14 0x00007ff16dffb9c0 in ?? ()
#15 0x00007ff1d67fbb7f in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 6 (LWP 4622):
---Type <return> to continue, or q <return> to quit---
#0  runtime.futex () at /opt/go/src/runtime/sys_linux_amd64.s:426
#1  0x000000000042ac62 in runtime.futexsleep (addr=0x545f640 <runtime.sig>, val=0, ns=-1) at /opt/go/src/runtime/os_linux.go:45
#2  0x00000000004108b6 in runtime.notetsleep_internal (n=0x545f640 <runtime.sig>, ns=-1, ~r2=false) at /opt/go/src/runtime/lock_futex.go:161
#3  0x0000000000410a8a in runtime.notetsleepg (n=0x545f640 <runtime.sig>, ns=-1, ~r2=224) at /opt/go/src/runtime/lock_futex.go:206
#4  0x0000000000443344 in os/signal.signal_recv (~r0=4410104) at /opt/go/src/runtime/sigqueue.go:116
#5  0x0000000002aaa802 in os/signal.loop () at /opt/go/src/os/signal/signal_unix.go:22
#6  0x000000000045c531 in runtime.goexit () at /opt/go/src/runtime/asm_amd64.s:2197
#7  0x72702d6e74622e7d in ?? ()
#8  0x6f683a7972616d69 in ?? ()
#9  0x6e74622e2c726576 in ?? ()
#10 0x7972616d6972702d in ?? ()
#11 0x000000c422728800 in ?? ()
#12 0x6d6972702d6e7462 in ?? ()
#13 0x697463613a797261 in ?? ()
#14 0x2d6e74622e2c6576 in ?? ()
#15 0x2e7972616d697270 in ?? ()
#16 0x2e2c657669746361 in ?? ()
#17 0x72642e206e65706f in ?? ()
#18 0x742d6e776f64706f in ?? ()
#19 0x74622e656c67676f in ?? ()
#20 0x72616d6972702d6e in ?? ()
#21 0x3a726f6c6f637b79 in ?? ()
#22 0x6361623b66666623 in ?? ()
#23 0x2d646e756f72676b in ?? ()
#24 0x33233a726f6c6f63 in ?? ()
#25 0x6f623b3162363732 in ?? ()
#26 0x6c6f632d72656472 in ?? ()
#27 0x65353832233a726f in ?? ()
#28 0x2d6e74622e7d6538 in ?? ()
#29 0x3a7972616d697270 in ?? ()
#30 0x2e2c657669746361 in ?? ()
#31 0x6d6972702d6e7462 in ?? ()
#32 0x697463612e797261 in ?? ()
#33 0x6e65706f2e2c6576 in ?? ()
#34 0x6f64706f72642e20 in ?? ()
#35 0x6c67676f742d6e77 in ?? ()
#36 0x72702d6e74622e65 in ?? ()
#37 0x61627b7972616d69 in ?? ()
#38 0x646e756f72676b63 in ?? ()
#39 0x6e3a6567616d692d in ?? ()
#40 0x6e74622e7d656e6f in ?? ()
#41 0x7972616d6972702d in ?? ()
#42 0x656c62617369642e in ?? ()
#43 0x702d6e74622e2c64 in ?? ()
#44 0x645b7972616d6972 in ?? ()
#45 0x5d64656c62617369 in ?? ()
#46 0x6573646c6569662c in ?? ()
#47 0x6c62617369645b74 in ?? ()
#48 0x6e74622e205d6465 in ?? ()
#49 0x7972616d6972702d in ?? ()
#50 0x72702d6e74622e2c in ?? ()
#51 0x69642e7972616d69 in ?? ()
#52 0x683a64656c626173 in ?? ()
#53 0x74622e2c7265766f in ?? ()
#54 0x72616d6972702d6e in ?? ()
#55 0x6c62617369645b79 in ?? ()
#56 0x65766f683a5d6465 in ?? ()
#57 0x73646c6569662c72 in ?? ()
#58 0x62617369645b7465 in ?? ()
#59 0x74622e205d64656c in ?? ()
#60 0x72616d6972702d6e in ?? ()
#61 0x2c7265766f683a79 in ?? ()
#62 0x6972702d6e74622e in ?? ()
#63 0x7369642e7972616d in ?? ()
#64 0x6f663a64656c6261 in ?? ()
#65 0x6e74622e2c737563 in ?? ()
#66 0x7972616d6972702d in ?? ()
#67 0x656c62617369645b in ?? ()
#68 0x7375636f663a5d64 in ?? ()
#69 0x6573646c6569662c in ?? ()
#70 0x6c62617369645b74 in ?? ()
#71 0x6e74622e205d6465 in ?? ()
#72 0x7972616d6972702d in ?? ()
#73 0x2e2c7375636f663a in ?? ()
#74 0x6d6972702d6e7462 in ?? ()
#75 0x617369642e797261 in ?? ()
#76 0x7463613a64656c62 in ?? ()
#77 0x6e74622e2c657669 in ?? ()
#78 0x7972616d6972702d in ?? ()
#79 0x656c62617369645b in ?? ()
#80 0x76697463613a5d64 in ?? ()
#81 0x73646c6569662c65 in ?? ()
#82 0x62617369645b7465 in ?? ()
#83 0x0000000000412d4f in runtime.writebarrierptr_prewrite1 (dst=0xc4209b9ac0, src=4271597) at /opt/go/src/runtime/mbarrier.go:189
#84 0x000000c420e7d068 in ?? ()
#85 0x000000c4209b9ac0 in ?? ()
#86 0x0000000000412ded in runtime.writebarrierptr (dst=0x5, src=3) at /opt/go/src/runtime/mbarrier.go:211
#87 0x0000000000412d4f in runtime.writebarrierptr_prewrite1 (dst=0xc4218c9650, src=48) at /opt/go/src/runtime/mbarrier.go:189
#88 0x0000000000411e24 in runtime.mallocgc (size=48, typ=0x3564540, needzero=true, ~r3=0xc420d81000) at /opt/go/src/runtime/malloc.go:723
#89 0x0000000000412418 in runtime.newobject (typ=0x3564540, ~r1=0xc4209b9f60) at /opt/go/src/runtime/malloc.go:820
#90 0x000000000040a90c in runtime.makemap (t=0x34a04e0, hint=0, h=0x0, bucket=0x0, ~r4=0x300000002) at /opt/go/src/runtime/hashmap.go:286
#91 0x00000000006b1bba in net/http.Header.clone (h=0xc4218c94a0, ~r0=0x8) at /opt/go/src/net/http/header.go:63
#92 0x000000000067bdd4 in net/http.(*Client).makeHeadersCopier (c#4=0xc4205ebc50, ireq=0xc420482700,
    ~r1={void (struct net/http.Request *)} 0xc4209b9df8) at /opt/go/src/net/http/client.go:622
#93 0x000000000067ad91 in net/http.(*Client).Do (c=0xc4205ebc50, req=0xc420482700, ~r1=0x0, ~r2=...) at /opt/go/src/net/http/client.go:500
#94 0x0000000001c0dbdf in k8s.io/kubernetes/vendor/github.com/docker/engine-api/client/transport/cancellable.Do.func1 (client=..., req=0xc420482700,
    result=0xc420d38660)
    at /Users/yu/work/docker/k8s-dev/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/docker/engine-api/client/transport/cancellable/cancellable.go:49
#95 0x000000000045c531 in runtime.goexit () at /opt/go/src/runtime/asm_amd64.s:2197
#96 0x00007ff1df4a8a38 in ?? ()
#97 0x000000c4205a4120 in ?? ()
#98 0x000000c420482700 in ?? ()
#99 0x000000c420d38660 in ?? ()
#100 0x0000000000000000 in ?? ()```

Please mention the complete repro steps, including how to download and install the package and the commands used to create, delete containers.
Also, can you try with latest Go 1.10.2 ?
Thanks.

@agnivade
Here is my script, create, delete Pods repeatly(30000 Pods each time) in a k8s cluster with 800 *(32U64G) Nodes.
I can't change go version this cluster, but I will create a 100 Nodes(1U2G) cluster for reproducing the error, and then try with latest Go 1.10.2.
I found the backtrace very strange, maybe  any other thread broken the stack?
could you give me any suggestions or inspiration?
#77 0x6e74622e2c657669 in ?? ()
#78 0x7972616d6972702d in ?? ()
#79 0x656c62617369645b in ?? ()
#80 0x76697463613a5d64 in ?? ()
#81 0x73646c6569662c65 in ?? ()
#82 0x62617369645b7465 in ?? ()
#83 0x0000000000412d4f in runtime.writebarrierptr_prewrite1 (dst=0xc4209b9ac0, src=4271597) at /opt/go/src/runtime/mbarrier.go:189
#84 0x000000c420e7d068 in ?? ()
#85 0x000000c4209b9ac0 in ?? ()
#86 0x0000000000412ded in runtime.writebarrierptr (dst=0x5, src=3) at /opt/go/src/runtime/mbarrier.go:211
#87 0x0000000000412d4f in runtime.writebarrierptr_prewrite1 (dst=0xc4218c9650, src=48) at /opt/go/src/runtime/mbarrier.go:189

Thanks for your help!

Ok, so you are using a python client to interact with a kubernetes cluster ?
Please correct me if I am wrong, but this seems more like a kubernetes bug rather than a Go bug. You may get more help filing an issue on the kubernetes repo.

@agnivade
yes,I am using a python client to interact with a kubernetes cluster.
Could you tell me why are you think  this seems more like a kubernetes bug?
I found that  "runtime.writebarrierptr (dst=0x5, src=3)" want assign 3 to  virtual memory address 0x5 which is not mapped, and  address 0x62617369645b7465 is not mapped too(not exists in page table)
Do you tend to think of the stack memory modified by k8s?
#80 0x76697463613a5d64 in ?? ()
#81 0x73646c6569662c65 in ?? ()
#82 0x62617369645b7465 in ?? ()
#83 0x0000000000412d4f in runtime.writebarrierptr_prewrite1 (dst=0xc4209b9ac0, src=4271597) at /opt/go/src/runtime/mbarrier.go:189
#84 0x000000c420e7d068 in ?? ()
#85 0x000000c4209b9ac0 in ?? ()
#86 0x0000000000412ded in runtime.writebarrierptr (dst=0x5, src=3) at /opt/go/src/runtime/mbarrier.go:211
#87 0x0000000000412d4f in runtime.writebarrierptr_prewrite1 (dst=0xc4218c9650, src=48) at /opt/go/src/runtime/mbarrier.go:189
#88 0x0000000000411e24 in runtime.mallocgc (size=48, typ=0x3564540, needzero=true, ~r3=0xc420d81000) at /opt/go/src/runtime/malloc.go:723

The stack trace in https://github.com/golang/go/issues/25207#issuecomment-385899814 is not a meaningful trace.  Frames 77 to 82 are clearly corrupt, so we can trust anything that it reports.  We can't draw any conclusions from the writebarrierptr report; there is no reason to think that function was called with those arguments.

@agnivade
yes,I am using a python client to interact with a kubernetes cluster.
Could you tell me why are you think this seems more like a kubernetes bug?
Do you tend to think of the stack memory modified by k8s?

Could you tell me why are you think this seems more like a kubernetes bug?

I don't have much experience with kubernetes, but since you are using a python client, I believe your stack trace is coming from the kube-agent binary ? Therefore, the issue does not lie in any code you have written.
Hence, following that logic, it is an issue with the software and not the programming language it was written in. (Something similar to if you see an issue with nginx, it is likely an issue with nginx and not with C compiler). That does not say it cannot be an issue with Go. It can, but this needs detailed debugging by the kubernetes team or by people having intimate knowledge about it. That's why I suggested to file an issue on the kubernetes repo.

@agnivade
OK, thanks for your suggestion, and I will open an issue on the kubernetes repo too.
Now I open issue on  Go repo  rather than kubernetes, because I thank kubernetes  should not modify the process stack, and the corrupt stack  does not cause the program to crash.

@ianlancetaylor
Thanks for your analysis.
I found the address in frames are illegal and not mapped, may frames be modify by any other(k8s bug) or go runtime bug?
Could you give me any suggestion for further debug?

HI @guangyouyu - Did you get any updates from the kubernetes team on this ? Have you tried with 1.10 ?


A blank message shows while the VM name exceeds its maximum string length

Steps to reproduce:
Create a VM filling a super long name, then click the save button
Result:
Unable to create a VM, along with a blank error message

Validated


[presto_check_op] PrestoCheckOperator with conn_id

Cleaning up previous PR mess with a clean branch.
@mistercrunch

super!


"Got stream request for unknown or finished stream" warning every 10 s in my log

What version of Music Assistant has the issue?
2022.8.2
The problem
I have this warning message repeating in my log every 10 secs for a couple of days already:
WARNING (MainThread) [music_assistant.stream] Got stream request for unknown or finished stream: 7ba7700c824a4108a4b601b211305065

How to reproduce
I am not exactly sure how to reproduce it, but I was having issues streaming to my browser_mod media player previously. After reloading the MA integration I managed to stream to my browser, then stopped streaming - probably that's when those warnings started appearing.
Reloading the MA integration does not stop the messages in log.
Relevant log output
No response
Additional information
No response
What version of Home Assistant Core are your running
2022.8.4
What type of installation are you running?
Home Assistant OS
On what type of hardware are you running?
Raspberry Pi

I fixed a small issue yesterday that might also fix this one. Check patch version 2022.8.3.
If the issue persists than have a look if you still have a rogue browser mod player somewhere

Updated MA to 2022.8.3.
So far I see no such behavior anymore. I will close this issue after a couple of days of monitoring.

Hi @paulius2k . Is this fixed?

not repeating anymore, closing


How to view APbev/AP3D in the results

Why is there no AP3D in this “outputs_07.txt” file?
Thank you for participating in our evaluation!
Loading detections...
number of files for evaluation: 3769
done.
save ./saves/pixor_e2e/predicted_label_eval/plot/car_detection.txt
car_detection AP: 0.000000 0.000000 0.000000
Finished 2D bounding box eval.
Going to eval ground for class: car
save ./saves/pixor_e2e/predicted_label_eval/plot/car_detection_ground.txt
car_detection_ground AP: 77.831551 59.772270 54.277523
Finished Birdeye eval.
Finished 3D bounding box eval.
Your evaluation results are available at:
./saves/pixor_e2e/predicted_label_eval

ok! Thank you for your reply


User can login

Through the UI, a user can log in using the same account as they would on the CLI.

This may be in reference to the embedded use case, which will be covered with another issue (something about fixing embedded auth)


Add some configs to StackedBarChart

on StackedBarChart:
added barPercentage
added hideBarLegend
added barRadius

No need to update the version here, however I am gonna need you to update index.d.ts and readme.

Thank you! this is now available in 4.3.0


null value for a slot

I wonder if I use the null value for a slot value in the right way. Let's say you want to trigger a specific acton if a specific entity value is not set, independent if another slot ist set. So:
* greet
  - utter_greet
  - utter_ask_help
* cost{"entity1": null}
  - slot{"entity1": null}
  - utter_something

So, a content of cost{"entity2": "here is something"} should underly above story because entity1 is null, right?
But I have the feeling that you have to train the situation with cost{"entity2": "here is something"} separately?

So what you're asking is not possible in that way.
but for your scenario you could do something like:
## no entity
* cost
  - utter_something

## entity
* cost{"entity1":"example"}
  - slot{"entity1":"example"}
  - utter_something_else

And if you want to distinguish between stuff like val1, val2 you could use categorical slots.

But what is the difference between:
* cost{"entity1": null}
  - slot{"entity1": null}
  - utter_something

and
* cost
  - utter_something

This should be same for one entity. But I have two entities. That is why I wanted to use
* cost{"entity1": null}
  - slot{"entity1": null}
  - utter_something

in the sense that this is right for
* cost{"entity2": "val2}
  - slot{"entity2": "val2}
  - utter_something

because entity1 is not set (null), right?

If the type of the slot os set to text you should get exactly that behaviour: a difference between null and any value, but the value itself doesn't matter as long as it is not null.

@tmbo tahnk you but I cannot follow you statement. Maybe you can reformulate what you mean. For a text slot is the follwoing the same?
*Kosten{"entity1": null}

and just
*Kosten

But the first one just excludes the case when entity1 is null, right? So if anoter entity2 is not null it falls in the first class wheresas the second cas excludes both cases (all entity values have to be zero), right?


mac chrome not working

all demos not working on newlast mac os and chrome.

Could not reproduce on Chrome 39 on Yosemite at http://filamentgroup.github.io/fixed-sticky/demos/demo.html. Please reopen if you have more details about your environment.


fix(docs): usages of brand color

Fixed usages of brand color in the tutorials and docs

Warnings

:warning:
There are no updates provided to CHANGELOG. Ensure there are no publicly visible changes introduced by this PR.

  Generated by :no_entry_sign: dangerJS


Expected response code 250 but got code 550

I tried using Magic Forms on one of my webpages. I get the following error:
"Expected response code 220 but got code 550, with message 550 5.7.1 relaying denied " on line 456 of /var/www/vhosts/hostname.nl/httpdocs/cms/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php
"
Is this a mistake on my part. I am totally new to October CMS.

I don't know if it helps, but I got that when my server (mailtrap for development purposes), wasn't set right. So maybe that is the issue?

It certainly is a server issue, judging by the code 550. However, October CMS is too confusing and I am struggling with this for the past 4 days.

I don't know, that happened to me recently and it was the mail configuration problem for me, not the server. I didn't get my settings right at first. Check if your Mai method, SMTP port, an SMTP encryption protocol, Username, and Password are correct in Settings->Mail configuration, that helped me. Either way, good luck!


Hide private metadata in notification payloads

Private metadata shouldn't be visible in the notification payload. We cannot remove it due to backward compatibility, instead, it will always return an empty dict.
Impact

[ ] New migrations
[ ] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations
[ ] Documentation needs to be updated

Pull Request Checklist

[ ] Privileged queries and mutations are guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migration files are up to date
[ ] The changes are tested
[ ] GraphQL schema and type definitions are up to date
[ ] Changes are mentioned in the changelog

Task linked: SALEOR-7046 The notification payload contains private_metadata for User and Order notifications