Delegated auth: Delegated client identifier cannot be located

Error when using pac4j authentication delegation after
login submission when the authentication flow goes back
from delegated authentication to authentication server.
The authentication server generates the following error:
ERROR [org.apereo.cas.web.DelegatedClientWebflowManager] - <Delegated client identifier cannot be located in the authentication request [https://mycasserver.com/cas/login?client_name=PERSONAS&ticket=ST-2827--XXX

This is due because DelegatedClientNavigationController uses
a JEESessionStore (DelegatedClientNavigationController
is used before redirecting to delegated server) while
DelegatedClientAuthenticationAction uses DistributedJ2ESessionStore
(DelegatedClientAuthenticationAction is used when cas server is handling
the redirect back from delegated server).
DelegatedClientNavigationController save some data [1] in JEESessionStore,
while DelegatedClientAuthenticationAction tries to get the data [2]
in DistributedJ2ESessionStore (hence a different session).
This issue is because DelegatedClientNavigationController bean is not
created by DelegatedAuthenticationWebflowConfiguration.
[1] see DelegatedClientWebflowManager.store
[2] see DelegatedClientWebflowManager.getDelegatedClientId

Hello, is there something I can do to make this PR progress ?
It has just been marked staled without further feedback
Thanks,
Adrian

DelegatedClientNavigationController save some data [1] in JEESessionStore,
while DelegatedClientAuthenticationAction tries to get the data [2]
in DistributedJ2ESessionStore (hence a different session).
[1] see DelegatedClientWebflowManager.store
[2] see DelegatedClientWebflowManager.getDelegatedClientId

Can you explain this in more detail? Where is JEESessionStore used?

Can you explain this in more detail? Where is JEESessionStore used?

Sure, I'll try to explain why DelegatedClientNavigationController is using JEESessionStore instead of DistributedJ2ESessionStore
DelegatedClientNavigationController bean can be produced from 2 locations:

from @Controller annotation in DelegatedClientNavigationController class itself
see https://github.com/apereo/cas/blob/3225e7cbe7f3a7afdea8a9bfab228858287e2481/support/cas-server-support-pac4j-webflow/src/main/java/org/apereo/cas/web/DelegatedClientNavigationController.java#L40-L44
Since it's a Spring bean, Spring will inject the dependencies using its constructor.
Spring will inject the 'default' sessionStore bean
(https://github.com/apereo/cas/blob/3225e7cbe7f3a7afdea8a9bfab228858287e2481/support/cas-server-support-pac4j-webflow/src/main/java/org/apereo/cas/web/DelegatedClientNavigationController.java#L55).
Now what is the 'default' sessionStore bean supposed to be ? Frankly I don't know, I've only seen 2 locations from where the sessionStore can be produced: https://github.com/apereo/cas/blob/6d4d456c29bafc9d2811417876f06ac7e33f6e01/support/cas-server-support-pac4j/src/main/java/org/apereo/cas/support/pac4j/config/support/authentication/Pac4jAuthenticationEventExecutionPlanConfiguration.java#L87-L91 or https://github.com/apereo/cas/blob/30b4621884243ccc251167d9d162cd7bd646886b/support/cas-server-support-oauth/src/main/java/org/apereo/cas/config/CasOAuth20Configuration.java#L826-L834.
I suppose in my setup, that last one was taken as the default.

from DelegatedAuthenticationWebflowConfiguration that produces the same DelegatedClientNavigationController bean but here it will use the right sessionStore: delegatedClientDistributedSessionStore (see https://github.com/apereo/cas/blob/31aeeec9033b98685d52b34249b101f9d13ee6be/support/cas-server-support-pac4j-webflow/src/main/java/org/apereo/cas/web/flow/config/DelegatedAuthenticationWebflowConfiguration.java#L217).
But since DelegatedClientNavigationController is already available, it won't try to produce it (since we use ConditionalOnMissingBean in this bean).

Strangely, if we remove this @ConditionalOnMissingBean, Spring will produce the bean from DelegatedAuthenticationWebflowConfiguration class and not the bean from its own @Controller annotation.

And I'll try to explain why DelegatedClientAuthenticationAction is using DistributedJ2ESessionStore:
It's produced from https://github.com/apereo/cas/blob/31aeeec9033b98685d52b34249b101f9d13ee6be/support/cas-server-support-pac4j-webflow/src/main/java/org/apereo/cas/web/flow/config/DelegatedAuthenticationWebflowConfiguration.java#L165-L186
And hence we have DelegatedClientAuthenticationAction and DelegatedClientNavigationController that use a different sessionStore (at least when we use both oauth for authentication and pac4j for delegating authentication to another CAS Server and when we don't set cas.authn.oauth.replicateSessions to true).
Sorry for the long answer, hope this helps,
Adrian

Thanks much for the notes.

Could you attach a screenshot in a debugger showing that private final SessionStore<JEEContext> sessionStore; is in fact a J2ESessionStore instance?
Could you attach an overlay setup demonstrating the scenario?
Could you design a test case that shows/duplicates this scenario?

Hello, I can reproduce the same issue :
Here are the screen shots:
DelegatedClientNavigationController

DelegatedClientAuthenticationAction

Here is a repo that reproduce the issue :
https://github.com/bpavot/cas-issue-4778
As far as I could see removing the cas-server-support-oidc dependency changes the implementation used in DelegatedClientNavigationController back to DistributedJ2ESessionStore.

Possible workaround is to redefine the  delegatedClientNavigationController Bean
@Configuration
public class ContextConfiguration {

    @Autowired
    @Qualifier("builtClients")
    private ObjectProvider<Clients> builtClients;

    @Autowired
    private DelegatedClientWebflowManager delegatedClientWebflowManager;

    @Autowired
    @Qualifier("delegatedClientDistributedSessionStore")
    private ObjectProvider<SessionStore> delegatedClientDistributedSessionStore;

    @Bean
    public DelegatedClientNavigationController delegatedClientNavigationController() {
        return new DelegatedClientNavigationController(builtClients.getObject(),
                delegatedClientWebflowManager,
                delegatedClientDistributedSessionStore.getObject());
    }
}

:warning: The same issue can be raised if the CAS server is not served over HTTPS. (specially on localhost) because in that case the Cookie DISSESSION is marked as Secure but the browser will not store/pass it along. Hence CAS loosing track of the session and raising above error.

Resolved. Thanks for the patch.

@mmoayyed sorry for not having answered before, I was very busy with my project delivery deadline (I saw you answer but didn't kno when exactly I was going to have some time - which is now btw :) ).
Thanks very much @bpavot for taking the lead on this one !

Hi, sorry to reopen this.
I'm having this same issue reported in the logs (In 6.2.0-RC5 with OAuth delegation). I've tried this workaround, but with no success.
Has this been corrected in future updates?

@Khorsan : Looking a bit more closely at this PR, I see that the PR was closed but the fix was not merged [1].
@mmoayyed : did you fix this issue in another way ? Otherwise, could you consider reopening this PR and applying the fix ?
Thanks,
Adrian
[1] https://github.com/apereo/cas/blob/01e59b04085c0f634df9693ba86ddc879f6b0f0e/support/cas-server-support-pac4j-webflow/src/main/java/org/apereo/cas/web/flow/config/DelegatedAuthenticationWebflowConfiguration.java#L221 is sill annotated with @ConditionalOnMissingBean

@mmoayyed : did you fix this issue in another way ? Otherwise, could you consider reopening this PR and applying the fix ?
Thanks,
Adrian
[1] https://github.com/apereo/cas/blob/01e59b04085c0f634df9693ba86ddc879f6b0f0e/support/cas-server-support-pac4j-webflow/src/main/java/org/apereo/cas/web/flow/config/DelegatedAuthenticationWebflowConfiguration.java#L221 is sill annotated with @ConditionalOnMissingBean

Maybe something was changed, as I see some tweaking in class names and in AUP API. I fear that this fix is not working anymore. For my external IdP, it wasn't.
Tomorrow I will test the repo linked to recreate the issue.

I've rewritten the solution by @bpavot on April 12th to use with v6.2.0-RC5:
package org.example.cas;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.ObjectProvider;

import org.pac4j.core.client.Clients;
import org.pac4j.core.context.session.SessionStore;

import org.apereo.cas.web.DelegatedClientWebflowManager;
import org.apereo.cas.web.DefaultDelegatedAuthenticationNavigationController;

@Configuration
public class ContextConfiguration {

    @Autowired
    @Qualifier("builtClients")
    private ObjectProvider<Clients> builtClients;

    @Autowired
    private DelegatedClientWebflowManager delegatedClientWebflowManager;

    @Autowired
    @Qualifier("delegatedClientDistributedSessionStore")
    private ObjectProvider<SessionStore> delegatedClientDistributedSessionStore;

    @Bean(name = "delegatedClientNavigationController")
    public DefaultDelegatedAuthenticationNavigationController delegatedClientNavigationController() {
        System.out.println("New session configuration");
        return new DefaultDelegatedAuthenticationNavigationController(builtClients.getObject(),
            delegatedClientWebflowManager,
            delegatedClientDistributedSessionStore.getObject());
    }

}

with spring.factories like this:
org.springframework.boot.autoconfigure.EnableAutoConfiguration=org.example.cas.ContextConfiguration

CAS prints to stdout that string when configuring, so I know its loading it.

I've lurked arround the network logs and found out that the IdP i want to delegate my login to is not returning the TST-xxx state. Can this be my issue?

I'd like to chime in on this as I am experiencing the same issue on 6.2.0-RC2. However, I am certain that I had this working at some point, but now it is not. So I think I am hitting a configuration issue on my end, and not a "bug" per say. But I don't know that for sure. I am a CAS user, not a DEV, so I can't help there, but if there is anything I can do for this from a testing perspective, let me know.
I have a stackoverflow question open for it here.

I'd like to chime in on this as I am experiencing the same issue on 6.2.0-RC2. However, I am certain that I had this working at some point, but now it is not. So I think I am hitting a configuration issue on my end, and not a "bug" per say. But I don't know that for sure. I am a CAS user, not a DEV, so I can't help there, but if there is anything I can do for this from a testing perspective, let me know.
I have a stackoverflow question open for it here.
I have access to configure the IDP as well as the CAS Server.

Have you tried the fix?

Have you tried the fix?

I have not. I'm assuming I have to build it with the patch correct? I have not built CAS before and will need to familiarize myself with the process.

Have you tried the fix?

I have not. I'm assuming I have to build it with the patch correct? I have not built CAS before and will need to familiarize myself with the process.

You just need to create a src in your CAS project (where you have build.gradle for your dependencies).
The Java part (looking at the package I put there) goes to src/main/java/org/example/cas/ContextConfiguration.java and the other part goes into /src/main/resources/META-INF/spring.factories. When you run gradle build it should compile it. When you start CAS, you will see the output of System.out.println("New session configuration"); in the logs

You just need to create a src in your CAS project (where you have build.gradle for your dependencies).

I'll try that and let you know then.

Gonna need some more guidance. The way that I install CAS is I pull the overlay template from this repo: https://github.com/apereo/cas-overlay-template.git. I then add a custom build.gradle file and then run the build.
The repo has a "src" directory and "main" within it, but after that, the template is quite different than your directory description above. That being the case, I could take educated guesses on what I need to pull into the "src" directory, but I can't be certain I would be pulling the right stuff.

I'm reading through the build docs now.
https://apereo.github.io/cas/developer/Build-Process-6X.html

The default overlay template does not contain any Java source code. You need to create those directories, in this case, you should have src/main/java/org/example/cas and src/main/resources/META-INF/. Refer to my previoous post to get info about the files.
You don't need to build from source. What you want is to use CAS dependencies to compile what you need. CAS is built on top of Spring technologies. If you take a look here, it's where you want to add stuff like Databases for tickets or LDAP support, as refered in the documentation.
For instance, if you want LDAP authentication, you need to add one line to your project's dependencies in build.gradle (link here):
implementation "org.apereo.cas:cas-server-support-ldap:${project.'cas.version'}"

This file is what manages your project and allows you to build it (again, forget CAS from source). It will resolve all dependencies from Maven Central Repository.
The source code in YOUR project is compiled and configured (automatically) after all of the other dependencies are configured, making it override the original configuration. It's a bit difficult to understand Spring concepts, I know..

Thanks @Khorsan for the breakdown. I understand now. I'm not familiar with the Spring workflow, but it makes sense.
So, please confirm that I am speaking the right language now. The steps I need to take are as follows:

Create dirs in my current project where I originally built CAS from, in my case, /opt/workspace/cas-overlay-template/src/main/:

java/org/example/cas
resources/META-INF

Drop the file ContextConfiguration.java into java/org/example/cas with the contents from your comment above
Drop the file spring.factories in resources/META-INF with the contents from your comment above
Verify my build.gradle file is configured for pac4j...which it looks like this now which is I think accurate:

// Enable delegated authentication (i.e. SAML)
implementation "org.apereo.cas:cas-server-support-pac4j-webflow:${project.'cas.version'}"

Rebuild the project as I normally would: ./gradlew clean build

So I suppose since I dropped those files in there and told spring.factories about the new Java file, it will build the project with that source and that function will override the existing one?

Yes, that's exactly it! spring.factories tells spring which class to auto-configure and this overwrites previous configurations

No dice. I must be missing some detail. I tried to add pac4j-core to the build.gradle file, but it had no effect. Looks like something fundamental I am missing.
[root@dev-cas01 cas-overlay-template]# ./gradlew clean build

> Task :compileJava FAILED
/opt/workspace/cas-overlay-template/src/main/java/org/example/cas/ContextConfiguration.java:9: error: package org.pac4j.core.client does not exist
import org.pac4j.core.client.Clients;
                            ^
/opt/workspace/cas-overlay-template/src/main/java/org/example/cas/ContextConfiguration.java:10: error: package org.pac4j.core.context.session does not exist
import org.pac4j.core.context.session.SessionStore;
                                     ^
/opt/workspace/cas-overlay-template/src/main/java/org/example/cas/ContextConfiguration.java:13: error: cannot find symbol
import org.apereo.cas.web.DefaultDelegatedAuthenticationNavigationController;
                         ^
  symbol:   class DefaultDelegatedAuthenticationNavigationController
  location: package org.apereo.cas.web
/opt/workspace/cas-overlay-template/src/main/java/org/example/cas/ContextConfiguration.java:20: error: cannot find symbol
    private ObjectProvider<Clients> builtClients;
                           ^
  symbol:   class Clients
  location: class ContextConfiguration
/opt/workspace/cas-overlay-template/src/main/java/org/example/cas/ContextConfiguration.java:27: error: cannot find symbol
    private ObjectProvider<SessionStore> delegatedClientDistributedSessionStore;
                           ^
  symbol:   class SessionStore
  location: class ContextConfiguration
/opt/workspace/cas-overlay-template/src/main/java/org/example/cas/ContextConfiguration.java:31: error: cannot find symbol
    public DefaultDelegatedAuthenticationNavigationController delegatedClientNavigationController() {
           ^
  symbol:   class DefaultDelegatedAuthenticationNavigationController
  location: class ContextConfiguration
6 errors

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.4/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 4s
5 actionable tasks: 4 executed, 1 up-to-date

Please make sure you're in 6.2.0-RC5

Ah; OK. I'm on RC2. It'll take a bit more work to get that build going.

Well, it wasn't as bad as I thought; which I'm sure you know. I changed the 2 to a 5 and the build succeeded. However, it didn't resolve the error at first. I did see the "New session configuration" message so I am sure the modification went in as expected.
At some point, I must have tainted my test case because it ended up working. So to prove that it was your fix that resolved it, I reverted back to the RC5 release without your fix, and it still worked. Then I reverted back to the RC2 release and it still worked. I am thoroughly confused at this point, and am going to blow away the instance and start over from my base configuration, and then try not to taint my test cases to derive the root cause.

Well, I figured it out! It has nothing to do with CAS per say. I had two back end CAS servers fronted by an NGINX instance for a reverse proxy. It was set for simple session persistence (stickiness) by IP. The issue with the delegated auth scenario is that the assertion that comes back from the IDP comes from obviously a different IP and thus would get routed to the other CAS instance.
I'm pretty sure there is a viable setup so that tickets between two CAS servers are seen by one another, but I'll have to investigate that further. For the time being, I have put my setup into active/passive mode, so to speak where I down one of the CAS servers in the NGINX config. Then if I need to failover, I can swap them out at the NGINX config. Not ideal, but will get me through my current use-case.

Thank you @Khorsan for the help. I wasn't much help to you, but I certainly understand how one would modify a CAS installation.


Can't find Xcode Command Line Tools

[x] are reporting a bug others will be able to reproduce and not asking a question. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh
[x] ran a brew command and reproduced the problem with multiple formulae? If it's a problem with a single, official formula (not cask) please file this issue at Homebrew/homebrew-core: https://github.com/Homebrew/homebrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
[x] ran brew update and can still reproduce the problem?
[x] ran brew doctor, fixed all issues and can still reproduce the problem?
[x] ran brew config and brew doctor and included their output with your issue?

Sometimes I get this message when brew upgradeing formulae.
The bottle needs the Xcode CLT to be installed.

That's weird because I'm running Xcode 10 on MacOS Mojave and CLT are installed. I found instructions for installing Xcode CLT so I tried that.
$ xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates

I'm not sure what it means by "Software Update". I definitely have the latest release of Xcode 10.
Xcode is convinced that CLT is installed.

brew config can find some of the tools like clang and git but not CLT.
$ brew config
HOMEBREW_VERSION: 1.7.6-24-gac801d4
ORIGIN: https://github.com/Homebrew/brew
HEAD: ac801d4d634e0dadb59695638b508f7249f5f4d5
Last commit: 6 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 1f874c6d5acfd2ae635a25d68eae0a855ed157fc
Core tap last commit: 2 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_DEV_CMD_RUN: 1
HOMEBREW_GITHUB_API_TOKEN: set
CPU: quad-core 64-bit haswell
Homebrew Ruby: 2.3.7 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/bin/ruby
Clang: 10.0 build 1000
Git: 2.17.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 1.8.0_25, 1.6.0_65-b14-468
macOS: 10.14-x86_64
CLT: N/A
Xcode: 10.0
XQuartz: 2.7.11 => /opt/X11

brew doctor told me that I should update CLT from the App Store. It also warned me about unbrewed headers and libraries but I don't think that's relevant.
Warning: A newer Command Line Tools release is available.
Update them from Software Update in the App Store.

There aren't any updates available in the App Store because I have the latest!
Any ideas?

I originally posted this in the Homebrew community discussion but @SMillerDev said it's a bug and that I should report it so here I am!

Exact problem here. Latest Homebrew, Xcode 9 on 10.13 and Xcode 10 on 10.14, in both cases Xcode-select shows CLT, etc installed. If Homebrew is installed into from scratch HB will insist on downloading the CLT. Test: I removed the HB installed CLT from /library/developer then: IF- HB is already installed and update/upgrade is issued HB uses Xcode CLT and does not download. So it only downloads for a new HB install, not an upgrade/update in which case it finds the Xcode CLT. Yes, this is a bug for sure. since I loaded many packages (for freeswitch) and they compiled fine using the Xcode CLT. So no need to install then twice.

CLT: N/A

This means you don't have the latest CLT working with Homebrew.

Xcode is convinced that CLT is installed.

Xcode reports its own CLT and not the system one.

@MikeMcQuaid So... what do I do?

@Kerndog73 Run xcode-select --install. If that doesn't work check for Mojave system updates. If that doesn't work reboot your machine and try both again. If that doesn't work try to download and install the latest CLT from https://developer.apple.com/downloads/more/

Installing CLT from the Apple website seems to have partially solved the problem.
CLT: <IP_ADDRESS>.1.1535735448
CLT headers: N/A

I created this issue because when I brew upgraded a formula, homebrew built it from source instead of downloading the bottle. Are the "CLT headers" required to install a bottle?

Are the "CLT headers" required to install a bottle?

For some bottles: yes.

How should I proceed?

@Kerndog73 It only applies to some formulae. If you look at the formulae you'll see the reasoning and if you can fix the issue so it's no longer required we'll review a pull request.

@MikeMcQuaid Interesting. Well, thanks for your help!

I still don't see the problem, and this is NOT specific to Mojave since it occurs the same on 10.13. If I install 10.13 or 10.14 from scratch, and install XCODE, then copy /usr/local from a system with HB already on, the install goes fine with 13 packages. nothing is missing, and a separate CLT is NOT installed into /library/developer. So if HB is installed from scratch it forces CLT DL even if CLT is already in XCODE. This is a recent change to HB, it had always knew Xcode with the CLT was installed. As in this test, clearly CLT is installed along with XCODE with no need for a separate CLT.

I am sure this is all well thought out and there is good reason for everything. You said the installer always installs the CLT but I am curious as to why HB can't use the one from Xcode? Especially since, in my case, with a freeswitch install, everything installed and compiled fine without CLT in /library/developer using the Xcode CLT. I guess my question is what is different from Xcode CLT than standalone CLT, I could find nothing on that subject. Thank you.
PS. I wrote and maintain the freeswitch macOS wiki pages and I would like some info to add to it regarding this in case others run across this, some people use Xcode for other things and may be puzzled by this behavior.

Thanks for your quick responses, I really appreciate it. On a clean system a fresh install of HB using the ruby install command causes the standalone CLT to be installed, before any packages are installed.
I was confused since I thought your comment meant a "package formula" triggered the standalone CLT, but no packages were referenced prior to CLT installation. Since it is not a package formula that triggers this, I realize it's the Homebrew ruby installation formula that triggers standalone CLT installation.
Interesting observation: I did a lot of research and the only difference I found is Metal which is in the Xcode CLT and not in the standalone CLT, there may be other differences but I could not find them. For curiosity, I installed HB on a clean system, deleted /library/developer CLT installed HB, ran Xcode-select to point to Xcode CLT. Then all 13 packages and freeswitch installed fine. At least that worked for me, may not for others. Thank you very much for the responses and all your hard work.

Thank you very much for the responses and all your hard work.

🙇 You're welcome ❤


Simplify converter chain to avoid duplication

#833 introduces the MarkdownConverter in the conversion chain, such that it now appears above IfNeededConverter and as part of the ChainedConverter. The reason is that, when a browser requests a document that is originally Markdown, the server would settle for the Markdown representation since browsers accept */* and IfNeededConverter introduces a barrier. Hence, the Markdown conversion would only be triggered for rendering error pages etc.
This duplication can become complicated if there are more and more converters.
However, the IfNeededConverter is needed to avoid erroring on non-covertible document types.
We might instead want to remove MarkdownConverter and IfNeededConverter from the chain, and add a DoNothingConverter to the chain, such that there is always a conversion. (This specific solution might however increase the complexity of chained path construction, since it always matches. So maybe it's better as a configuration option to ChainedConverter instead, to allow a zero-length chain.)

I just discovered that the ChainedConverter already does a content-type check before starting conversion https://github.com/solid/community-server/blob/850c590a5185b75c55a1060cd35a8d155d91b36a/src/storage/conversion/ChainedConverter.ts#L179-L183

But that's if no conversion is needed. The additional case is: if no conversion improves the result.

Done in #1030. Will be part of v3.0.0 release.


feat(HMS-3392): add blueprint's version in image table

This PR adds a new version column in the image table,  only when a blueprint is picked:

Thanks for your work on this PR!
I've noticed that the version number is correctly showing up using the stage data:

However not with the msw data:

So maybe the msw data is missing something? Can you check that out please?

ah I approved the wrong PR sorry!

/retest

/retest

Thanks @lavocatt, the blueprint_version was missing. now msw should work


luci-app-ntpc - broken working and visible error when " 'option freq' '0' " to /etc/config/ntpclient

...as the topic.

Probably ntpc is used very often since sysntpd takes care of what most folks would want.  Most likely if others confirm ntpc is not working in general (rather than specific options) it will get marked BROKEN.

ok - there is one simple solution, which previously simply did not notice - just enough there do not enter anything - then you can save changes to the luci without error - the question of whether this is the right way to configure NtpClient and whether it actually work? I would add that I use jesj only the bottom of any configuration and correct functionality dnscrypt-proxy.

Seems like you've solved your problem, and it's more of a help issue for the forum or IRC.  Certainly obscure and needs more eyes than the issue tracker.


Livesync not working with TypeScript and iOS emulator

While following the Groceries-tutorial I've experienced a problem with livesync and watch. Whenever I make a change in a typescript-file the file-change is detected, compiled and then the app is synched back to the emulator-device. But the app will not reload in the emulator. I have to manually restart the app to the see the changes. The reloading works fine on Android, but not on iOS.
I have also tried creating a clean project based on the template tns-template-hello-world-ng. Using  livesync with this project works perfectly on iOS.
tns-cli: 2.0.1
tns-core-modules: 2.0.1
tsc: 1.8.10
xcode: 7.3.1
Here is the full log from my terminal:
tns livesync ios --emulator --watch
Executing before-prepare hook from /projects/nativescript/groceries/hooks/before-prepare/nativescript-dev-typescript.js
Found peer TypeScript 1.8.10
Project successfully prepared
Executing before-livesync hook from /projects/nativescript/groceries/hooks/before-livesync/nativescript-angular-sync .js
Transferring project files...
Successfully transferred all project files.
Applying changes...
Successfully synced application org.nativescript.groceries on device 5906C111-9503-4683-9E72-A9A416077589.
Executing before-watch hook from /projects/nativescript/groceries/hooks/before-watch/nativescript-dev-typescript.js
Found peer TypeScript 1.8.10
May 21 00:40:27 Macbook-Pro groceries2[37764]: assertion failed: 15E65 13E230: libxpc.dylib + 71506 [6F98A9DA-B46E-3A2B-88D4-92F1EC77477F]: 0x7d
May 21 00:40:27 Macbook-Pro groceries2[37764]: CONSOLE LOG file:///app/tns_modules/nativescript-angular/application.js:90:32: Page loaded
May 21 00:40:27 Macbook-Pro groceries2[37764]: CONSOLE LOG file:///app/tns_modules/nativescript-angular/application.js:92:32: BOOTSTRAPPING...
May 21 00:40:27 Macbook-Pro groceries2[37764]: CONSOLE LOG file:///app/tns_modules/nativescript-angular/dom_adapter.js:54:20: Setting DOM
May 21 00:40:27 --- last message repeated 1 time ---
May 21 00:40:27 Macbook-Pro groceries2[37764]: CONSOLE LOG file:///app/tns_modules/@angular/core/src/facade/lang.js:370:16: Angular 2 is running in the development mode. Call enableProdMode() to enable the production mode.
May 21 00:40:27 Macbook-Pro groceries2[37764]: CONSOLE LOG file:///app/tns_modules/nativescript-angular/application.js:95:36: ANGULAR BOOTSTRAP DONE.
12:40:28 AM - Compilation complete. Watching for file changes.

12:40:49 AM - File change detected. Starting incremental compilation...

12:40:50 AM - Compilation complete. Watching for file changes.

Executing before-prepare hook from /projects/nativescript/groceries/hooks/before-prepare/nativescript-dev-typescript.js
Project successfully prepared
Transferring project files...
Successfully transferred all project files.
Applying changes...
Successfully synced application org.nativescript.groceries on device 5906C111-9503-4683-9E72-A9A416077589.

Hey @abinici,
Sorry you ran into problems here. I haven’t run into this problem myself so I’m not sure what’s up. The output looks correct, as I can see that the TypeScript hook in hooks/before-prepare/nativescript-dev-typescript.js seems to be running as expected.
One quick idea: it looks like you renamed your project to “groceries” and there’s a chance that rename threw things out of sync, as livesync matches based on app id. Could you try running rm -rf platforms/ node_modules/ lib/ hooks, tns install, tns run ios --emulator, and tns livesync ios --emulator --watch again to see what happens?

This is the problem I think you’re hitting https://github.com/NativeScript/nativescript-cli/issues/1770

Hey @tjvantoll,
I followed your steps and voila, it began working again. You are right about the renaming, I changed the folder-name during the tutorial.
But I think you should improved this or make it less fragile, because renaming a folder during a project is not unusual. Maybe the CLI-tool should do a quick check before compiling the code?

Yep totally agreed. That’s what the open issue on the CLI repo is going to address. Thanks!

Good afternoon and tried to run on my android device physical an app with nativescrip 2.0 with angular2 doing tns create App --ng which I believe then I add the platform android with tns platform add android and when I run tns livesync android --watch me out A compilation error: TypeScript compiler failed with exit code null, I use linus debian 8, someone could help me
NOTE: when I create without angular2 (--ng), the app runs prefectly on my device


100% ( Hack And Trick)FREE PSN GIFT CARD CODES 2024🎮💯✔ Free PSN Codes 2024⚡ PSN Code Giveaway 2024🎮💯

Here are the active Free PSN Codes that are available for redemption for the players in October 2024.
 The answer is No. We all know that the developers of the PlayStation have already told the players not to believe in any codes which are available for free. So we suggest you not use the codes available on the website. However, the codes we have listed are from different sources and can be used without issues. We suggest that you should not trust any promising sites that will give you Free PSN Codes once you complete certain steps.
Is Free PSN Codes Legal?
No, Free PSN Codes are not legal to use. Using Free PSN Codes is considered illegal and against the terms of service stipulated by PlayStation’s developers. They’ve explicitly warned users against utilizing free codes, citing the risk of account bans for those who violate this guideline. Any engagement with websites offering free PlayStation Codes is strongly discouraged by developers due to the high likelihood of them being fraudulent, providing invalid codes or requiring users to complete numerous tasks to access these codes.
PlayStation Store
Most of the players complete the different steps, and they end up sharing personal information about themselves, or they mistakenly perform some activity, due to which the data and privacy of the players are breached, and the websites fool them. We suggest that you should not trust any promising sites that will give you Free PSN Codes once you complete certain steps. If you do so, do it at your own risk.
Is There Any Free PSN Codes Generator?
While exploring options for obtaining Free PSN Codes, many individuals may come across Free PSN Codes Generators online. It’s common for players to inquire about the legitimacy of these generators and whether the codes they produce are valid. Several websites purport to offer PSN Codes, but it is crucial to approach them with caution as many of them are not legitimate.
Advertisements
Advertisement
When players delve into searches for PSN codes, they are often confronted with offers from Free PSN Codes Generator websites, promising access to codes once a set of specified tasks are completed. However, navigating such websites requires caution as most are constructed with the intention to deceive and can pose serious risks to user data.
Such dubious websites often request the completion of surveys or similar tasks, promising free PSN codes upon completion. The potential hazards to personal data and the likelihood of fraudulent activity necessitate caution when encountering such sites, and it is advised not to engage or complete any steps they propose.
While these websites may claim to offer ‘free’ PSN codes, the inherent risks and the high probability of them being scams should deter users from participating. Engaging with such sites and following through their proposed steps should be considered risky, and users should undertake such actions at their own discretion and responsibility.
In summary, it is always safer to avoid websites and services offering Free PSN Codes through generators, as the majority are deceptive and can lead to potential harm or loss of personal information. The pursuit of legitimate and safe alternatives is always recommended.
However, accessing all these exciting features on the PlayStation Network requires a paid subscription. The good news is that there are several ways to get free PSN codes lists, PSN gift card codes, and PSN code generators that can help you get access to all the great content on the PlayStation Network.
CLICK HERE TO GENERATE PSN-CODES
What are Free PSN Codes?
Free PSN codes are unique digital codes that you can use to get access to the PlayStation Network without paying for a subscription. These codes can be redeemed on the PlayStation Store and used to purchase games, movies, and other digital content. The codes are usually given out by Sony or third-party websites as part of promotions, giveaways, or as a reward for completing certain tasks.
Powered By
GET 100$ PSN-GIFTS FREE HERE
What are Free PSN Gift Card Codes?
Free PSN gift card codes are digital codes that can be used to purchase games, movies, and other digital content on the PlayStation Store. These codes are usually given out by Sony or third-party websites as a reward for completing certain tasks, as part of promotions or giveaways. The PSN gift card codes can be redeemed on the PlayStation Store to purchase digital content.
What is a Free PSN Code Generator?
A free PSN code generator is a software tool that generates unique PSN codes that can be redeemed on the PlayStation Store. The generator works by using complex algorithms to create unique codes that are not yet in use. These codes can then be used to purchase games, movies, and other digital content on the PlayStation Store. Free PSN code generators are often available on third-party websites and can be downloaded for free.
Are Free PSN Codes Safe?
While there are many websites and tools that claim to offer free PSN codes and PSN gift card codes, not all of them are safe. Some websites may ask you to provide personal information or download software that can harm your computer or steal your personal information. Therefore, it's essential to be cautious when searching for free PSN codes or using a PSN code generator.
The best way to ensure that you are using a safe and legitimate PSN code generator is to research the website or tool before using it. Look for reviews and feedback from other users to see if they have had a positive experience using the generator. Additionally, be wary of websites that ask you to provide personal information or download software before providing you with a code.
How to Get Free PSN Codes and Gift Card Codes? Participate in Promotions and Giveaways
Sony often runs promotions and giveaways that offer free PSN codes and gift card codes. These promotions can be found on the PlayStation Blog, social media pages, or through email newsletters. Keep an eye out for these promotions and follow the instructions to participate.
Use Rewards Programs
Several websites offer rewards programs that allow you to earn points that can be redeemed for PSN codes and gift card codes. These websites usually require you to complete surveys, watch videos, or complete other tasks to earn points. Once you have enough points, you can redeem them for PSN codes and gift card codes.
Use a Free PSN Gift Card Code Generator
Free PSN Gift Card code generators are available on third-party websites and can be used to generate unique codes that can be redeemed on the PlayStation Store. However, it's important to be cautious when using these generators and only use them from trusted websites.
Conclusion
Getting free PSN codes and gift card codes can be a great way to access all the exciting content on the PlayStation Network without having to pay for a subscription. However, it's essential to be cautious when searching for free codes and only use safe and legitimate methods to get them.


Fix/live seek and media chrome upgrade

This resolves some edge cases with live seeking behavior and corresponding UI, which are particularly relevant for new pdt clipping available from Mux Video.
It also implements the "seekablechange", which can be used by Media Chrome to more reliably propagate mediaSeekable state changes.

Looks good, 1 blocking ask!


Issue when Building for Android

release -p .' was 1 instead of 0. (FastlaneCore::Interface::FastlaneShellError)
> Task :gradle:compileJava NO-SOURCE
> Task :gradle:compileGroovy UP-TO-DATE
> Task :gradle:pluginDescriptors UP-TO-DATE
> Task :gradle:processResources UP-TO-DATE
> Task :gradle:classes UP-TO-DATE
> Task :gradle:jar UP-TO-DATE

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':audio_waveforms'.
> Could not create an instance of type com.android.build.api.variant.impl.LibraryVariantBuilderImpl.
   > Namespace not specified. Specify a namespace in the module's build file. See https://d.android.com/r/tools/upgrade-assistant/set-namespace for information about setting the namespace.

     If you've specified the package attribute in the source AndroidManifest.xml, you can use the AGP Upgrade Assistant to migrate to the namespace value in the build file. Refer to https://d.android.com/r/tools/upgrade-assistant/agp-upgrade-assistant for general information about using the AGP Upgrade Assistant.

@appinteractive may i know which version you are using?

Hey I got it to work. The issue came from a newer version of gradle (8.2) and kotlin (1.8.x)
Now I downgraded to gradle 7.5 and kotlin 1.7.x and it seams to work fine.


Adds build status, coverage status and download badge to readme

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Added option to specify path to phantomas executable.

Fixed some issues with running test on OSX and added some verbose
error messages when invoking phantomas process.

LGTM, thanks for the patch!

Should be OK now.

:+1:


[Ready] Changes centering calculations for modals

Fixes #7 and https://github.com/elementary/website/issues/1255
I don't know how to test this so please tell me how to do so, or test it yourself and confirm the proposed changes are sufficient. Thank you!

To test you should be able to just open the index.html file in the repository and click any modal button there.
I think we're going to have to expressly include a container though, styling the modals normal parent will probably break some other styling. It's probably best if we have a container that is explicitly set as 100% width and 100% height to center in.

Alright, tested it and everything and it's good. Let me know if anything else needs doing (differently).

I'm closing this in favour of #13 where I've done basically the same thing by moving the existing overlay around and using that for positioning instead of creating a whole new element. Thanks for the help!

No problem. I actually tried to move it to the existing overlay, but I couldn't figure out how to resolve the resulting opacity issue. I'll look at your implementation to see how it was done.

I passed the opacity into an rgba background so that it wouldn't affect children like an actual opacity style would.

Yep, just saw the code now. I thought about that, but then I realized I'd have to figure out how exactly you handled the fade out. I also see you refactored the relevant bits  significantly, even changing the overlay from a class to an ID, which isn't something I could have done.

Yeah, I ended up just breaking the fadeOut in favor of CSS Transitions. jQuery was getting some pretty poor framerates over on elementary.


Link to api is broken

https://typeapi.barelyhuman.dev/package/@barelyhuman/tocolor@next

it's supposed to be pkg and not package...


Set key for message when using function publish

Motivation
In java functions, when a user calls publish, the outgoing message does not have a key set even though in the input message has a key.  By default, set the same key for the outgoing publish method.  This is the behavior when sending messages that are returned by the function
In python functions,  the message key is never set.  Make python functions consistent with java functions

@sijie in my previous comment, I already said I was going to add the tests.
Please read my responses in their entirety before responding.
On Mon, Apr 8, 2019 at 8:07 PM Sijie Guo<EMAIL_ADDRESS>wrote:

@sijie commented on this pull request.
In pulsar-client-cpp/python/pulsar/functions/context.py
https://github.com/apache/pulsar/pull/4005#discussion_r273310264:

@@ -126,7 +126,8 @@ def record_metric(self, metric_name, metric_value):
@abstractmethod
def publish(self, topic_name, message, serde_class_name="serde.IdentitySerDe", properties=None, compression_type=None, callback=None):

Sure I might be commenting in between commits. but my question is still
there : how can we guarantee the fix is correct? where are the tests for
this change?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/apache/pulsar/pull/4005#discussion_r273310264, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADcir6hlv1dcxwEj9LyuwhTw3xsfOlI-ks5vfAPtgaJpZM4ci9Wj
.

@jerrypeng got it now. thank you

@merlimat @jerrypeng @sijie please review this PR again. I have rewritten this PR to use the change @merlimat made for MessageBuilder

@sijie thanks for the review.


Run well for the first time but the next time something is wrong

FATAL {
  err: Error: 1 external link(s) are broken.
      at C:\Users\Xin\Desktop\hexo\loafing.cn\node_modules\hexo-broken-links-che
cker\lib\console.js:116:31
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

@forliuyifei  Need more details. Please, provide demo repo with the issue.


Update to Cosmos SDK v0.42.1

Just the sdk update and some minor test fixes.

LGTM.
Did you try running go mod tidy? I think there may be some cleanup needed in the go.sum

this isn't quite done yet, the Gravity V2 code won't adopt a erc20 representation of an Ethereum asset due to changes in the way token metadata works. Once I have that sorted I'll tidy and merge.


runtime: infinite loop in ./src/runtime/proc1.go lockextra() on FreeBSD

Howdy,
We've been seeing an issue recently on a couple of FreeBSD boxes. It appears as if the application keeps running as should, but CPU usage rises to 100% and the load increases to infinite numbers.
Truss showed that the application was doing a lot of sched_yield syscalls. The following keeps repeating:
49270: sched_yield(0x6aa450,0x7fffd96cbdc8,0xbcbcb0,0xfffffffffffff000,0x0,0x2e) = 0 (0x0)
49270: sched_yield(0x6aa450,0x7fffd9ccedc8,0xbcbcb0,0xfffffffffffff000,0x0,0x2e) = 0 (0x0)
49270: sched_yield(0x6aa450,0x7fffd98ccdc8,0xbcbcb0,0xfffffffffffff000,0x0,0x2e) = 0 (0x0)
Stack trace from the core dump:
#0  runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
306	/usr/local/go/src/runtime/sys_freebsd_amd64.s: No such file or directory.
[Current thread is 1 (Thread 8031ee400 (LWP 101729))]
(gdb) bt full
#0  runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
No locals.
#1  0x0000000000484dcb in runtime.lockextra (nilokay=false, ~r1=0x0) at /usr/local/go/src/runtime/proc1.go:1065
        old = 0
#2  0x00000000004849ef in runtime.needm (x=0 '\000') at /usr/local/go/src/runtime/proc1.go:942
        mp = 0x0
#3  0x00000000004b04f1 in runtime.cgocallback_gofunc () at /usr/local/go/src/runtime/asm_amd64.s:742
No locals.
#4  0x00000000004b041a in runtime.cgocallback () at /usr/local/go/src/runtime/asm_amd64.s:714
No locals.
#5  0x00000000006aa488 in _cgoexp_564f5a6b344f_Go_xxfi_connect (a=0x7fffd7cbedc8, n=32)
        fn = {void (struct github.com/Freeaqingme/gomilter._Ctype_struct_smfi_str *, github.com/Freeaqingme/gomilter._Ctype_char *, struct github.com/Freeaqingme/gomilter._Ctype_struct_sockaddr *,
    github.com/Freeaqingme/gomilter._Ctype_sfsistat *)} 0x7fffd7cbed50
#6  0x000000000075d23d in crosscall2 () at /usr/local/go/src/runtime/cgo/asm_amd64.s:36
No locals.

I'm not 100% sure whether this thread is also the thread causing all the problems based on the core dump that I have, but given that lockextra() calls osyield() in a for loop it seems very plausible.
Having said that, I also run this app on a few other very similar boxes where this problem does not show. Difference in configuration is that on the systems that do show these symptoms I use cmd := os/exec.Command(); cmd.Run(). But I have no indication that that particular code is having any issues ( https://github.com/Freeaqingme/ClueGetter/blob/61f0089/src/cluegetter/mailQueue.go#L107 ).
So far I have not been able to reproduce or trigger this behavior on Linux. The binary is compiled using go 1.5.1, although the code in proc.go (proc.go & proc1.go were merged) and sys_freebsd_amd64.s haven't been modified since. FreeBSD version: 10.1-RELEASE #0 r274401.

Does your program do a lot of callbacks from C to Go?
Doing a web search turned up this: http://mpc.lists.freebsd.bugs.narkive.com/IiDB406k/bug-192926-new-patch-sched-yield-is-a-no-op.  Is it possible that that bug applies to your system?

Does your program do a lot of callbacks from C to Go?

Yes. That's 90% of the input for my application. I believe line 5 of the stack trace refers to this function, which is invoked from a C library (the sendmail milter library): https://github.com/Freeaqingme/gomilter/blob/master/gomilter.go#L211

Is it possible that that bug applies to your system?

I did see the issue, but hadn't related it because the fix was applied well before this kernel was compiled. I'll see if this particular build included the fix for that bug report.
Ftr, I'm not entirely sure where the bug would be. In theory it could be a locking issue in my application code (or so I was told on IRC), but I'm really not able to find it there. It could of course also be an issue with the kernel, but determining where exactly still goes a little over my head.

If that sched_yield bug does exist in your kernel, then you are in trouble because the spin lock in lockextra will burn all your CPU, which I think is approximately what you are seeing.
If your kernel does not have that sched_yield bug, then I think the next step is to verify that this particular loop is the one that is generating all the sched_yield calls.

The kernel was built with commit r274401, which includes the fix for the FreeBSD bug linked above.

the next step is to verify that this particular loop is the one that is generating all the sched_yield calls.

Any idea on how to do so?

Maybe it's good to mention. The C library (libmilter) spawns a new thread per incoming connection, then does a callback to my go code. The machine this happens on handles a few connections at a time, max.
However, doing 'info threads' on the core dump shows that at the moment the machine shows the high load, there are well over 300 threads:
(gdb) info threads
  Id   Target Id         Frame
  320  Thread 802406400 (LWP 101329) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  319  Thread 802406800 (LWP 100443) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  318  Thread 802406c00 (LWP 100451) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  317  Thread 802407000 (LWP 100632) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  316  Thread 802407400 (LWP 100646) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  315  Thread 802810400 (LWP 100924) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  314  Thread 802810800 (LWP 101236) 0x0000000801b613ba in poll () from /lib/libc.so.7
  313  Thread 802c10400 (LWP 101237) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  312  Thread 802c10800 (LWP 101322) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  311  Thread 802c10c00 (LWP 101419) runtime.kevent () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:340
  310  Thread 802c11000 (LWP 101420) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  309  Thread 802c11400 (LWP 101421) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  308  Thread 803013400 (LWP 101422) 0x0000000801af8a8a in _sigwait () from /lib/libc.so.7
  307  Thread 803014800 (LWP 101395) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  306  Thread 803013800 (LWP 100673) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  305  Thread 803015000 (LWP 101277) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  304  Thread 803014000 (LWP 100126) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  303  Thread 803014c00 (LWP 100895) runtime.sys_umtx_op () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:21
  302  Thread 803014400 (LWP 101096) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  301  Thread 803015c00 (LWP 101417) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  300  Thread 803016400 (LWP 101430) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  299  Thread 803016800 (LWP 101431) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  298  Thread 803015400 (LWP 101432) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  297  Thread 803016000 (LWP 101433) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  296  Thread 803015800 (LWP 101434) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  295  Thread 803016c00 (LWP 101435) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  294  Thread 803017000 (LWP 101436) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  293  Thread 803017400 (LWP 101437) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  292  Thread 803017800 (LWP 101438) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
...
  144  Thread 803104000 (LWP 101586) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  143  Thread 803104400 (LWP 101587) 0x0000000000484daf in runtime.lockextra (nilokay=false, ~r1=0x0) at /usr/local/go/src/runtime/proc1.go:1062
  142  Thread 803104800 (LWP 101588) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  141  Thread 803104c00 (LWP 101589) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  140  Thread 803105000 (LWP 101590) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  139  Thread 803105400 (LWP 101591) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  138  Thread 803105800 (LWP 101592) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  137  Thread 803105c00 (LWP 101593) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  136  Thread 803106000 (LWP 101594) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  135  Thread 803106400 (LWP 101595) runtime.lockextra (nilokay=false, ~r1=0x0) at /usr/local/go/src/runtime/proc1.go:1062
  134  Thread 803106800 (LWP 101596) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  133  Thread 803106c00 (LWP 101597) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  132  Thread 803107000 (LWP 101598) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  131  Thread 803107400 (LWP 101599) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  130  Thread 803107800 (LWP 101600) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  129  Thread 803107c00 (LWP 101601) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  128  Thread 803108000 (LWP 101602) runtime.atomicload64 (ptr=0xff4020 <runtime.extram>, ~r1=1) at /usr/local/go/src/runtime/atomic_amd64x.go:30
  127  Thread 803108400 (LWP 101603) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
...
  13   Thread 8031eb400 (LWP 101717) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  12   Thread 8031eb800 (LWP 101718) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  11   Thread 8031ebc00 (LWP 101719) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  10   Thread 8031ec000 (LWP 101720) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  9    Thread 8031ec400 (LWP 101721) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  8    Thread 8031ec800 (LWP 101722) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  7    Thread 8031ecc00 (LWP 101723) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  6    Thread 8031ed000 (LWP 101724) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  5    Thread 8031ed400 (LWP 101725) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  4    Thread 8031ed800 (LWP 101726) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  3    Thread 8031edc00 (LWP 101727) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
  2    Thread 8031ee000 (LWP 101728) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
* 1    Thread 8031ee400 (LWP 101729) runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
(gdb)

Backtraces all look very similar:
(gdb) thread 238
[Switching to thread 238 (Thread 803083400 (LWP 101492))]
#0  runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
306	in /usr/local/go/src/runtime/sys_freebsd_amd64.s
(gdb) bt full
#0  runtime.osyield () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:306
No locals.
#1  0x0000000000484dcb in runtime.lockextra (nilokay=false, ~r1=0x0) at /usr/local/go/src/runtime/proc1.go:1065
        old = 0
#2  0x00000000004849ef in runtime.needm (x=0 '\000') at /usr/local/go/src/runtime/proc1.go:942
        mp = 0x0
#3  0x00000000004b04f1 in runtime.cgocallback_gofunc () at /usr/local/go/src/runtime/asm_amd64.s:742
No locals.
#4  0x00000000004b041a in runtime.cgocallback () at /usr/local/go/src/runtime/asm_amd64.s:714
No locals.
#5  0x00000000006aa488 in _cgoexp_564f5a6b344f_Go_xxfi_connect (a=0x7ffff57abdc8, n=32)
        fn = {void (struct github.com/Freeaqingme/gomilter._Ctype_struct_smfi_str *, github.com/Freeaqingme/gomilter._Ctype_char *,
    struct github.com/Freeaqingme/gomilter._Ctype_struct_sockaddr *, github.com/Freeaqingme/gomilter._Ctype_sfsistat *)} 0x7ffff57abd50
#6  0x000000000075d23d in crosscall2 () at /usr/local/go/src/runtime/cgo/asm_amd64.s:36
No locals.
#7  0x00007ffff57abdc8 in ?? ()
No symbol table info available.
#8  0x0000000000000020 in ?? ()
No symbol table info available.
#9  0x0000000000000036 in ?? ()
No symbol table info available.
#10 0x00007ffff57abe00 in ?? ()
No symbol table info available.
#11 0x0000000000403c40 in ?? ()
No symbol table info available.
#12 0x00007ffff57abf28 in ?? ()
No symbol table info available.
#13 0x000000080307c280 in ?? ()
No symbol table info available.
#14 0x0000000801e20ff0 in ?? () from /lib/libc.so.7
No symbol table info available.
#15 0x000000080307c280 in ?? ()
No symbol table info available.
#16 0x0000000801e20ff0 in ?? () from /lib/libc.so.7
No symbol table info available.
#17 0x00007ffff57abe00 in ?? ()
No symbol table info available.
#18 0x0000000000403c88 in Go_xxfi_connect ()
No symbol table info available.
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)

I don't know what is happening.  The fact that it is happening for you on FreeBSD but not on GNU/Linux suggests a FreeBSD specific problem, but I don't know what it could be other than surprising behaviour of sched_yield.
Here is how it is supposed to work.  There is a linked list of extra M structures to use for a callback from C to Go on a thread started by C (the structures are not needed  on threads started by Go).  That linked list starts out with one entry on it.
When there is a callback from C to Go, the code calls needm which calls lockextra(false).  The lockextra function spins waiting for an M structure to be available.  When one is available, if it is the only one available, the code sets the needextram flag.  It then starts the callback.
When the thread gets to Go proper, but before it actually calls the Go callback function, it checks the needextram flag.  If it is set, the thread calls newextram.  The newextram function allocates a new M structure and adds it to the list (using another call to lockextra, this time passing true).
When the Go callback function completes, the runtime calls dropm.  This adds the M structure back on the list, using another call to lockextra(true).
  The first one to succeed will create a new M structure, handing it over to the next thread.  When the callback completes, the M structure will be put back on the list.
So the behaviour we are likely to see is that the first time there are a bunch of parallel callbacks, there is contention in lockextra until all the M structures are allocated.  When all the callbacks complete, the linked list will have a bunch of M structures, so there should be no more issues until there is again a bunch of even more parallel callbacks.  Is that the behaviour you see?  An initial set of collisions that resolve, followed by normal behaviour?
The parallel callbacks vying for an M can theoretically starve the callback adding a new M to the list.  This is not supposed to happen because of the call to osyield, and because getting the lock and failing to find an M to use will call usleep(1), giving time for the new M to appear.
The call to usleep does add another possibility for OS-specific differences.  The FreeBSD/AMD64 specific code looks fine to me, though.  Does your truss listing show any calls to nanosleep?

The fact that it is happening for you on FreeBSD but not on GNU/Linux suggests a FreeBSD specific problem

I agree. But, I haven't been able to 100% reproduce the input these freebsd boxes are getting on linux. So the fact that I haven't seen it on linux is not entirely conclusive. I'm planning to add a linux box to the set of boxes that have been demonstrating this behavior to see if it will occur there as well.

Is that the behaviour you see? An initial set of collisions that resolve, followed by normal behaviour?

Not entirely. I think we can agree on the fact that there's some contention somewhere, resulting in the 300 threads listed above. However, once the load begins to rise, the application doesn't recover. Even when we wait 20 minutes before killing it, no cgo callback is processed since the beginning of the outage. Other goroutines, including ones that invoke os/exec.Command().Run() continue running as should.
I think this last bit must be related somehow, given that I run the application on 10 machines. 3 of these machines run the os/exec.Command().Run() code path, and somehow these are the only instances exhibiting any problems.

Does your truss listing show any calls to nanosleep?

The truss output that we captured is only a 100 lines long. Because 90 lines of these were showing the exact same line we didn't bother capturing anything more. However, of these 100 lines there's no nanosleep() calls.

How long does it take this program to run on your FreeBSD system?  On your GNU/Linux system?
package main

/*
#include <stddef.h>
#include <time.h>
#include <pthread.h>

#define THREADS (300)

extern void GoFn();

static void* thread(void* darg) {
    struct timespec ts;

    int *arg = (int*)(darg);
    ts.tv_sec = 0;
    ts.tv_nsec = 1000 * (THREADS - *arg);
    nanosleep(&ts, NULL);

    GoFn();

    return NULL;
}

static void CFn() {
    int i;
    pthread_t tids[THREADS];

    for (i = 0; i < THREADS; i++) {
	    pthread_create(&tids[i], NULL, thread, (void*)(&i));
    }
    for (i = 0; i < THREADS; i++) {
	    pthread_join(tids[i], NULL);
    }
}
*/
import "C"

import (
    "fmt"
    "time"
)

//export GoFn
func GoFn() {
    time.Sleep(time.Millisecond)
}

func main() {
    t := time.Now()
    C.CFn()
    fmt.Println(time.Since(t))
}

Below I've got some results. Beware though that the linux machine has 8 cores (including hyperthreads), the freebsd machine only has two:
```FreeBSD # GOTRACEBACK=crash GOPATH=pwd /usr/local/go/bin/go run -race  threads.go
runtime/cgo
cc: warning: argument unused during compilation: '-pthread'
runtime/race
cc: warning: argument unused during compilation: '-pthread'
command-line-arguments
cc: warning: argument unused during compilation: '-pthread'
1.64223369s

```Linux # time  GOTRACEBACK=crash GOPATH=`pwd` /usr/local/go/bin/go run  -race  threads.go
113.884271ms````

I've repeated the above a dozen times or so, all yielding similar results. One time however I again got a load that skyrocketed on FreeBSD. In order to stress the system a little bit more, I upped the THREADS constant to 650.

```Linux # while true; do ./threads ; sleep 1; done
1.523410828s
271.040985ms
587.719518ms
681.638101ms
334.02606ms
366.330778ms```

All results are similar to the ones posted above, the 1.5s is the highest number observed (even with GOMAXPROCS set to 1, results are not too dissimilar averaging at ~650ms).

Doing the same on FreeBSD though:
```FreeBSD # while true; do ./threads ; sleep 1; done
751.868957ms
34.801136295s
3m59.019588714s
1.51715983s
12.29715116s
587.125403ms
4.297773294s
2.025705987s

Every few runs the load spikes. With the run that took >3 minutes, the load went up to 600. The >3 minute run is not an exception, but rather a regular phenomenon. Even though the symptoms are the same (load spikes to semi-infinite amounts), internally it's a little different. Truss shows:
nanosleep({0.000001000 })			 = 0 (0x0)
nanosleep({0.000001000 })			 = 0 (0x0)
nanosleep({0.000001000 })			 = 0 (0x0)
nanosleep({0.000001000 })			 = 0 (0x0)
nanosleep({0.000001000 })			 = 0 (0x0)

A random backtrace from during such an event:
#0  runtime.usleep () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:272
272		RET
(gdb) bt full
#0  runtime.usleep () at /usr/local/go/src/runtime/sys_freebsd_amd64.s:272
No locals.
#1  0x00000000004589b6 in runtime.lockextra (nilokay=false, ~r1=0x0) at /usr/local/go/src/runtime/proc1.go:1069
        old = 0
#2  0x000000000045852f in runtime.needm (x=0 '\000') at /usr/local/go/src/runtime/proc1.go:942
        mp = 0x0
        _g_ = 0x0
#3  0x0000000000480971 in runtime.cgocallback_gofunc () at /usr/local/go/src/runtime/asm_amd64.s:742
No locals.
#4  0x000000000048089a in runtime.cgocallback () at /usr/local/go/src/runtime/asm_amd64.s:714
No locals.
#5  0x000000000042c988 in _cgoexp_454a349e7b7e_GoFn (a=0x7fffb4da7f8f, n=0) at command-line-arguments/_obj/_cgo_gotypes.go:48
        fn = {void (void)} 0x7fffb4da7f10
#6  0x00000000004829dd in crosscall2 () at /usr/local/go/src/runtime/cgo/asm_amd64.s:36
No locals.
#7  0x00007fffb4da7f8f in ?? ()
No symbol table info available.
#8  0x0000000000000000 in ?? ()
No symbol table info available.
(gdb)

According to https://golang.org/cl/18455, a sleep on FreeBSD always sleeps for the remainder of the time slice, which I think increases the sleep time from 1 microsecond to 1 millisecond.  That might explain why FreeBSD tends to be slower than GNU/Linux when there is a lot of contention among threads waiting for an M structure.

@ianlancetaylor Thanks for the link, hadn't found that one yet. That could surely explain why it's a little slower on FreeBSD than on Linux. But, if performance was a little worse that's something I could live with (in this particular instance performance is hardly a concern). However, the fact that usleep() gives up the rest of the time slice, is sort of a constant factor I'd think. That wouldn't yet explain why every once in a while it would lock up on sched_yield() right? Or, for that matter, why your test program takes 25 times as long in some runs compared to other runs.

Yes, there is something here I don't understand.  But I'm thinking that I might understand why it's so much worse on FreeBSD.  As you say, it's only a constant factor, but it's a constant factor of up to 1000.
Now that you point it out I also see a wide variance running my program on GNU/Linux.  In 100 runs, the minimum is 18.227649ms, the maximum is 723.168546ms, the median is 53.283891ms, the mean is<PHONE_NUMBER>7ms.

I was just able to come up with this little snippet. I ran it ~10 times, and with two runs it showed the same problem almost right away. That is, load soaring, truss showing predominantly sched_yield() calls, but also some nanosleep() calls. A representative snippet:
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
nanosleep({0.010000000 })			 = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff93c9d80,0x5c6620,0x443b2922,0xffffffff8158df70,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff97cbd90,0x5c6620,0x443b28ef,0xffffffff8158e080,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff93c9d80,0x5c6620,0x443b2922,0xffffffff8158df70,0x443b2525) = 0 (0x0)
clock_gettime(4,{11447280.710362564 })		 = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
clock_gettime(0,{1452803833.553310724 })	 = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff97cbd90,0x5c6620,0x443b28ef,0xffffffff8158e080,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff95cad80,0x5c6620,0xffff807f009315f0,0xffffffff8158fae0,0x443b2075) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff93c9d80,0x5c6620,0x443b2922,0xffffffff8158df70,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff91c8d80,0x5c6620,0x443b2465,0xffffffff8158ef50,0x443b2075) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff95cad80,0x5c6620,0xffff807f009315f0,0xffffffff8158fae0,0x443b2075) = 0 (0x0)
sched_yield(0x14,0x7ffff93c9d80,0x5c6620,0x443b2922,0xffffffff8158df70,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff93c9d80,0x5c6620,0x443b2922,0xffffffff8158df70,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff93c9d80,0x5c6620,0x443b2922,0xffffffff8158df70,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
write(1,"2016-01-14 21:36:37.595712734 +0"...,42) = 42 (0x2a)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff97cbd90,0x5c6620,0x443b28ef,0xffffffff8158e080,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff91c8d80,0x5c6620,0x443b2465,0xffffffff8158ef50,0x443b2075) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
nanosleep({0.010000000 })			 = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff95cad80,0x5c6620,0xffff807f009315f0,0xffffffff8158fae0,0x443b2075) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
clock_gettime(4,{11447280.725765584 })		 = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8fc7d90,0x5c6620,0x443b291d,0xffffffff81590700,0x443b2525) = 0 (0x0)
sched_yield(0x14,0x7ffff8bc5d80,0x5c6620,0x443b290b,0xffffffff8158edd0,0x443b24c1) = 0 (0x0)
sched_yield(0x14,0x7ffff8dc6d80,0x5c6620,0x443b28fa,0xffffffff8158e020,0x443b2525) = 0 (0x0)

Code:
package main

/*
#include <stddef.h>
#include <time.h>
#include <pthread.h>

#define THREADS (50)

extern void GoFn();

static void* thread(void* darg) {
    struct timespec ts;

    int *arg = (int*)(darg);
    ts.tv_sec = 0;
    ts.tv_nsec = 1000 * (THREADS - *arg);
    nanosleep(&ts, NULL);

    GoFn();

    return NULL;
}

static void CFn() {
    int i;
    pthread_t tids[THREADS];

    for (i = 0; i < THREADS; i++) {
        pthread_create(&tids[i], NULL, thread, (void*)(&i));
    }
    for (i = 0; i < THREADS; i++) {
        pthread_join(tids[i], NULL);
    }
}
*/
import "C"

import (
	"bytes"
	"fmt"
	"os/exec"
	"time"
)

//export GoFn
func GoFn() {
	time.Sleep(time.Millisecond)
}

func main() {

	go cmdTimer()
	ticker := time.NewTicker(time.Millisecond * 1000)
	for _ = range ticker.C {
		C.CFn()
	}
}

func cmdTimer() {
	t := time.Now()

	cmdRun := func(id string) {
		cmd := exec.Command("iostat")
		cmd.Dir = "/"
		var out bytes.Buffer
		cmd.Stdout = &out
		cmd.Run()
		fmt.Println(t.String() + " "  + id)
	}

	ticker := time.NewTicker(time.Millisecond * 500)
	for _ = range ticker.C {
		go cmdRun("1")
		go cmdRun("2")
		go cmdRun("3")
		go cmdRun("4")
	}

}

Using a larger number of threads (e.g. 250 instead of 50) triggers the issue almost immediately.

At this point in the release cycle I think we have to postpone to 1.7.

Yeah, makes sense. For what it's worth, we migrated this application off to linux 6 days ago and it hasn't shown since - under an identical load as with which we saw this behavior before within 12 hours. So far so good. knocks on wood

@ianlancetaylor What shall we do with this issue? Personally I have no interest in pursuing this issue any further. It appears the exact same set of syscalls is performed on FreeBSD as is on Linux, and it works perfectly fine on the latter. My conclusion would be that it's some sort of race condition in the FreeBSD scheduler

@ianlancetaylor Scratch what I said above about FreeBSD. I also just  witnessed the very same issue on Linux. Difference with FreeBSD would be that we witnessed the same issue there every day, and so far (3 to 4 months in) it's the first day we've seen it on Linux.
Strace output simply showed lots of sched_yield() calls, I also have a pprof dump. The only thing I wasn't able to attain was a coredump (at least I was able to install gdb with a load of 1000...).

I think I'm hitting this issue too on linux 3.16. I eventually see all of my threads that call C code from go start spending 100% of their time in runtime.osyield. Strace also shows a tone of sched_yield calls.
I'm currently seeing if using fewer threads makes this less likely to happen. If it happens again I'll see if I can repro with a smaller example.

/cc @dvyukov

@adamfaulkner What version of Go? Can you provide a repro?

This was postponed to 1.7 from 1.6. Will it miss the boat again? We only have a week or so to formulate a fix here.

100% CPU consumption is expected effect of a global spin mutex. If more than 1 thread is doing cgo callback on external thread, all but one wait. If average rate of calls is higher than 1/num_threads, then the process locks up as described.
Shared library support added another global mutex on that path. Now it is also used by C traceback support. If we resolve lockextra, the bottleneck will just move to another place.

@dvyukov I'm not sure I follow you entirely. I don't think having some sort of bottleneck is a problem, but the fact that everything locks up, and then stays locked up is a problem. If we can prevent the latter I'd be a happy man.

The description says "It appears as if the application keeps running as should".

That's a valid point. What I meant to say was that because the load keeps on rising, even though my load balancers stop feeding the application new data. So eventually the entire machine becomes unusable, and the application needs to be killed. I wanted to see if things would settle automatically, but it didn't. However, the load by that time had grown to >1000 so eventually the entire machine 'locks' up due to load.

I don't think a high loadavg will cause an operating system to lock up.
Loadavg is just a measure of the number of processes on the run queue.
What is the resource that causes your machine to lock up, my guess is
available memory. Can you please give some more details.
On Wed, 8 Jun 2016, 05:07 Dolf Schimmel<EMAIL_ADDRESS>wrote:

That's a valid point. What I meant to say was that because the load keeps
on rising, even though my load balancers stop feeding the application new
data. So eventually the entire machine becomes unusable, and the
application needs to be killed. I wanted to see if things would settle
automatically, but it didn't. However, the load by that time had grown to

1000 so eventually the entire machine 'locks' up due to load.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/13926#issuecomment-224382763, or mute
the thread
https://github.com/notifications/unsubscribe/AAAcA4JTpjeA_OJ-7zTTOmuTcElmGvTLks5qJcGHgaJpZM4HDelZ
.

I've tried to reproduce the issue on tip on linux/amd64 using the program from https://github.com/golang/go/issues/13926#issuecomment-171775379 (as is and with various changes that might increase probability of bugs in extram).
Works like a charm. No hangs.

We clearly aren't going to do anything here for 1.7.

For 1.8 perhaps we should consider storing extram on the C side, not the Go side.  Then we can use C mutexes and condition variables, and avoid the busy wait.
Of course better still would be to avoid the global lock, but it's not clear to me how to do that.


Search input field missing in new outline in Code 1.32.1

I just updated to Code 1.32.1 on Windows and look at the Outline pane.
I'm sure there used to be an input field to filter functions by name. Where is it gone?

Now you can just type and the filter query will appear


Feature/ci mgmt

Create feature branch to test out ci.mgmt pipeline.

@mitchdenny
As your suggestion I've made a feature branch to test out the CI integration. Still trying to figure the rules, since ci.data and ci.client should not run here, and new ci.mgmt should run.

@weidongxu-microsoft do you have any ETA on this work? We really would like to get the root of the repo cleared up and into this new structure so folks coming to the repo can more easily see the readme file and the structure will become more consistent with the other language repos. Java is the last repo to conform to this new structure.

@weshaggard I am still working on it (but at a lower priority). Code restructuring is solved. But there is lots of try-and-error while configuring the pipeline (with too much yaml intangled and too little documationation for guidance), with the gap need to be tried-out and I am not comfortable to do this even on the feature branch. i probably will try that on another fork under my github account to faciliate (without affecting others) to see which triggers with on all these pipelines.

I'd suggest we do this in steps if we can. First step is to restructure then the next step is change the pipelines.

@mitchdenny @joshfree
What is your opinion?
Do we try to do it in steps (with first step just move the repositories and config the mgmt.yaml with say 50 lines of explicit excludes on client/data-plane lib), then config the pipeline for the moved mgmt lib; or we still try to figure out the pipeline in this PR as well?
If later does anyone have better way to test the triggering of pipeline without actually sending dummy PR to do the trigger? It is because this PR also trigger client/data-plane build which was excluded and should not be trigger and I've no idea why.

 You do this via the Azure Pipelines UX. Find your pipeline, run it, and select the branch.

/check-enforcer evaluate

@mitchdenny
I've manually added the ADO pipeline for keyvault mgmt.
https://dev.azure.com/azure-sdk/internal/_build?definitionId=1217&_a=summary
https://dev.azure.com/azure-sdk/public/_build?definitionId=1218&_a=summary
Please take a review. Thanks.
There appears to be 2 pipeline with same config, one in public, one in internal. What is the purpose?
I still do not know how to use PipelineGenerator. Would you give me a sample command line with parameters (e.g. I've no idea what "Name of an environment variable which contains a PAT." is).

@JimSuplizio @weshaggard
Here is still restructure + pipeline combined. Please let me know your opinion on it.
Note since the transfer to azure.core is not done on mgmt yet, these are all legacy mgmt SDKs (and as Jim mentioned in email, some of them will not build. One cause might be that the "azure-arm-client-runtime" package later than 1.6.5 no longer have "test" jar published, hence for pom had to specify 1.6.5 as test.)
And CI is not using the shared yaml template, but using a similar configure as eng/pipelines/mgmt.yml
I am not sure whether in future we would have certain period that the legacy code / pipeline would co-exist with new code / pipeline based on azure.core, and whether we need to prepare for it at present (say, ci.mgmt-legacy.yml, instead?).

@weidongxu-microsoft I've had a look through this and I think that this is a reasonable approach for now. Once this is merged, I'd actually like to evaluate whether we can convert the management, track 1 and track 2 client pipelines on a per service directory basis.
The reason is that I am conscious of a current performance issue in Azure Pipelines where they make a request to GitHub for each pipeline associated with the repository (and templates it references). Until they optimize this doubling up on all the pipelines could be a perf/stability issue. I think in time that will be fixed though.
We shouldn't block on this now - I just wanted to give you an indication of where my head is at.

@mitchdenny Thanks for your comments. I will wait a few days for @JimSuplizio @weshaggard 's input as well.
BTW, do you have sample usage on PipelineGenerator? I assume this is the tool to create CI pipeline automatically in batch. And when we resturcture multiple services, We will need the tool to configure the CI pipeline (instead of manually here).

I'll send you a link offline, our invocation of the pipeline generator sits in a template on a private repo.

I agree with the approach of just updating the existing mgmt.yml files for now to account for the restructuring. It looks like you have currently only moved the keyvault directories so I assume you plan to do the others as well? Perhaps in another PR.

@weshaggard
This PR is actually separating the pipeline (and make new dedicated CI for keyvault mgmt) as well.
If this one is OK, next plan is PR no.2 to move one more service, and configure the pipeline generator job to pick up it (with yaml) and create an CI in DevOp automatically.
And then we could do in batch in PR no.3 ++.
History is this:
At first we planned to move code only, and continue using mgmt.yml for CI. However this got complicated by DevOp not supporting wildcard in yaml include/exclude clause so we had difficulty to either include the specific mgmt folder or exclude data/client folder under sdk.
And after that we are trying to sperate the CI as well. And complications with that make this PR take quite some time to form.
Now I think Mitch and I could agree that the sperated mgmt CI in keyvault is in good shape and it is now possible to continue with other services.
@JimSuplizio I would still like your opinion. I do not want to surprise you with any CI change or change in sdk folder, since you probably work on CI and other utils under sdk folder as well (e.g. I do not want to move this and suddently your python script now updating thing that it does not support to update).

Jim raised an issue that there be quite many CI pipeline does not define SdkType=client. I will take some time to review this case as well.

Jim raised an issue that there be quite many CI pipeline does not define SdkType=client. I will take some time to review this case as well.

This is by design. The default (from memory) is that the pipelines are treated as track 1 pipelines (client). But for those that aren't we define them as data. All it really controls is the spin up of Jetty at this point since the new track 2 libraries don't require it.

@mitchdenny Thanks. The issue is that previously there is at most 2-value in profile selecting, so it is such:
    <profile>
      <id>data</id>
      <activation>
        <property>
          <name>env.SDKTYPE</name>
          <value>data</value>
        </property>
      </activation>
      ...
    </profile>
    <profile>
      <id>client</id>
      <activation>
        <property>
          <name>env.SDKTYPE</name>
          <value>!data</value>
        </property>
      </activation>
      ...
    </profile>

And the activation for client is loose as !data. So SdkType could be client, could be not defined, could be anything other than data.
Now after we added mgmt, it would be 3-value, and maven does not support e.g. !data && !mgmt in profile activation. So I modified it to such:
    <profile>
      <id>data</id>
      <activation>
        <property>
          <name>env.SDKTYPE</name>
          <value>data</value>
        </property>
      </activation>
      ...
    </profile>
    <profile>
      <id>client</id>
      <activation>
        <property>
          <name>env.SDKTYPE</name>
          <value>client</value>
        </property>
      </activation>
      ...
    </profile>
    <profile>
      <id>mgmt</id>
      <activation>
        <property>
          <name>env.SDKTYPE</name>
          <value>mgmt</value>
        </property>
      </activation>
      ...
    </profile>

And the activation for client would thus be strictly as client.
So far I think this would not be cause, since either CI for client already defined SdkType=client, or we can spec !mgmt to existing CI without SdkType defined and add mgmt for newly added ci.mgmt.

With this change we'll need to go through all our Java pipelines and set a variable which controls this. Its not the end of the world, although perhaps this is then something that we should embed in the ci.yml, ci.data.yml and ci.mgmt.yml files instead of the definition.
At some point in the future we'll ditch ci.data.yml as it won't be necessary and then we could potentially look at converging ci.yml and ci.mgmt.yml (thats a bit further off though).
So in this PR then could you add a variable called SDKTYPE=client to ci.yml, SDKTYPE=data for ci.data.yml and SDKTYPE=management for ci.mgmt.yml?
That we we won't need to have the custom config per pipeline...

@mitchdenny
Do you mean you would like to have SdkType in yaml, instead of in DevOp variable?
OK. I think it is reasonable to concentrated on yaml and avoid customize CI on DevOp.

Exactly. When we just had two flavors of build we could get away with sprinking the variable in the definition here and there and relying on !data to opt client on by default (since we only have a handful of track 1 builds). But this change would require it everywhere so we may as well put it in the YAML.

@mitchdenny @weidongxu-microsoft  if we change to set SDKType=client for client builds that means that developers building client pipelines are going to always have the SDKType variable set in order to build client libraries. Right now, the way things are setup the SDKType only needs to be set for Data. Before making this type of change it should probably get approval from @JonathanGiles because I'm not sure that requiring an environment variable for client builds, when one wasn't required before, is going to fly. It's unfortunate that maven won't let allow multiple properties (or a single property with multiple value checks) inside of the activation tags for a profile.

@JimSuplizio
IMHO, the implication of adding SdkType=client precondition is really for the developers.
CI we got all kinds of variable/template/parameters and not everyone would required to know what kind of setting would work for CI or not, mainly we just depends on existing setting and CI running there.
However if any developer is actually using this pom.service.xml for daily build, they might found out that suddenly it does not build their library (because of this added env). Of course they would only need to specifically specify the profile via "-Pclient", similar to "-Pdata".
Personally I am kind of in the camp of make "maven clean package" just work for everybody, so I sort of prefer a configure that without any setting, maven would build all of the libs. But this is not very related to this PR.

because I'm not sure that requiring an environment variable for client builds, when one wasn't required before, is going to fly. It's unfortunate that maven won't let allow multiple properties (or a single property with multiple value checks) inside of the activation tags for a profile.

Hrm. Good point Jim - it might actually be easier to introduce a 2nd aggregator POM pom.mgmt.xml which is similar to pom.service.xml (names TBC) which is focused on management plane to split this out.

I would be in favor of moving to a model where we have a single pom.xml file in each sdk/[service] directory which replaces pom.service.xml. This would be a future aggregate of client, data and management plane.

@mitchdenny
I will process with a 2nd pom.mgmt.xml. It will minimal the impact on client/data.

Link to a prior PR https://github.com/Azure/azure-sdk-for-java/pull/5103

/azp run

/check-enforcer evaluate

https://github.com/Azure/azure-sdk-for-java/issues/5682

Thank you @weidongxu-microsoft for starting the migration of management plane SDKs under the /sdk root folder!  This will really help clean up the azure-sdk-for-java repo and get it clean and consistent with the other azure-sdk-for-* repo directory structures. 😄


hiding bar element

hi
creating custom legend, doing following on a bar chart does not hide the actual bar element:
meta = chart.getDatasetMeta(datasetindex);
meta.data[index].hidden = !meta.data[index].hidden;
it works fine if you set meta.hidden = !meta.hidden, but in my case i'm not interested hiding the whole dataset.
and I do run update afterwards, and everything also changes. The bar that I want to hide just don't disappear
kind regards
c_bb

@coolbombom what are you trying to achieve? You want to toggle individual items out of the dataset?

Closing as stale


Make JsonPathError implement std::error::Error

Make JsonPathError implement std::error::Error so it can be encapsulated as a valid source by libraries or programs that implement their own error type.
Specifically this is usefull when using thiserror, for example now we can have the following error type in our program that uses jsonpath:
use thiserror::Error;

#[derive(Error, Debug)]
pub enum MyError {
    #[error("General error: {0}")]
    GeneralError(String),

    #[error("JSON path error {0}")]
    InvalidJsonPath(#[from] jsonpath_lib::JsonPathError),

    #[error("Some other error")]
    SomeOtherError,
}

This patch would also make the error type compatible with the anyhow crate.


yarn dev failing with error from react-server-dom-webpack module

Hi, I am trying to run the demo and seeing an error when accessing http://localhost:3000. Any clue why I am seeing this error?
arul@MacBook-Pro next-server-components % yarn dev
yarn run v1.22.10
$ next
Loaded env from /Users/arul/Documents/GitHub/nextjs/forks/next-server-components/.env
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

ready - started server on http://localhost:3000
event - compiled successfully
event - build page: /
wait  - compiling...
event - compiled successfully
event - build page: /api
wait  - compiling...
[BABEL] Note: The code generator has deoptimised the styling of /Users/arul/Documents/GitHub/nextjs/forks/next-server-components/libs/send-res.build.js as it exceeds the max of 500KB.
event - compiled successfully
API resolved without sending a response for /api?location=%7B%22selectedId%22%3Anull%2C%22isEditing%22%3Afalse%2C%22searchText%22%3A%22%22%7D, this may result in stalled requests.
/Users/arul/Documents/GitHub/nextjs/forks/next-server-components/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack.development.js:562
  var reader = stream.getReader();
                      ^

TypeError: stream.getReader is not a function
    at startReadingFromStream (/Users/arul/Documents/GitHub/nextjs/forks/next-server-components/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack.development.js:562:23)
    at /Users/arul/Documents/GitHub/nextjs/forks/next-server-components/node_modules/react-server-dom-webpack/cjs/react-server-dom-webpack.development.js:594:5
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
arul@MacBook-Pro next-server-components %

I think it's an issue in the react-server-dom-webpack package. Currently the work around is to switch to old node versions such as v12. We will look into a proper fix.

I think it's an issue in the react-server-dom-webpack package. Currently the work around is to switch to old node versions such as v12. We will look into a proper fix.

Switching to v12 works like a charm! Thank you @shuding.

Switching to v12 works like a charm! Thank you @shuding.

I noticed a warning from nextjs every time index page is loaded, do you think this could be related to react-server-dom-webpack package?
API resolved without sending a response for /api?location=%7B%22selectedId%22%3Anull%2C%22isEditing%22%3Afalse%2C%22searchText%22%3A%22%22%7D, this may result in stalled requests.

I noticed a warning from nextjs every time index page is loaded, do you think this could be related to react-server-dom-webpack package?
API resolved without sending a response for /api?location=%7B%22selectedId%22%3Anull%2C%22isEditing%22%3Afalse%2C%22searchText%22%3A%22%22%7D, this may result in stalled requests.

Switching to v12 works like a charm! Thank you @shuding.

What was the exact node version you switched to? I've tried 12.21.0 but unfortunately I'm still getting the same error

@mattfwood It works with 12.21.0, i see the same error, but it doesn't terminate the process. With 15.11.0, it terminates the process.

The demo is fully rewritten now, close this issue as outdated 🙏