proposal: crypto: add crypto engines

Crypto Engines
The Go standard library cryptography implementation is very comprehensive and
widely used. The problem, however, comes when a user wants to swap out the
implementation for another, while using the same interface. This has already
happened upstream with the dev.boringcrypto [0] branch. Use cases for using an
alternate implementation include FIPS compliance (where a user may need to call
into a FIPS validated module), and interfacing with external crypto devices
(e.g.  accelerator cards, specialty hardware).
It would be easy enough to maintain a package that does this already. However,
the problem comes when trying to swap crypto implementation in a project you do
not own. This would also ensure consistency so that there is a formal way to
declare that all or part of the crypto operations provided by the standard
library should be implemented by some outside package.
This proposal outlines a way for an alternate implementation of the crypto
functionality to be loaded / unloaded via function calls, and allow the
alternate crypto implementation to live in a separate package maintained outside
of the Go source tree. It also allows all this without the user having to modify
import paths, allowing a user to load an engine once and have that change
propagated everywhere automatically.
Engine implementation
Users should be able to load a single global engine which if present will be
used instead of the standard library implementation. The design allows engines
to replace all or part of the standard library implementations. There is no
concept of having multiple engines. Loading an engine makes it the default for
any crypto implemented by that engine. This means that a single engine could
replace all or part of the crypto functionality present in the crypto package
in the standard library.
Engine interface
All engines to be registered internally must conform to a specific interface.
The interfaces will be focused on specific areas of cryptography following the
package boundaries in the standard library.
type Engine interface {
	AES()    AESEngine
	Cipher() CipherEngine
	DES()    DESEngine
	...
	TLS()    TLSEngine

	Setup()   error // Engines would provide their own setup method.
	Cleanup() error // Engines would provide their own cleanup method.
}

type AESEngine interface {
	NewCipher(key []byte) (cipher.Block, error)
}
...

The goal would be that all crypto operations could be handled by an engine
implementation, including TLS specific functionality (PRF, et al).
Loading
The loading of an engine should be simple and straightforward. Only a single
engine can be loaded at any given time. This means one must unload an engine
before attempting to load another one.
The user is presented with 3 options:

Load an engine directly by having the engine source code and directly
calling LoadEngine with the struct.
Load an engine via Go plugin functionality [1].
Load an engine based on the environment variable
GO_CRYPTO_ENGINE=<path>. This would cause the program to load the
engine from <path> at runtime. <path> should point to a valid Go
plugin.

The basic outline for these functions would look like:
package crypto

// LoadEngine loads the engine specified by `e`.
// This engine will be used for any crypto functionality
// it provides, falling back to the standard library for
// any functionality not provided by the engine.
func LoadEngine(e Engine) error {
	...
}

// LoadEnginePlugin will load the plugin located
// at `path` and subsequently create and register
// an engine from it.
func LoadEnginePlugin(path string) error {
	...
}

// engineFromPlugin wraps the plugin in an Engine
// implementation suitable for usage.
func engineFromPlugin(p *plugin.Plugin) Engine {
	...
}

It would also be very easy to unload an Engine, if need be, via:
func UnloadEngine() error {
	...
}

It is the responsibility of the engine to maintain correctness. In other words,
the engine must be responsible for maintaining locks, reference counting, or any
other method to ensure that when EngineCleanup is called the resources used by
that engine will not be left in an invalid state.
Usage
The standard library code will branch based on the engine and specific
implementation being non-null.
For example:
// NewCipher creates and returns a new cipher.Block.
// The key argument should be the AES key,
// either 16, 24, or 32 bytes to select
// AES-128, AES-192, or AES-256.
func NewCipher(key []byte) (cipher.Block, error) {
	k := len(key)
	switch k {
	default:
		return nil, KeySizeError(k)
	case 16, 24, 32:
		break
	}
	if e := getEngine(); e != nil && e.AES() != nil {
		return e.AES().NewCipher(key)
    	}
	return newCipher(key)
}

Real world use cases
The upstream Go repository currently has a branch [0] that is being maintained
separate from the main master branch which adds support for calling into
BoringCrypto [2]. Instead of maintaining an entirely separate branch, the
functionality could be rewritten as an engine and included in any project that
may need it.
Additionally, if a Go user has a requirement that their code meet FIPS
requirements and would not like to use (or cannot use) the dev.boringcrypto
branch, this new functionality would allow the user to pick their crypto library
of choice (OpenSSL, LibreSSL, et al).
Finally, if there are users who wish to take advantage of certain hardware for
crypto, that too could be accomplished via engines.
Implementation details
Originally when drafting this proposal I wanted to get deep into specifics of
how this functionality would be implemented as far as how the engine is
represented internally (after loading / assignment). However after a few drafts
and further thinking I figured I'd leave it a bit more abstract at first lest
this devolve into a code review of sorts. I'd rather the discussion be around
the idea of engines as a feature. If necessary I can draft a further design doc
on some of the proposed specifics. Otherwise, we can leave that to a later date
in a proper code review setting.
Summary
Engines would provide an easy way to swap all or part of the standard library
crypto functionality with one provided by the user. There are already real world
use cases and examples of folks using other crypto libraries either by creating
their own package [3][4][5], or even by upstream maintaining a separate
branch to support BoringSSL.
This proposal suggests a way to remove the maintenance burden and make the Go
standard library crypto implementation more extensible.

A use case: business reasons (FIPS, etc.) cause us to have to use one of a couple crypto libraries, depending on the situation. We wrote a library that copies the stdlib's crypto API and uses different "backends" based on build tags.
One of the major pain points is instead of import "crypto/aes" we have to write import "some/project.tld/crypto/aes". In order to be sure that we don't accidentally import the wrong crypto/aes, we have to have tests that scan our code base and check for bad imports. This is a fraught endeavor.
Another pain point is when libraries don't implement entirely what we need. In this case, we have to fall back to the Go stdlib which results in sloppy and error-prone backend code.
Additionally, we have no real way to switch backends at runtime depending on, e.g., whether FIPS mode is enabled on the host machine.

CC @FiloSottile @agl

Originally when drafting this proposal I wanted to get deep into specifics of
how this functionality would be implemented as far as how the engine is
represented internally (after loading / assignment). However after a few drafts
and further thinking I figured I'd leave it a bit more abstract at first lest
this devolve into a code review of sorts. I'd rather the discussion be around
the idea of engines as a feature.

I understand this desire and it does make sense; There's no need to get stuck on implementation details while still pondering on whether we even need such a thing. Having said that, I would prefer to see a proposal in an early stage for the interfaces where there's an implementation of at least two separate engines.
Once too often I've seen someone gobble up "a single generic interface to rule them all", which turns out to be basically a copy of their own implementation, and not be compatible with any alternatives. If we set out to create a single interface to rule them all (as I see this proposal), we should at least know that that is realistic and that it fulfils all requirements of other (potential) engines as well.
If that turns hard to do -for example because no other implementations exist yet - perhaps a small case study of other similar initiatives (e.g. in the C-ecosystem) could be done to show how these evolved and where they lack - if at all.
Finally, it's not mentioned explicitly in the proposal why this should be part of the standard library, rather than be a third party package that simply encapsulates the existing crypto stuff. Am I right in assuming it's necessary because you also want other things in the standard library to use this engine?
All in all, +1 from me :)

@Freeaqingme I don't want to speak for @derekparker, but I know that my +1 would change to a -1 if this didn't affect the stdlib's crypto. My previous comment sort of implies why, but since the default crypto for Go is the stdlib's crypto, there'd still be lots of pain points if this was implemented as a 3rd party library.

I understand this desire and it does make sense; There's no need to get stuck on implementation
details while still pondering on whether we even need such a thing. Having said that, I would prefer
to see a proposal in an early stage for the interfaces where there's an implementation of at least
two separate engines.

I understand that and can work to provide a working example if necessary once discussion on this proposal has progressed. I just wanted to get the conversation started here.

Finally, it's not mentioned explicitly in the proposal why this should be part of the standard library, rather than be a third party package that simply encapsulates the existing crypto stuff. Am I right in assuming it's necessary because you also want other things in the standard library to use this engine?

Sorry, I though I has clarified that. There are a few reason for this being part of the standard library as opposed to simply an external package. One, there are already many external crypto packages for Go and yet this proposal still exists. The issue isn't with the ability to use other crypto with the Go programming language itself, the issue is being able to swap crypto back ends without changing import paths. This is an important detail for a few reason:

crypto/tls: the TLS stack uses crypto/* (of course) and if you want to use the Go stdlib TLS to use FIPS validated crypto based on a system being booted in FIPS mode, there's really no way to accomplish that as of now, even with changes in the program code using TLS.
crypto/tls: there is crypto functionality implemented in this package that would also need to be swapped out (PRF, etc).
Large projects: the goal would be for projects like Kubernetes to be able to simply load a new crypto engine without changing tons of import paths and worry about diverging from the standard library crypto interfaces.
Many more examples I can provide is the above is not adequate.

The dev.boring crypto branch (and other forks based on it) are created so that Go programs built with that version of the toolchain just simply call into other crypto libs without user code having to do much, if anything. That is the real goal here, to distill and simplify this need of having to use crypto other than pure-Go standard library crypto, and being able to do it without using an "unofficial" branch or fork of the language.
Hope that all makes sense! I'm excited by the support and discussion around this.

type Engine interface {
    ...
}

What happens when a new algorithm gets added to the stdlib?  Do all existing engines become obsolete?

What happens when a new algorithm gets added to the stdlib? Do all existing engines become obsolete?

Yes, that potential is there. However, looking back at the past few releases:
Go 1.12 - No new crypto functions added
Go 1.11 - Added 1 new function, 1 new method
Go 1.10 - Added 3 new functions
Go 1.9 - No new crypto functions
These changes only come in minor version updates which happen on a 6 month cadence. I feel that if a person needs to go through the trouble of using and maintaining a cryptographic library, the maintenance burden of adding ~4 new functions to said library over the span of ~2 years (from 1.9 to 1.12) doesn't seem insurmountable.

Alternatively, would it be possible to have a single interface that returns which algo's are supported, and then have a single interface per algorithm?

Alternatively, would it be possible to have a single interface that returns which algo's are supported, and then have a single interface per algorithm?

Could you sketch out what you mean by that?

Sorta related to having a “single interface per algorithm,” check out how bazil.org/fuse/fs handles multiple optional methods.

Sorta related to having a “single interface per algorithm,” check out how bazil.org/fuse/fs handles multiple optional methods.

ACK. That seems like an interesting addition in the interface approach. That could solve the problem of making older engines obsolete.

It would definitely be nice to have some way of transparently adding FIPS or distro managed crypto support to Go. I have some concerns with doing this at runtime though:

If the Go crypto library is always used as a fall back then I don't think that there is a way to verify that you are actually using your crypto engine and not ending up in the std implementation. Typos might result in the engine code not being called and yet everything might well still work since the Go crypto library is used as a fall back. This seems like it would be an issue for people working in regulated environments.

It is likely that this proposal would make most of the crypto API opaque to the compiler because of the additional level of indirection. For example, escape analysis will fail for more parameters passed into crypto calls. If you want to use a crypto engine you are probably going to need to pay this price anyway but without significant cross-package optimizations (e.g. to figure out that LoadEngine() isn't called anywhere and therefore the engine won't be present) all crypto library users will also end up paying for it, even if they aren't using crypto engines.

Personally I think it would be nicer to have a clean way to swap out standard library packages at compile time using modules or a compiler option. That way you could statically check that the binary doesn't contain any Go crypto code (unless the replacement you are using links to it as a fallback - not sure what sort of mechanism that would use). Also this approach would have no impact on people who just want to use the standard Go crypto library as-is.

@mundaym I'm not sure I agree that it should be required at compile time. But I think the underlying concern that it should be verifiable which engine is being used is a valid one. I suppose that's something that can also be done at runtime. E.g. a call to crypto.ActiveEngine() that returns a struct with a field to identify the engine in use. That could be printed while starting the application, shown through a http handler, etc.

@Freeaqingme I guess it depends on your level of paranoia (or the size of the fines...) :). A call to crypto.ActiveEngine() would certainly suffice for some use cases but I don't think it can guarantee that std crypto isn't using its own implementation due to a bug.
Does anyone know what sort of guarantees are required for FIPS compliant programs? Do they just need to avoid calling non-FIPS code or should they also avoid linking against it?

Well, if you don't use the stdlib crypto anywhere, then those symbols would be unused? They could then be stripped from the binary if so desired, right? However, were the application compiled to remove all debug symbols, it may be hard to analyze if std crypto is still used, so therefore I think it's at least required to be able to determine it at runtime. Removing them at compile time would be a nice bonus, but  I think they could be stripped after compiling as well.

I think that this issue is trying to solve two distinct problems:

wholesale replacement of the crypto stack for regulatory reasons.

replacement of low-level cryptographic primitives with
platform-specific versions.

Both are useful, but they involve different tradeoffs.  In case (1), you
want to make sure that the stdlib code is never called, even if that means
failing in cases that are not covered by the replacement.  In case (2),
you want to gracefully fall back to stdlib whenever necessary, but you
care about performance very much (e.g. you don't want every hardware-
accelerated SHA-1 to cause a piece of data to escape to the heap).
Should this issue be restricted to just case (1)?
(In which case I fear that compile-time replacement of the stdlib crypto
code is the only workable solution.)

There has been talk of potentially splitting parts of the standard library into modules. If the crypto/ packages were part of a separate module, then you could use the replace directive to replace the crypto implementation. That seems like a simpler mechanism than the one proposed here.
Additionally, if your primary goal is to use a different TLS stack, it might be easier to make just crypto/tls pluggable, as @FiloSottile has suggested in the past: https://github.com/golang/go/issues/21753, https://github.com/golang/go/issues/32466#issuecomment-506490451, https://github.com/golang/go/issues/30241#issuecomment-465806600

If the Go crypto library is always used as a fall back then I don't think that there is a way to verify that you are actually using your crypto engine and not ending up in the std implementation.

This is a very great point, and one that is addressed in forks such as the boringcrypto branch by panic'ing if a non-boring code path is executed if boring mode is activated. I agree this proposal is a bit lax on this, but to be honest I mostly wanted to get the conversation started. I think we should have a way of explicitly failing instead of falling back to the stdlib.

Does anyone know what sort of guarantees are required for FIPS compliant programs? Do they just need to avoid calling non-FIPS code or should they also avoid linking against it?

They must fail when calling non-FIPS code.

There has been talk of potentially splitting parts of the standard library into modules. If the crypto/ packages were part of a separate module, then you could use the replace directive to replace the crypto implementation. That seems like a simpler mechanism than the one proposed here.

That would be interesting, but would have to wait until Go 2.

Maybe this is a distraction, but as a thought experiment, what if this proposal were instead "garbage collection engines" or "channel engines" or "reflect engines" or "encoding/json engines?" Would any of those make sense to try to support? Why or why not?
Obviously some other implementations of Go do substitute different garbage collection and channel and reflect implementations, though not as far as I know encoding/json. The existence of those alternate Go implementations that carry pieces more appropriate to their use case has not caused us to try to make the garbage collector or runtime or reflection systems "pluggable". I wonder if crypto is like those or more like encoding/json or somewhere in the middle.
For dev.boringcrypto I of course had to answer that question, and I answered it by saying that crypto was like other key pieces like GC and runtime and that if you changed it you simply had a different Go. Is that the wrong answer? If so, why?
Is crypto different solely because of FIPS? Or something more general?

For dev.boringcrypto I of course had to answer that question, and I answered it by saying that crypto was like other key pieces like GC and runtime and that if you changed it you simply had a different Go. Is that the wrong answer? If so, why?

I would argue yes. My main reasoning being that nobody will be prevented and / or charged large fines due to running Go binaries because of the garbage collector, runtime, or channel implementation, reflect or encoding implementations. However, certain companies simply cannot run Go binaries in situations requiring certain standards of cryptography due to, as you mentioned, government mandates such as requiring FIPS compliance. That's the fundamental issue here, is that by not allowing a standard upstream way for users to switch crypto implementations based on actual business needs and requirements we limit what can be used in the Go ecosystem, or at least make it way more difficult.
The argument against that would be "just use an external package". However I feel if it really were that easy, the dev.boringcrypto branch would never have been created and internally within Google there would be a bunch of rewrite rules or similar to swap projects using crypto/* with some other package. Also, as mentioned elsewhere in this thread, simply swapping out the crypto packages with a user defined one does nothing for other internal Go packages such as net which will always use the stdlib crypto anyways. So now not only does a user need to use an external crypto library, they also need to use their own net/http stack which uses their own TLS implementations. One of the beautiful aspects of the Go language is in the fact that you need not look much further than the standard library in order to build pretty much anything you'd like. Having to replace large chunks of that standard library to meet these business needs kind of takes away from that I think.
Now, along with that I think there are things that should be updated in this proposal to reflect the discussion. Namely, I think there should be some way to prevent falling back to stdlib crypto if an engine is loaded to prevent situations where non-FIPS code gets called in situations where it would be inappropriate. However I think those details can be hashed out later in this thread, certainly before any kind of implementation. I just want to address the basic need for something like this and stress that cryptography is fundamentally different than a runtime or garbage collector.

For dev.boringcrypto I of course had to answer that question, and I answered it by saying that crypto was like other key pieces like GC and runtime and that if you changed it you simply had a different Go. Is that the wrong answer? If so, why?

That's a legit question I suppose. I'm not 100% informed when it's about the alternate GC or runtime implementations, so I might make some (incorrect) assumptions here...
I assume the alternate GC or runtime stuff is implemented because of technical considerations. It's done by people who have a different vision on certain technical aspects of these components, and therefore want to do it themselves. Simply because they can do it "better" (mind the quotes! :)).
However, this proposal is not about companies that want to swap the crypto implementation because of technical reasons. If they feel they can do better, they may temporarily fork it, but only to contribute those changes back to upstream later (like CloudFlare did with the TLS asm implementation).
The companies that are looking to use a FIPS implementation (and rule out usage of stdlib crypto), do so because of business considerations, rather than technical considerations. They're not looking to maintain a fork because they can do it better, they simply have a business requirement to substitute a single component (crypto) with another. That's why I think that allowing for alternate crypto implementations is different from allowing alternate GC or channel implementations.
Of course we can say that those concerns are not the concerns of the Golang community and therefore a fork is the way to go. Had this been proposed 10 years ago, it might have been very well because the language was less mature and resources were better spent on other things. I think that now that Go is gaining more and more traction, it could be a good moment to consider it.
In the early days of Android, vendors also wanted functionality that was not present in the stock linux kernel. This in turn lead to many forks, often barely updated, hardly ever kept in sync with upstream. This reflected badly on the Linux community, and lead to a worse user experience for the end user. In the end, much effort (by Greg Kroah-Hartman among others) was invested into getting it all back into sync. This in turn has lead to an increased amount of available developer hours on a single 'implementation' of Linux, rather than having them divided.
I realize that Go != Linux  and that a programming language != kernel, but it could be an argument of why having forks could be a bad thing in the long term.

Let me start by saying that as the current maintainer of dev.boringcrypto, I am sympathetic to the pain of maintaining a fork, and I understand where this is coming from.
I think you make a clear argument that there is a business case for swapping crypto backends. I'm not convinced it's qualitatively different from the business case for swapping other parts of the language though: for example, the GC and runtime make Go incompatible with embedded devices, which also "limit[s] what can be used in the Go ecosystem".
There are however ways in which swapping cryptography backends is peculiar technically, and they all suggest leaning against this proposal.

It was tried before, not with great success. I regularly hear complaints about the disparate implementations of the Java Cryptography Extension, and the OpenSSL engines add a lot of complexity to their codebase.
The cryptography standard library is not a monolithic and stable feature set, like garbage collection or channels might be. New algorithms are added regularly, and having to account for the possibility that they might not be implemented by the engine would add a layer of complexity in some of the most gnarly places, like TLS certificate selection and algorithm negotiation.
Cryptography is complex by nature, so a lot of its complexity budget is spent on the algorithms themselves, leaving less space for this kind of flexibility. This is acknowledged by golang.org/design/cryptography-principles in two places: the first instrument of Secure being "reduced complexity", and Practical specifying that we "focus[] on common use cases to stay minimal".

Overall, it's a tradeoff. Not making cryptography engines pluggable puts the cost of maintaining a FIPS fork on the organizations that need it (Google included), instead of adding complexity to the standard library used by most. Also considering that it's mostly well-staffed organizations that need FIPS compliance, I think that's the right call, and I don't think we should implement this.
Maintaining one of the simplest cryptography libraries in the whole ecosystem is an exercise in resisting complexity, and it sometimes comes at the cost of this kind of flexibility.
(This focuses on the FIPS case of switching the whole engine, because for providing hardware implementations of specific values we have interfaces like crypto.Signer, and if we need to add more like it you should feel free to open specific issues, as I'd be happy to consider them. They already proved valuable and powerful.)

Thanks for your thoughtful reply @FiloSottile.

Let me start by saying that as the current maintainer of dev.boringcrypto, I am sympathetic to the pain of maintaining a fork, and I understand where this is coming from.

I was definitely awaiting your reply and thoughts on this as we have talked before about the dev.boringcrypto branch in private mail and I knew we were likely experiencing the same pain points.

I think you make a clear argument that there is a business case for swapping crypto backends. I'm not convinced it's qualitatively different from the business case for swapping other parts of the language though: for example, the GC and runtime make Go incompatible with embedded devices, which also "limit[s] what can be used in the Go ecosystem".

Thank you. One counter I would provide here is that Go was never intended as a language suitable for embedded devices. As a language that is geared towards writing performant server software, I think that differentiates the crypto packages as they are necessary these days, as running a server without TLS and proper security in any kind of production environment is irresponsible at this point.

There are however ways in which swapping cryptography backends is peculiar technically, and they all suggest leaning against this proposal.

It was tried before, not with great success. I regularly hear complaints about the disparate implementations of the Java Cryptography Extension, and the OpenSSL engines add a lot of complexity to their codebase.
The cryptography standard library is not a monolithic and stable feature set, like garbage collection or channels might be. New algorithms are added regularly, and having to account for the possibility that they might not be implemented by the engine would add a layer of complexity in some of the most gnarly places, like TLS certificate selection and algorithm negotiation.
Cryptography is complex by nature, so a lot of its complexity budget is spent on the algorithms themselves, leaving less space for this kind of flexibility. This is acknowledged by golang.org/design/cryptography-principles in two places: the first instrument of Secure being "reduced complexity", and Practical specifying that we "focus[] on common use cases to stay minimal".

I appreciate this feedback and the time you took to provide specific examples of where this might be a burden in the future if implemented as described.

Overall, it's a tradeoff. Not making cryptography engines pluggable puts the cost of maintaining a FIPS fork on the organizations that need it (Google included), instead of adding complexity to the standard library used by most. Also considering that it's mostly well-staffed organizations that need FIPS compliance, I think that's the right call, and I don't think we should implement this.
Maintaining one of the simplest cryptography libraries in the whole ecosystem is an exercise in resisting complexity, and it sometimes comes at the cost of this kind of flexibility.

I completely understand and agree that I wouldn't want to do anything to unnecessarily complicate the language, as the simplicity of it is one of the main factors that drove me towards being a Go programmer many years back now.
I guess my last question would be: are we doomed to maintain forks for the rest of time, or is there still room for brainstorming a solution which have an acceptably small complexity vs flexibility tradeoff? As I mentioned before, if this proposal is not "the one" I'm not offended, my goal was to provide a jumping off point and open the conversation.

With respect to comparing alternate garbage collectors and alternate cryptography implementations, there's one pertinent difference: the business reasons for cryptography backends usually necessitate multiple backends. I'm unaware of any business reasons that would require, say, three different garbage collectors or runtimes.
On Windows, the correct option for a FIPS-compliant Go package is to use the CNG API (in particular, BCrypt.dll or NCrypt.dll). On Linux, one might choose OpenSSL. In addition to OS-specific packages, however, those who need FIPS compliance often have to deal with client-specific libraries or hardware.
This greatly complicates forks, because now instead of just one BoringSSL fork you're required to maintain N forks. Or, try to combine the forks into one. But that's not entirely possible because you can't always include client A's cryptography library (or your own code that accesses client A's hardware) in your fork if that fork is provided to client B.
And often you have to provide your fork to client B because the type of clients that require FIPS compliance quite often need to build the software themselves, or at least have a trusted intermediary build it for them.
For something like FIPS-compliant cloud storage this probably isn't a big deal. You can pick and choose your desired VM and only maintain a fork that runs on that VM. But the lack of pluggability makes writing software that the clients run on their own machines incredibly difficult.

Also considering that it's mostly well-staffed organizations that need FIPS compliance...

I very strongly disagree. Large companies like Google, Amazon, Cloudflare, etc. are not the only companies that require FIPS compliance.

With respect to comparing alternate garbage collectors and alternate cryptography implementations, there's one pertinent difference: the business reasons for cryptography backends usually necessitate multiple backends. I'm unaware of any business reasons that would require, say, three different garbage collectors or runtimes.
On Windows, the correct option for a FIPS-compliant Go package is to use the CNG API (in particular, BCrypt.dll or NCrypt.dll). On Linux, one might choose OpenSSL. In addition to OS-specific packages, however, those who need FIPS compliance often have to deal with client-specific libraries or hardware.

This is a great point as well. I'm coming from the point of view of maintaining a fork calling into OpenSSL (as opposed to BoringSSL) for RHEL, however the idea of potentially needing multiple forks within the same organization is a major pain point for consumers.

I just wanted to ping this thread as it seems the discussion may have stalled. As previously mentioned, this proposal may not be the silver bullet, however I would like to continue moving things forward. Ideally, users will not have to maintain forks of the stdlib in order to use a different crypto library.
If this isn't the solution, what can we do or propose going forward to alleviate the pain of those of us who are trying to use Go in a FIPS related setting?
cc @FiloSottile @rsc @agl

Here is my attempt to greatly summarize the main points and propose a compromise as FIPS 140-3 is coming very quickly and this would be the time to address this:
@FiloSottile brought up the key problems with Crypto Interface Engines (summarized greatly):

Behavioral differences in engine implementations cause issues
The interface will be prone to change due to new algs
It can be difficult to package complex behavior settings for an implementation into an universal interface

@rsc pointed out that this is mainly being driven by FIPS. @Freeaqingme brought that performance on specific machines could be another driver for crypto engines, but that seems to be a secondary concern as code cannot run certain environments if it does not have FIPS vs having it run better. This seems to also track with the fact that @derekparker and @FiloSottile are both maintaining FIPS Branches using different (but related) backends.
Right now the suggested paths forward have been:

Nothing changes, BoringCrypto and OpenSSL are kept in two different branches, but share much of the same code
The interface is adopted, large effort is spend to prevent @FiloSottile 's 3 pitfalls and while this opens up many different engine integration (personal note: hw accelerators are cool) it mainly will be used to just drop in FIPS .

Here is my suggested compromise: implement a crypto engine interface solely in the FIPS branches.
type FIPSEngine interface {
}

Addressing the 3 pitfalls in reverse order:

It can be difficult to package complex behavior settings for an implementation into an universal interface
These behavior settings are already being set in the mentioned FIPS branches. I would throw them all into a tlsconfig-like struct and worry about cleaning them up later.

The interface will be prone to change due to new algs
FIPS locks the algorithms used so the interface should change much less frequently.

Behavioral differences in engine implementations cause issues
The behavior is defined by FIPS and the implementation in the main FIPS branch (most likely BoringCrypto). It is up to everyone else to figure out how to implement it.This should allow the OpenSSL and Boring Branches to coordinate more. It also allows for alternatives to be considered, but puts the pressure alternatives them to match the behavior defined by FIPS.

Is this perfect? No cause I am sure someone who want to encrypt tons of data on disk may not care about some of the strict FIPS requirements. This should however help everyone coordinate on their main requirement, FIPS. It also opens the door up to alternative backends (hardware and software). Who knows, it could even allow more go code to be used and drop the cgo overhead.

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

Added to active to try to move discussion toward a decision. I asked in https://github.com/golang/go/issues/33281#issuecomment-518733756 about the motivation, and it sounds like the answer is "for FIPS" not "because crypto should be pluggable generally".
We still don't support uses of BoringCrypto at all, but it's worth noting that dev.boringcrypto has been merged into the main tree. So replacing crypto should be a matter of replacing just the one crypto/internal/boring package. Does that make it easier for people to maintain forks? Or does that provide a useful chokepoint for other replacements?

Ignoring FIPS compliance, there is still a good argument for "pluggable crypto" specifically to take advantage of crypto hardware. I know I cannot take advantage of a crypto accelerator in one of my embedded targets currently.  While crypto accelerators will likely be niche and covered by a fork, I think things like YubiKey and TPM integration have growing interests. I have not developed for the YubiKey yet, but I believe it is capable of performing all the needed PKI actions for TLSv13 internally, so the private key would not have to leave its boundary.  The only way I have seen Go leverage a YubiKey currently is to extract the key from it.

Hardware modules are supported through the crypto.Signer interface and friends, and there are definitely packages that do that, such as piv-go. If there are missing interfaces, we could consider adding those. In my experience, interfaces have been a cleaner way to support hardware modules than the engines approach of OpenSSL or the PKCS#11 approach of NSS, both of which caused complexity explosions.

@FiloSottile I don't see any examples of it, but yes the crypto.signer should allow the PKI to be offloaded to the YubiKey. Also tls.Rand is an interface that allows offloading. I guess the question is why not provide the same offloading capability for other cryptographic operations (symmetric key signing and ciphers, hashing, etc.)?
As for the engine vs interface debate, I think interfaces, like crypto.signer, could at least greatly drop the level of effort in making sure an application only performs a cryptographic operations the given way.

Added to active to try to move discussion toward a decision. I asked in #33281 (comment) about the motivation, and it sounds like the answer is "for FIPS" not "because crypto should be pluggable generally".

I think you’re correct in that being the main motivation, however I also feel that FIPS in and of itself is enough motivation to provide support for this feature. FIPS is a real-world requirement with regard to cryptography that companies must deal with. Go, having a cryptography package, should also have facilities to enable this sort of compliance out of the box. Go is very much a “batteries included” language with a robust and comprehensive standard library, so forcing users who must care about FIPS due to regulations into maintaining or using a fork of the standard library seems like the wrong approach. I would very much be interested in ways to make Go cryptography FIPS certifiable by somehow being able to have a Go crypto plugin (read: shared library of sorts) that can be certified, alleviating the need to use another crypto implementation, but that also implies the standard library being able to load a cryptographic implementation and use it, which is essentially what we’re already talking about here.

We still don't support uses of BoringCrypto at all, but it's worth noting that dev.boringcrypto has been merged into the main tree. So replacing crypto should be a matter of replacing just the one crypto/internal/boring package. Does that make it easier for people to maintain forks? Or does that provide a useful chokepoint for other replacements?

This change does indeed make things somewhat simpler from a fork maintainer perspective, however I think it’s still not ideal. Carrying patches to weld things together can be brittle and time consuming as rebasing can get messy over time, and in the end we’re still not shipping or using the “real” upstream Go toolchain, which carries with it its own set of risks and complications. A simple solution could be to allow Go module replace directives to replace stdlib packages so that one could simply swap that package out with another in a more supported way, however that proposal has already been declined previously[1].

Hardware modules are supported through the crypto.Signer interface and friends, and there are definitely packages that do that, such as piv-go. If there are missing interfaces, we could consider adding those. In my experience, interfaces have been a cleaner way to support hardware modules than the engines approach of OpenSSL or the PKCS#11 approach of NSS, both of which caused complexity explosions.

It’s true that interfaces allow for swapping out certain implementations, but that only affects code the user is writing directly. Other packages, and even other stdlib packages such as crypto/tls, may still be using the regular stdlib implementation. What this proposal allows for that interfaces don’t is the ability to swap out the entire crypto package implementation for all packages, even ones that are only included as a dependency that the developer did not write themselves.

We still don't support uses of BoringCrypto at all, but it's worth noting that dev.boringcrypto has been merged into the main tree. So, replacing crypto should be a matter of replacing just the one crypto/internal/boring package. Does that make it easier for people to maintain forks? Or does that provide a useful chokepoint for other replacements?

Speaking as a maintainer of the Microsoft Go fork, which implements CNG and OpenSSL crypto backends, yes, merging dev.boringcrypto back the the main tree has made our life much more easy. This is a small experience report:

The release infrastructure only needs to track one release branch, no more parsing of misc/boring/RELEASES.
Boring releases were released with a couple of days of delay, which was a problematic for security releases.
Boring is now a first-class citizen in the crypto packages, even if it is internal, which leads to a better boring integration. This translates into less patching and hacks on our side. For example, GOEXPERIMENT=boringcrypto is a pattern that we have replicated with GOEXPERIMENT=opensslcrypto and GOEXPERIMENT=cngcrypto, and having crypto/internal/boring/bcache as a separate package allows us to use it without depending on the whole boring package.

Going back to the Crypto Engines proposal, my gut feeling is that it would make our work as engine (aka backend) maintainer more difficult:

Reliably implementing UnloadEngine is not trivial nor clear to me which are the expectations if there are engine-allocated life objects when unloading.
This proposal does not mention how to deal with engines that can't implement the full engine contract. For example, our CNG backend does not implement SHA224, AES CTR mode, ECDSA P-224, and rsa.PSSSaltLengthAuto because the underlying does not have these capabilities.

Summarizing, I agree with @FiloSottile when he says that crypto.Signer and friends are the right level of abstraction for swapping crypto implementations, unless you need FIPS or some other niche requirement that is all-or-nothing.
On the other hand, @derekparker made a good point with this proposal: rolling your own crypto should be easier. Merging dev.boringcrypto is a step in the right direction, and I think that this path still has potential to simplify our work. I would explore it before turning to crypto engines.

Thanks for your detailed input @qmuntal!

Going back to the Crypto Engines proposal, my gut feeling is that it would make our work as engine (aka backend) maintainer more difficult:

Reliably implementing UnloadEngine is not trivial nor clear to me which are the expectations if there are engine-allocated life objects when unloading.

I think that can be part of the overall discussion. If unloading an engine proves problematic maybe it's not something we should allow. E.g. loading an engine would be a one-time, irreversible step.

This proposal does not mention how to deal with engines that can't implement the full engine contract. For example, our CNG backend does not implement SHA224, AES CTR mode, ECDSA P-224, and rsa.PSSSaltLengthAuto because the underlying does not have these capabilities.

The original proposal allows for this in a way. The engine itself is represented by an interface, with different algorithm types as methods. Taking a look back at the code example:
...
	if e := getEngine(); e != nil && e.AES() != nil {
		return e.AES().NewCipher(key)
    	}
...

You could imagine a scenario where an engine implementation only implements part of the interface, returning nil for those not implemented, falling back to stdlib implementation.

Summarizing, I agree with @FiloSottile when he says that crypto.Signer and friends are the right level of abstraction for swapping crypto implementations, unless you need FIPS or some other niche requirement that is all-or-nothing.

What about the case for wanting to ensure that every package in your program is using the crypto implementation you choose? Even with this interface approach the code you write can use whatever crypto you want, but if you import a package that imports crypto/aes directly there's nothing you can do about it at this point. I think you should be able to change things consistently across your entire binary.

On the other hand, @derekparker made a good point with this proposal: rolling your own crypto should be easier. Merging dev.boringcrypto is a step in the right direction, and I think that this path still has potential to simplify our work. I would explore it before turning to crypto engines.

I agree that we should exhaustively explore all options, crypto engines being just one of them and a way to get this conversation started with at least one concrete direction. Allowing replace directives to work against stdlib packages, or making the crypto package loadable via plugins (which could then mean the stdlib crypto library could be FIPS certified), or allowing for an engine approach similar to what's described in this proposal are all ideas.
To be clear I'm not staunchly in favor of the approach I proposed, I just wanted to get the conversation started and provide at least one path forward. The ultimate goal is to be able to supply user-provided crypto without the use of patch files, forks, etc...

I have started to write this comment half a dozen times and it's probably still a bit incoherent. Sorry about that.
One of the factors here I am not sure of is how much of a goal it is that a FIPS certified system contains no non-certified crypto code at all (certainly it is easier to prove that code that is not present is not being executed). If it is a goal, then it is certainly easier for a compile-time approach (like GOEXPERIMENT=boringcrypto).
It would be convenient for us (Ubuntu/Canonical) if we didn't have to recompile and separately distribute each Go binary that might be used on a certified system, but that seems incompatible with the above unless we build all Go applications to use openssl for crypto (which is probably OK for us as a distro maintainer, but it's quite a divergence from upstream), or this proposal is adopted to such an extent that all Go programs use some plugin-based interface to find their crypto code from a system path (which just doesn't seem like how Go works). I'm curious how other distros are handling this issue?

Our experience with BoringCrypto has been that each backend ends up imposing new constraints on the standard library. To the extent that the existence of BoringCrypto in the tree makes other backends easier to drop in, that's great. But I don't think we want to take on the extra work of supporting a general backend interface, for the reasons given earlier in the discussion.

The Zen of Python says:
There should be one– and preferably only one –obvious way to do it.[a]
This is a good principle for Go too.
So should Go allow us to chose between different crypto engines, to chose
different flavours or security? Or should it just provide one high quality engine which does the right thing?
The Go way is to be opinionated, do the latter.
If the Go stdlib is updated to allow me to chose which crypto engine to use,
then how do I chose which engine to use? And how will this make my code more secure?
Some people have argued that making Crypto plug-able is necessary in order to comply with government rules.
If the US government can demonstrate that the Go standard Library suffers from vulnerabilities which are fixed by FIPS compliance, then I am sure that these vulnerabilities will be fixed by the Go team.
But government rules can change over time and in arbitrary ways.  In the 1990s, the US government ruled that
all software written for the Department of Defence should be written in Ada. Who knows what governments will mandate in the future. Should we make all aspects of the language pluggable so that we can comply with current and future government rules? That way lies madness...

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

I completely understand the arguments towards keeping the language simple and continuing to provide the "batteries included" approach to the standard library, and especially the mindset of having one and simply one correct way to do things. These are all aspects that drew me to this language and community in the first place, and I wouldn't want to sacrifice them.
That being said, I believe we have already added complexity with the merging of the boringcrypto codebase to the standard library. Would this proposal have any merit / chance of acceptance if we continue the direction of hiding the complexity of it's implementation behind a GOEXPERIMENT flag? That way anybody who does not need the complexity of swapping crypto backends and dealing with FIPS doesn't have to worry about it, and we can isolate these changes a bit more.
(cc @rsc)

No change in consensus, so declined.
— rsc for the proposal review group

No change in consensus, so declined. — rsc for the proposal review group

Any thoughts on my last comment? https://github.com/golang/go/issues/33281#issuecomment-1284272560


chore(deps): update dependency ecmwf/cdsapi to v0.7.3

This PR contains the following updates:

Package
Update
Change

ecmwf/cdsapi
patch
0.7.1 -> 0.7.3

Release Notes

ecmwf/cdsapi (ecmwf/cdsapi)
v0.7.3
Compare Source
What's Changed

Update to cads-api-client by @EddyCMWF in https://github.com/ecmwf/cdsapi/pull/121

Full Changelog: https://github.com/ecmwf/cdsapi/compare/0.7.2...0.7.3
v0.7.2
Compare Source
This release enforces the usage of an updated version of cads-api-client (the api client for accessing the new CDS/ADS).
To check what's changed, please visit the cads-api-client change log  cads-api-client change log
Notes:

Technical/code issues related to the cdsapi applicable to the new system only, may be raised in the [cads-api-client](https://github.com/ecmwf-projects/cads-api-client repository or alternatively using the ECMWF Support Portal. Any other issues raised here or in the cads-api-client will be ignored.
Any other issues, requirements and suggestions for improvement to the  cdsapi  e.g. issues related to datasets should be raised with our Support team via the ECMWF Support Portal or alternatively shared with other users on the forum.

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.


cannot execure jar from Lubuntu

I have a fresh Lubuntu installation 16.10, desktop is LXDE
uploaded .jar file
upon execution command line
$ java -jar SwiftExplorer-1.0.8-jar-with-dependencies.jar
i get
"Error: Unable to access jarfile SwiftExplorer..."
I tried everything found in blogs: enable execution, install Java, install jdk... always with the same outcome
soooooo frustrated...

Try to install openjfx (for Java FX on Linux)


enable_time: true doesn't seem to work

I've been struggling to get Flatpickr to allow time selection. I'm using some custom code to generate the data attributes in HTML but I can confirm that HTML (from element inspector) shows the correct data-flatpickr-enable-time attribute:
<input autocomplete="off" data-controller="flatpickr" data-flatpickr-date-format="Y-m-d h:i K" data-flatpickr-enable-time="true" value="" type="text" name="shipment[dispatched_at]" readonly="readonly">
When a date is selected, the default time of 12 PM shows up but it can't be changed.

Closed at it's an isolated issue. I was using a custom CSS file that only worked after using the provided styles.


cmd/fillstruct: add option to select by line number.

Fixes #3.

Hm, I get a "could not find file" error when I try to use -line?
[~/go/src/a]% cat a.go
package main

import (
        "net/mail"
        "strings"
)

func main() {
        _, _ = mail.Address{}, strings.Reader{}
}

[~/go/src/a]% ~/go/bin/fillstruct -offset 81 -file a.go
{"start":71,"end":85,"code":"mail.Address{\n\tName:    \"\",\n\tAddress: \"\",\n}"}

[~/go/src/a]% ~/go/bin/fillstruct -line 9 -file a.go
fillstruct: could not find file "a.go"

@Carpetsmoker: I updated the PR, PTAL.

Thanks!
package main

import (
        "net/mail"
)

func main() {
        _ = mail.Address{}
        _, _ = mail.Address{}, mail.Address{}
}

[~/go/src/a]% ~/go/bin/fillstruct -file a.go -line 9
{"start":80,"end":94,"code":"mail.Address{\n\tName:    \"\",\n\tAddress: \"\",\n}"}
{"start":96,"end":110,"code":"mail.Address{\n\tName:    \"\",\n\tAddress: \"\",\n}"}

That output for the line with two structs isn't valid JSON:
[~/go/src/a]% ~/go/bin/fillstruct -file a.go -line 9 | python -m json.tool
Extra data: line 2 column 1 (char 84)

Not sure what would be a good/sane way to handle this; put it in an array? Or maybe the current behaviour is fine and clients should treat every output line as its own JSON document?
It would also make more sense to print it in reverse, by the way; I think. Because replacing the first struct would influence the offset of what follows (making the offsets that fillstruct prints invalid).
I can also give both -line and -offset:
[~/go/src/a]% ~/go/bin/fillstruct -file a.go -line 9 -offset 80
{"start":80,"end":94,"code":"mail.Address{\n\tName:    \"\",\n\tAddress: \"\",\n}"}
{"start":96,"end":110,"code":"mail.Address{\n\tName:    \"\",\n\tAddress: \"\",\n}"}

[~/go/src/a]% ~/go/bin/fillstruct -file a.go -line 9 -offset 2
{"start":80,"end":94,"code":"mail.Address{\n\tName:    \"\",\n\tAddress: \"\",\n}"}
{"start":96,"end":110,"code":"mail.Address{\n\tName:    \"\",\n\tAddress: \"\",\n}"}

It looks like the -offset is ignored. Maybe that should be an error? Or maybe it should try to find something at the -offset first, and then fall back to -line if that fails? That would make it possible to run fillstruct on a line with two structs, and have it fill only one of them.

Another small thing is an inconsistency with -offset if there aren't any structs; with the same file as above:
[~/go/src/a]% ~/go/bin/fillstruct -file a.go -line 2
[~/go/src/a]% echo $?
0

[~/go/src/a]% ~/go/bin/fillstruct -file a.go -offset 2
fillstruct: no struct literal found at selection
[~/go/src/a]% echo $?
1

@Carpetsmoker, thank you alot for your detailed feedback.
About the JSON output: fillstruct is used primarily by editor
integrations. So if the output as JSON array with the elements in
reverse order makes the integration easier, I'm happy to implement it
that way.
I like the idea of first trying to use the -offset information and,
if there was no struct literal found at that position, use the -line
information.
About the inconsistent error: when no struct literal is found: it
should definitively be consistent. I'm not sure whether it is better
to show an error or not. Also, I could not find a convention among
similar tools, e.g. honnef.co/go/tools/cmd/keyify does report an
error if there was no struct literal found at the given position,
while github.com/fatih/gomodifytags does not. I'm leaning more
towards showing an error and returning a non-zero exit code.

@Carpetsmoker, have you had the opportunity to look at my latest changes?

Ugh. I don't think I got notified about that, or if I did, it slipped through. Sorry about that >_<
I'll check over the weekend!

Seems to work well :+1: PR for vim-go here: https://github.com/fatih/vim-go/pull/1607

Great, thanks for your feedback.
The tool now always prints a JSON array instead of a JSON object to stdout. That's a breaking change—do you think I can just merge this PR or will it break vim-go for existing users? I'm not sure how to handle that. Do you bundle vim-go releases with the accompanying tools? Or should I tag releases so that users can pull a certain version of fillstruct for a certain version of vim-go? Thanks.

This PR will break vim-go's :FillStruct; it needs #1607 (which won't work with the current master).
We don't really have a strategy for managing this at the moment; :GoInstallBinaries is just a loop with go get ..
People reporting issues due to outdated Go tools is actually fairly common. We may need to think about a way to do this better, like bundling dependencies or something to that effect.
CC @bhcleek @fatih

I think this is a necessary change and we should merge https://github.com/fatih/vim-go/pull/1607 as soon as this PR is merged...

OK, thank you for your feedback—I'll try to keep breaking changes at a minimum.


chore(main): release 0.37.1

:robot: I have created a release beep boop
0.37.1 (2024-07-24)
Bug Fixes

bump guzzlehttp/guzzle from 7.9.1 to 7.9.2 (#639) (61de914)
bump laravel/framework from 11.16.0 to 11.17.0 (#640) (2c7a8c4)
bump league/commonmark from 2.5.0 to 2.5.1 (#638) (549978c)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

0.37.1
:sunflower:


Alias anon types in tiny-go

This fixes a bug where previously when an anonymous type had already been defined, we simply replaced the definition. However, this did not work as other parts of the codegen could already be referring to the anon type by that previous name.
This change keeps track of all the names used for a type. Then in the codegen for an anonymous type, it first looks to see if the type was already defined, and if so, it generates aliases for the previously used name.
I imagine we'll want to add a test that would have caught this. Do we want to add a new test or to change an existing test to exercise this? I think it should be as simple some uses in the wit file.

. Do we want to add a new test or to change an existing test to exercise this?

Let's add a new codegen test. @rylev

@Mossaka I extended an existing test because that actually seemed easier. I confirmed that the test fails without the fix.


UHF-3899: Add js functionality that disables other selects if one is …

…selected
How to test:

Checkout this branch and also require this branch: composer require drupal/hdbt:dev-UHF-3899_high_school_search_filter_changes.
make drush-cr
make shell
drush locale-import fi /app/public/themes/custom/hdbt_subtheme/translations/fi.po
drush locale-import sv /app/public/themes/custom/hdbt_subtheme/translations/sv.po
Import keywords from TPR using this command: drush migrate:import tpr_ontology_word_details
drush cr
For reason unknown to man keyword import needs to be done twice on the first time: drush migrate:import tpr_ontology_word_details
drush cr
Log into the site and go to /admin/config/school-settings path and set the year to 2021.
Create high school search and make sure there is enough items so that you get some filters for the paragraph.
If you select one filter the other one should turn disabled. If you return the value to "All" then the filter that was disabled should be usable again.
The disabled select should look like it does in hds (https://hds.hel.fi/components/dropdown).
Test the functionality with screen reader as well.

FYI: Still missing Swedish translations for the help-text so won't be merging before we have those.
Also check this one:
https://github.com/City-of-Helsinki/drupal-hdbt/pull/227

Looks good, had some issues with translations as discussed, but lets see if it works on test. Approved 🚀


template extraction issue

Hi,
Seeking help with an issue on template extraction (step 4).
A key seems to be missing from a dict. (see the attached)

Been able to generate the uspto  mapped csv successfully.
Thanks.

Hello.
This action, the "hasunmappedradicalatom" was introduce in rxnutils 1.2.
Could it be that you have an old version rxnutils installed in your environment?
Could try to do "pip install -U reaction-utils"?

Hello
Thanks for the response.
After the update, the error is not there.
However, the template extraction pipeline still fails (while executing template_pipeline.sh)

Did you install a Jupyter kernel?
ipython kernel install --name "aizynthtrain" --user

Hello
Yes.
I have rerun the train env set-up but have the same error.

Hello,
I'm having the same problem with kernelspec.
After following your suggestions (git pull & poetry update), no improvement.
I tried to specify the python_kernel in aizynthtrain/utils/configs.py in class TemplatePipelineConfig(BaseModel):  like this  python_kernel: str = "aizynthtrain"    --> no improvement.
A check with the command (jupyter kernelspec list) showed me that the kernel exists.
output = aizynthtrain       /home/.local/share/jupyter/kernels/aizynthtrain

Hello.
We encountered the same issue with our internal setup, and we found a solution. It is not elegant but it works for now
The thing is that is requires a new feature in the papermill package that has not been released, but this worked for us
conda activate aizynthtrain python -m pip uninstall papermill python -m pip install git+https://github.com/nteract/papermill.git


Output pose detection None

I want to use mediapipe/modules/pose_detection/pose_detection.tflite I try to watch the name of nodes graphs, but get RuntimeWarning
RuntimeWarning: Unexpected end-group tag: Not all data was converted graph_def.ParseFromString(f.read())

My code:
from tensorflow.python.platform import gfile
with gfile.FastGFile(TFLITE_FILE_PATH,'rb') as f:
    graph_def = tf.compat.v1.GraphDef()
graph_def.ParseFromString(f.read())
tf.import_graph_def(graph_def, name='')
graph_nodes=[n for n in graph_def.node]
names = []
print(len(graph_nodes)) # 0

Also i try get output from layer Identity:0, but get error that there is not that layer
Thanks

Hi @watermellon2018, Could you provide the complete details with respect to the use case and standalone code to reproduce the issue. Thank you!

@kuaashish
def detection(input_data):
    TFLITE_FILE_PATH = 'modules/pose_detection/pose_detection.tflite' # some path

    from tensorflow.python.platform import gfile
    with gfile.FastGFile(TFLITE_FILE_PATH,'rb') as f:
        graph_def = tf.compat.v1.GraphDef()
    graph_def.ParseFromString(f.read())
    tf.import_graph_def(graph_def, name='')
    graph_nodes=[n for n in graph_def.node]
    names = []
    print(len(graph_nodes))
    for t in graph_nodes:
        print(t.name)

path_img = 'frame.png'
img = cv2.imread(path_img)
img, _ = image_to_tensor(img)
img = img[:, :, ::-1]
img = img[np.newaxis, :]
img = img.astype(np.float32)
print(img.shape)
detection(img)

Hi @watermellon2018, Could you please provide the standalone code to reproduce the issue as shared code is not reproducible. Please find the gist for reference. Thank you!

@kuaashish
def detection(input_data):
    TFLITE_FILE_PATH = 'modules/pose_detection/pose_detection.tflite' # some path

    from tensorflow.python.platform import gfile
    with gfile.FastGFile(TFLITE_FILE_PATH,'rb') as f:
        graph_def = tf.compat.v1.GraphDef()
    graph_def.ParseFromString(f.read())
    tf.import_graph_def(graph_def, name='')
    graph_nodes=[n for n in graph_def.node]
    names = []
    print(len(graph_nodes))
    for t in graph_nodes:
        print(t.name)
def image_to_tensor(img, size=(224, 224)):
    img = cv2.resize(img, size)
    return img

path_img = 'frame.png'
img = cv2.imread(path_img)
img, _ = image_to_tensor(img)
img = img[:, :, ::-1]
img = img[np.newaxis, :]
img = img.astype(np.float32)
print(img.shape)
detection(img)

I solved this way:
def detection_model(input_data):
    interpreter = tf.lite.Interpreter(TFLITE_FILE_PATH)
    interpreter.allocate_tensors()

    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()
    interpreter.set_tensor(input_details[0]['index'], input_data)

    interpreter.invoke()

    detection = interpreter.get_tensor(output_details[0]['index'])
    score = interpreter.get_tensor(output_details[1]['index'])
    return detection, score # 1 2254 12 / 1 2254 1

Hi @watermellon2018, Good to hear that the issue has been resolved. Could we move this issue to close status as it has been resolved and hope does not require further support. Thank you!

Hi @watermellon2018, Could you please respond to the above comment. Thank you!


Create and lock entire resource

Hi,
is it possible to create and "lock" a entire resource? Like it was possible with the Resource Locker Operator?
We've tried different ways but none was working. Maybe somebody got an example for us :)
Thanks in advance!
potter

there wasn't much demand for that feature so I removed in the patch-operator. We also noticed that often the ability to create a resource can be met with operators like namespace-config-operator or kyverno.

Oh that's a bummer, but thank you for the hint to the other projects I will check them out ;)


Investigate new suit manifest

https://datatracker.ietf.org/doc/draft-moran-suit-manifest/ has a 3rd draft that also includes sample code to both create and parse these manifests. Investigate this new format.

This has been replaced by https://datatracker.ietf.org/doc/draft-ietf-suit-manifest/ in the process of becoming an RFC. We can track this for implementation efforts for SUIT.


child_process fork override breaks bull.js sandboxed processes

Issue description
The change introduced here https://github.com/wclr/ts-node-dev/blob/d498c0f602aa7e0f23492aadf41da944d501decc/src/wrap.ts#L32-L46
breaks the default behavior of the fork method in node.  This impacts the bull.js library which uses fork calls to create queue processors https://github.com/OptimalBits/bull/blob/33076495ab7d256eeed83e806a566846bc383117/lib/process/child-pool.js#L51-L65.
Overriding node internal like this create unexpected behavior.
Context
OS version (is it docker or host?), ts-node-dev version
ts-node-dev v1.1.1
Did you try to run with ts-node?
Did you try to run with --files option enabled?
Yes
Did you try to run with --debug option enabled?
No
Do you have a repro example (git repo) with simple steps to reproduce your problem?
Not at the moment but I will try to create one if I get the time

Hmmm, looks like I've ran into the same issue in #255
So I can't use Next.js anymore.


Update google-cloud-bigquery to 1.134.0

Updates com.google.cloud:google-cloud-bigquery from 1.122.2 to 1.134.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.google.cloud", artifactId = "google-cloud-bigquery" } ]

labels: library-update, semver-minor

Superseded by #236.


feat!: Move Dockerfile to the root of the project

Overview
Hello team,
This is an small change.
In order to standardize the repositories, I've moved the Dockerfile from the docker folder to the root of the project as celestia-app is, keeping the docker folder for possible uses like, creating other Dockerfiles, scripts, etc...
Next step will be to use a common CI from the .github repository.
Checklist

[x] Required CI checks are passing
[x] Linked issues closed with keywords

Closes Issue: 38

hey team!
thank you all for the review! 😊
I'll need a volunteer to click on merge when you can 😇
thanks in advance!


adding Neptune editor example

I think the failed build came from The command "grep -Eo "https?://[a-zA-Z0-9./?=_-]+" README.md | while read u; do echo $u; curl -LIs $u | grep "HTTP/1.1 200 OK\|HTTP/2.0 200" || exit 1; done" exited with 1. - not sure what that is about; I didn't touch the readme.

Looks amazing, thanks!


benchmark use JITBEAM for llama and gpt2

try this again

oh those are the clear_l2 kernels

speed is similar now


Bumping jsx-ast-utils to enable objectRestSpread

Here's the requisite bump in jsx-ast-utils to enable objectRestSpread.
This should address #327 and close https://github.com/evcohen/eslint-plugin-jsx-a11y/pull/356 when published — for some reason the package.json showing for eslint-plugin-jsx-a11y in NPM has<EMAIL_ADDRESS>but should be ^2.0.0.

LGTM, but this can't pass until v2.0.2 is available.
Separately, it's not necessary because ^2.0.0 automatically includes v2.0.2.

Just wanted to get a change in to make sure that a new version of
eslint-plugin-jsx-a11y gets into npm, so whichever way does it (bumping
this version, or just publishing a new release) works for me!
On Thu, Dec 7, 2017 at 4:27 PM Jordan Harband<EMAIL_ADDRESS>wrote:

LGTM, but this can't pass until v2.0.2 is available.
Separately, it's not necessary because ^2.0.0 automatically includes
v2.0.2.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/evcohen/eslint-plugin-jsx-a11y/pull/363#issuecomment-350136259,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABERRnxP__9QXG1ShXibXXfuXVyll30Dks5s-IJcgaJpZM4Q537d
.

This should come for free now that v6.0.3 is published


Why a cant use play/pause buttons?

JS:
let elements = document.querySelectorAll(".el");
    let playBtn= document.querySelector('.play');
    let pauseBtn= document.querySelector('.pause');
    const circle = anime({
	    targets: elements,
	    left: "240px",
	    borderRadius: ["0%", "50%"],
	    duration: 1000,
	    easing: 'easeInOutQuad',
	    autoplay: false,
	    // loop: true,
    });
playBtn.onclick = animation.play;
pauseBtn.onclick = animation.pause;

HTML:

	<button class="play">Play</button>
	<button class="pause">Pause</button>

Why my play/pause buttons doesnt work?


Replace examples with valid words

Because the example synonyms contain invalid special character "-" (Hyphen/dash), replace them with valid words.

@nkgami : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@jboback
Can you review the proposed changes? IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
#assign:@MicrosoftDocs/public-repo-pr-review-team

#sign-off


use value of :ssh_server_domain for plugin connections

Should fix issue #431

it is now possible to use gitolite from an external server
add ssh port display in repository-urls only if needed (Fancy).

Done in commit https://github.com/jbox-web/redmine_git_hosting/commit/10aec4c1c4ea6bd71e0c8165b3b28791eb767e6f

Thank you ! It is the right solution.
I'll try it asap.


Memory leak

Although DLLExport is working perfectly here I have an issue with memory consumption. However, it looks like this is only caused by missing knowledge on my side. Maybe someone can point me to the right direction. Scenario:
VS2017 Project 32Bit. A c++ DLL loads a CLI DLL by using LoadLibrary. It queries values from the CLI DLL y using:
C++
typedef wchar_t*(__stdcall OPCREAD)(char opcitem); // Read OPC item
sprintf(buf1, "%s", OpcRead(m_stat.pini[m_stat.iAliveCommand].command)); // iAliveCommand].command = *char
CLI
[DllExport(CallingConvention = CallingConvention.StdCall)]
public static string Read(string opcitem)
The CLI DLL returns a string
This is working perfectly however, with every call to the CLI DLL the memory usage increases by a few bytes which blows up the program.
If I just return an integer from the CLI DLL it looks like the issue is gone, of course this is not a solution.
Can someone please tell me which side is responsible for the issue and how this can be fixed. Yes, I have read:

Do not use complex types for arguments & for return value.
For example, IntPtr instead of String.

But I have no idea how this is correctly done on both sides, especially how to free memory.
Thank you in advance!!
Martin

Except for your leaks in your code the CLR will also use marshalling mechanism anyway. Because even integer type is not really equal to an unmanaged type. Thus IntPtr is recomended (at least by me) for more control even if some types still may work perfectly for some cases.
As I already wrote:

Types are not identical. PInvoke-mechanism mainly operates with the sizes for the data, not its types.
A call at the received address can be with any similar signature (compatible with required arguments and optionally return value).

That is, the ~call-mechanism is flexible enough to avoid some specifications.
For more information, please use MSDN and other related documentation.
But you can also use my old an examples and explanations, for example:

Did you look our examples? I specially illustrated this via detailed screencasts for most popular ways. Including various host sides (Conari will perfectly help for CLR side)
So for complex types and IntPtr you should see exactly Part-2:

https://www.youtube.com/watch?v=QXMj9-8XJnY

https://github.com/3F/DllExport/wiki/Examples-Part-2  (+source code from an examples)

But I have no idea how this is correctly done on both sides, especially how to free memory.

+including alloc/free samples.

I got it fixed by myself. For anyone facing the same issue here's what worked for me:
Leave the C++ typedef as it is e.g. typedef char*(__stdcall DEVICEPTR)(char c);
DEVICEPTR DevicePtr;
DevicePtr = (DEVICEPTR)GetProcAddress(hClcltr, "GetString");
if (DevicePtr)
{
	printf("String; %s\n", DevicePtr("First attempt Ptr"));
	printf("String; %s\n", DevicePtr("Second attempt Ptr"));
}

C#
I changed public static string GetString(string InString) to:
public static IntPtr GetString(IntPtr InPointer)
{
// Marshal the unmanaged string to managed memory.
ReceivedString = Marshal.PtrToStringAnsi(InPointer);
// Perform CLI tasks here...
		// Marshal the managed string to unmanaged memory.
		OutPointer = (IntPtr)Marshal.StringToHGlobalAnsi(ReceivedString);

		// Free the unmanaged string.
		// Marshal.FreeHGlobal(OutPointer); // This also works

		return OutPointer;
	}

After executing the function I call the following CLI code from C++:
	[DllExport(CallingConvention = CallingConvention.StdCall)]
	public static void CleanString()
	{
		if (OutPointer != IntPtr.Zero)
			Marshal.FreeHGlobal(OutPointer);
	}

	#region Private Members
	private static string ReceivedString = null;
	private static IntPtr OutPointer = IntPtr.Zero;
	#endregion

Hope this helps anyone who faces a similar problem.
Martin


DS-3190: Refactor org.dspace.rdf.* to use spring.

https://jira.duraspace.org/browse/DS-3190
dspace-rdf was developed as standalone module. At the end of the
development most of it was moved to be included into dspace-api. Instead
of using Spring it used its own class that instantiated all necessary
classes. This PR solves the problem described in DS-3190 by refactoring
org.dspace.rdf to adapt patterns used in DSpace 6. It introduces
[dspace]/config/spring/api/rdf.xml to configure spring to instantiate
all necessary classes. org.dspace.rdf.RDFConfiguration was removed
completely, the configuration keys are centralised in
org.dspace.rdf.RDFUtil. Instead of org.dspace.rdf.RDFConfiguration and
DSpace's old ConfigurationManager the new ConfigurationService is now
used everywhere. Configuration properties shouldn't be hold in variables
anymore so that RDF profits from the new autoreload function from
DS-2654.

To the (limited) extent that I understand the purpose of dspace-rdf, this patch looks sensible.
+1 by inspection.

+1 via code review. This looks sane to me, and it fixes the existing RDF issues on master.


Icon synonyms

Hi there,
thank you for your great work with these icons! ❤
I noticed that you have some kind of synonyms on your website, since when I search for "fitness", there are also some sports icons displayed there. I want to include such a search in my project as well and was curious whether you can share your already available synonyms/metadata regarding this?
Best regards

Ah, just found them at https://atlasicons.vectopus.com/api/category/categories. 🙂


Change extended key version bytes to BIP32

-Most compatible solution

I'll leave it here for reference: https://www.reddit.com/r/litecoin/comments/48wd2e/why_arent_the_litecoin_bip32_serialization_values/

The proper values are the ones in the reference client. The same convention of using xpub/xprv was also adopted by Electrum-LTC. You may run into software using Ltub/Ltpv, but that format should be considered non-standard.
By the way, let me point out that this is only a presentational matter. Extended keys can be converted from the Ltub/Ltpv convention to the standard one relatively easily.

We'll follow the same path and use xpub/xprv starting with 0.12.1

utACK c874a0f

Looks good for me, thanks!


The block's damage state does not reset bug on consecutive break

https://github.com/user-attachments/assets/82ea2ba8-24a7-4aff-a051-333d881fb9c4
A bug where consecutive breaks don't reset.
commit hash: 4b121393724f3f1da4ac3edcb3a3af0b79093928

for anyone interested in fixing this, you should likely start by looking here.
https://github.com/andrewgazelka/hyperion/blob/e272fa0255dd9344e0db44945cd844a66c8915c8/crates/hyperion/src/simulation/handlers.rs#L264-L294

I don't have this issue

I don't have this issue

I do. hold down mine while block replaces itself.
also these are unimplemented 🤷
tag-1             | 2024-12-17T20:54:51.987194Z ERROR crates/hyperion/src/ingress/mod.rs:619: failed to process packet BorrowedPacketFrame { id: "0x1d", body: b"\0\0\0\x02\x80\0\x01\xf0\x13\x01\x13" }: unimplemented StartDestroyBlock
tag-1             | 2024-12-17T20:54:57.874901Z ERROR crates/hyperion/src/ingress/mod.rs:619: failed to process packet BorrowedPacketFrame { id: "0x1d", body: b"\0\0\0\x03@\0\x01\xf0\x13\x01\x15" }: unimplemented StartDestroyBlock

I've started working on this


Build standalone game

Hello,
I have used RoadArchitect to create road network in runtime using script.
I need to build all roads when the player starts the game, but I am getting build error because of UNITY_EDITOR! Is there anyway to change the source code to build standalone game?
Or is there any patch to support this?
Thank you

Upstream RoadArchitect does not support building roads at runtime and likely never will.
However, I think @FritzsHero may have been working on this functionality in his fork.

I have write access to this repository, and I'm certainly willing to merge
in PRs, but don't have time for much else myself.
On Mon, Jul 19, 2021, 12:56 embeddedt @.***> wrote:

Upstream RoadArchitect does not support building roads at runtime and
likely never will.
However, I think @FritzsHero https://github.com/FritzsHero may have
been working on this functionality in his fork
https://github.com/FritzsHero/RoadArchitect.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/MicroGSD/RoadArchitect/issues/90#issuecomment-882704884,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEFQ6Z7L47U6ZDJFLVRJFU3TYRKMTANCNFSM5AUCP7MQ
.


Deprecated code##3

What is the code smell/issue found?
Deprecated classes, interfaces should be avoided for using, inheriting, extending. Deprecation is basically an indication that a particular class has been superseded and will be eventually be removed. The deprecation period allows us to make a smooth transition away from the aging, soon-to-be-retired technology.
How is this code smell relevant?
Deprecated code is code that is very old in age and no longer being used, this can increase the complexity and will make maintainability of the code difficult.
How can we resolve this issue?
To resolve this issue we have removed "getSettings()" as it is deprecated and replaced it with "get_Settings()"

rejected
The class SettingsUtil does not have a method get_Settings(), so this code does not compile. I don't understand how it solves the problem of deprecated code.


Dash/Minus in column name causes UNBOUND_SQL_PARAMETER in bind values

I am trying to build an ORM over existing database (which I have no architectural influence over). I am running into an issue where some of the column names have a "-" in their name. In short, this syntax is carried over into the bound parameter where it causes issues.
This trimmed down example outlines this issue:
class Person(Base):
    __tablename__="persontable"
    id: Mapped[int] = mapped_column(name="id", primary_key=True)
    first_name: Mapped[str] = mapped_column(name="first-name")

Note how the column name in the table has a dash in it.
Now, if I build a query that would parametize this value and run it:
stmt = select(Person).where(Person.first_name=="John")
# print(stmt):
# SELECT persontable.`id`, persontable.`first-name` FROM persontable WHERE persontable.`first-name` = :first-name_1

with Session.session() as session:
   johns = session.scalars(stmt)

An error is raised
sqlalchemy.exc.DatabaseError: (databricks.sql.exc.ServerOperationError) [UNBOUND_SQL_PARAMETER] Found the unbound parameter: first.
The way I see it is that :first-name_1 is not taken as is, but the - is interpreted as a subtraction operator.
I assume this is an oversight in implementing the dialect. I am hoping someone could assist with a workaround or implement a solution. Please note that renaming the column on the table directly is not an option for me.

Your analysis is correct. The SQL parser within databricks is reading  the parameter marker as an arithmetic operation. This isn't uncommon for some databases, including Postgres. There is a workaround for this described by the author of sqlalchemy here. Just modify your mapped_column definition like so:
first_name: Mapped[str] = mapped_column(name="first-name", key="first_name")

Fantastic, this works. Glad to see there is a workaround for this already!


chore: release  java-client 0.3.0

:package: Staging a new release
0.3.0 (2024-11-13)
Features

java-client: add VC client builder (#28) (046c204)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/carbynestack/thymus/releases/tag/java-client-v0.3.0 :sunflower:


[in_app_purchase][Android]: queryPastPurchases returns empty list in release build (works fine in debug mode)

Steps to Reproduce

Run app in release mode with flutter run --release
Make a purchase for subscription (weekly)
Get subscription successfully
When subscription renews, queryPastPurchases returns empty string

** It works fine in debug mode on Android, and works fine both in release and debug mode on iOS**
Expected results: Fetch the past purchases with success
Actual results: Got an empty list of QueryPurchaseDetailsResponse

Code sample

@hbt1903
Can you provide flutter doctor -v, plugin version and a minimal reproducible code sample that shows the issue ?

flutter doctor -v
[✓] Flutter (Channel stable, 2.10.0, on macOS 12.2 21D49 darwin-arm, locale tr-TR)
• Flutter version 2.10.0 at /Users/hbt/development/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 5f105a6ca7 (7 days ago), 2022-02-01 14:15:42 -0800
• Engine revision 776efd2034
• Dart version 2.16.0
• DevTools version 2.9.2

[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
• Android SDK at /Users/hbt/Library/Android/sdk
• Platform android-32, build-tools 32.1.0-rc1
• Java binary at: /Applications/AndroidStudio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• CocoaPods version 1.11.2
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
[✓] VS Code (version 1.64.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.34.0
[✓] Connected device (1 available)
• Chrome (web)          • chrome          • web-javascript          •Google Chrome 98.0.4758.80
[✓] HTTP Host Availability
• All required HTTP hosts are available
• No issues found!

Code Sample
I work for a company so it is not possible to provide a reproducable code sample since we handle a lot of work on our servers (like verification of receipts, storing receipt datas)
Basically I initialize the app with the following function and I expect to get past purchase details succesfully.

Btw. I am sure that the applicationUserName = UserService.instance value does not change

Future<void> _initAndroid() async {
    InAppPurchaseAndroidPlatformAddition.enablePendingPurchases();
    _iap = InAppPurchase.instance;
    InAppPurchaseAndroidPlatformAddition _androidAddition = _iap.getPlatformAddition<InAppPurchaseAndroidPlatformAddition>();
    QueryPurchaseDetailsResponse _resp = await _androidAddition.queryPastPurchases(
      applicationUserName: UserService.instance.userId,
    );
    List<GooglePlayPurchaseDetails> _details = _resp.pastPurchases;
    pastPurchaseDetailsAndroid = _resp.pastPurchases;
    _details.forEach((element) async {
      _handleIAPResponse(
        await ReceiptVerifier.verifyReceiptAndroid(element),
      );
    });
  }

I expect to have _details a non-empty list after making a purchase.
It is empty only in the release mode on Android.

@hbt1903
Can you check this issue comment and related code snippet and see if it helps ?

@darshankawar
Thank you very much, I'll try it 👍
But still, isn't it weird that working properly in debug mode?

@darshankawar
restorePurchases seems to works fine in release mode, thank you!


Emulate variadic JVM functions

http://stackoverflow.com/questions/28951503/how-can-i-create-a-function-with-a-variable-number-of-arguments

For the time being we just call Call*MethodA().


lsp-sample not working from VSIX package

Hello,
Repro steps:

clone the repository
open  lsp-sample in VSCODE
npm install
npm run compile
start debug
extension is working
close vscode debug window
npm install vsce -g
change publisher name in package.json
vsce package
Ctrl+P -> Extensions: Install from VSIX...
selected the generated VSIX file
extension is installed with sucess
open a text file
extension doesn't work

It seems that the language server is not being started for some reason. I tried with other lsp samples and the same happens. They don't work within the VSIX package. Is there any step I am missing?
Thanks in advance...
Rodrigo

I am seeing this too. I followed the exact steps. I can see the extension listed as installed, but it's not working when installed using VSIX package.

I found that the problem was a package not being included in the VSIX. The following issue highlights the problem: https://github.com/microsoft/vscode-extension-samples/issues/412
I found that when I launched VSCode and looked in the Developer Tools, I could see an error due to lru-cache not being found.
Modifying my vscodeignore as per the Issue above solved my problem and I could use the extension.
--- a/.vscodeignore
+++ b/.vscodeignore
@@ -11,7 +11,8 @@ client/node_modules/**
!client/node_modules/vscode-languageclient/**
!client/node_modules/vscode-languageserver-protocol/**
!client/node_modules/vscode-languageserver-types/**
-!client/node_modules/semver/**
+!client/node_modules/{minimatch,brace-expansion,concat-map,balanced-match}/**
+!client/node_modules/{semver,lru-cache,yallist}/**

Updated the example to have a correct .vscodeignore file


PropPanel: Rename "Target" to "Called element" for CallActivites

Describe the Bug
Currently the zeebe:calledElement Props are having a group label called Target

=> This is very confusing to the user.
Steps to Reproduce

Open a Call Activity in Camunda Cloud

Expected Behavior
Rename to Called element. This is in line with

e.g., BusinessRuleTask (Called decision)
The XML

Environment
Camunda Modeler 4.11.1

cc @andreasgeier @barmac

Makes sense

In the BPMN specification, we named it 'Called Element' because it allows to not only call processes but also global tasks that are defined once in a model and can be reused. Since our engines don't support global tasks, we could make it even more intuitive for our users and name it 'Called Process' in our UI.

@MaxTru, @andreasgeier and I discussed this and we want to rename this to "Called element" for consistency sake. It's the same thing so let's use the same name.

Fixed upstream via https://github.com/bpmn-io/bpmn-js-properties-panel/pull/516


Fix on-prem login link for login modal

Closes https://github.com/plotly/streambed/issues/11288

@tarzzz I just changed the branch to 3.0-onprem like our previous PR, but let me know if you would like me to change it back and/or make another PR for another branch.

This looks good..
Please feel free to merge it after fixing test failures.


Move to eslint from jshint & jscs

Started using eslint for the d2-ui components. Should start using the same styleguide here based on https://github.com/dhis2/eslint-config-dhis2

Done for src not yet done for test code @ see #40


[6.0🍒][Concurrency] Allow 'nonisolated' to be applied to mutable storage of 'Sendable' type on a globally-isolated value type.

Explanation: Allow 'nonisolated' to be applied to mutable storage of 'Sendable' type on a 'Sendable' value type.
Scope:  Concurrency model improvement.
Issue:  rdar://130992526, rdar://131339439
Original PR: #74958, #75084
Risk: Low (additive change)
Testing: Added tests to the testing suite.
Reviewer: @hborla

@swift-ci test

@hborla could you please approve?

@swift-ci test


Typescript

Can I use Dayjs-jalali in typescript?

I plan to do some fixes on the library. Also, adding types to better usage in Typescript.
However, you can use it now in Typescript.


Puppet nodes can't find modules otherwise

The modulepath config settings was deprecated a while ago in favor of basemodulepath, which is why this commit was merged back in May. Are things not working for you as expected with newer Puppet versions?

Hi,
It does not work as is.
vagrant@client1:~$ puppet --version
3.4.3

Ok. So puppet > 4.0 should be used.

Actually, it requires a change anyway since Vagrantfile uses box = 'ubuntu/trusty64' and by default its comming with puppet 3.4.3

Maybe something like: https://github.com/petems/vagrant-puppet-install can be used

I've been working on Puppet 4 support, but haven't found the time lately:
https://github.com/martialblog/puppet-sandbox/tree/update-to-puppet-4
But it's basically working.


feat: config as own crate

What I'm imagining is using an alias when importing a contract:
import_contract!(hello_world);

Then you would need to use an env variable to set the network so that it could look up the alias. You could also do the same for keys, but this one would make it a much better experience and make the hello_world  module feel like a real rust import.

Feels right to move logic like the shared args out of the network commands module into a shared place due to the common nature and the args not being scoped to the network command.
The refactor looks good to me in that context 👏🏻 .

This separates it out so that we can begin the process of pulling it out into it's own crate.

That being said, the refactor groups cli code together, but doesn't separate out config parsing/writing logic from tightly coupled cli logic, so I'm not sure this change sufficiently prepares a module for extraction to another crate.

This should become it's own crate to allow devs to use accounts and aliases when compiling contract.

Could you share more about this workflow? I wouldn't expect compiling contracts to embed aliases or account data into contracts, and even if the cli needed to do that the cli can already access them.
I am probably missing a critical part of how this will serve folks, but moving config into a separate crate seems very low impact / priority at the moment. What am I missing?

Originally posted by @leighmcculloch in https://github.com/stellar/stellar-cli/pull/1479#pullrequestreview-2200072839

import_contract!(hello_world);

Early on there was an issue open for something like this:

https://github.com/stellar/rs-soroban-sdk/issues/728

However, once contract upgradability was added, the contract ID no longer points to a consistent code file which would make reproducibility challenging. Reproducible builds is something being pursued, see this Discord thread for the latest on that.
Another issue with importing contracts from an alias is that it assumes the alias is part of the project that gets shared, but the local .soroban directory is git-ignored, and a user might have the alias stored outside the project either in a parent directory or in the global config.
To explore auto-downloading the contract it'd probably need to accept a network passphrase (or name) and the wasm hash. It's relatively easy for folks to figure out what the wasm hash is from an explorer.

I guess for me I'm interested in using it for our loam which has a contract and wasm-hash registry. I have had this working for my previous project smartdeploy. But that used it's own alias system. But what was nice is you could do smartdeploy install hello_world and it would add it to your local target directory, and I had the plan of adding it to the Cargo.toml. Loam now has an environment.toml which will be where the contracts aliases to be installed will be.
Anyway we can copy the code that is required, but we wanted loam_sdk to not depend on the CLI to get access to the config.
As for the issue with the CLI args being intertwined. I agree it's an issue, but it's also useful for those that would want to make their own CLI with access to the config logic. The solution to still using it in an sdk to to hide the clap parts behind a feature.
So the really the question is do we want to provide the config as a separate tool? Would there be users beside loam that would be interested?

I understand the motivation in reducing code duplication across CLIs that'll have similar uses and similar UIs using the CLI config benefiting from sharing the clap setup, but areas of code we've used that same pattern with between the stellar-rpc-client and the cli have made it difficult to maintain the CLI so I think we need to keep the CLI presentation layer within the CLI.

Could we close this ticket? Is this important for Q3?

It's not critical for the cli and without getting the boundary right I think this issue risks reducing the maintainability of the cli. So I suggest we close and not focus on it in the near future.


Documentation example for Installing Industrial Modules requires clarification and missing comma

Describe the bug
Documented instructions to install the Industrial Modules on IoT Edge lack clarity and have a typo (missing comma) in example.
To Reproduce
Steps to reproduce the behavior:

Go to 'https://github.com/Azure/Industrial-IoT/blob/master/docs/deploy/howto-install-iot-edge.md#install-the-industrial-modules'
Read the instructions
Figure out exactly what you're meant to do

Suggested Documentation Addition
Add a sentence after "The Device Twin configuration JSON can be found in the Azure Portal under IoT Hub -> IoT Edge -> [your IoT Edge device] -> Device Twin." explaining to the person what it is that they are expected to do (not just providing an example).
You'll need to add the "tags" section into the JSON document in between the version and properties sections.
Fixed Example Code
The example code for both Linux and Windows seems to be missing the comma after the tags section insert - resulting in an error in Azure Portal and Device Twin changes not accepted.

@gregeva Thanks for reporting this. What do you think of the changes in the attached PR?


Partition databases between CPU cores

We already have the infrastructure for thread-per core, but we still need partition databases by core and steer requests to the right partition.

I have experience with building thread per core mesh shard using monoio, by looking at the commit history you've initially went with monoio as your runtime of choice, might I ask why you decided to ditch it and go for the tigerbeetle inspired I/O loop ? Is it because of the simulator ?

Yes, wanted to go for TigerBeetle style approach for deterministic simulation testing.