Sort file error - no such file or directory "defaultSort.json"

I'm receiving this error whenever I run your tool.
Sort file error - no such file or directory "defaultSort.json"

It doesn't seem to be affecting anything as it completes and sorts my spec.
I've worked around by specifying --sortFile ""

Sorry, I don't have time to make a contribution here today -- But I wanted to let you know that I really appreciate you building this tool. It's solving a real pain point for us in our large spec file and I'm amazed you're the first one to build/share something like this. Thanks!

Hi @philipbjorge
Sorry for the inconvenience. The --sortFile parameter should be optional.
I want to make the experience as good as possible, can you provide me some information:

the full command and parameters that you run to get this error
if you run it on Windows or Mac or Linux machine

I assume you are using the latest version of the package.
Once, I have the input I'll try to reproduce the problem and try come with a fix.

@thim81 - No inconvenience :) - Thank you for the tool.
The command:
npx openapi-format descriptions/v1/api.yaml --output descriptions/v1/api.yaml
Using<EMAIL_ADDRESS>
I suspect the issue is...

--sortFile defaults to defaultSort.json - https://github.com/thim81/openapi-format/blob/main/bin/cli.js#L19
cli tries to read it and fails - https://github.com/thim81/openapi-format/blob/main/bin/cli.js#L90
Then the formatter loads up the defaultSort from the package - https://github.com/thim81/openapi-format/blob/main/openapi-format.js#L152

Cheers!

@philipbjorge You were right.
Thanks to your example, I was able to reproduce it and fix it.
I just pushed a new version (1.2.1) which solves this issue.


Are most microfacet-based OSL shaders physically incorrect because of fresnel?

Been trying my hand at OSL a fair bit the last couple days - it is pretty awesome to be honest, great work!  I'm trying to implement Disney's PBR shader in pure OSL and I hope to contribute it back to this project once done.  I have run into a few problems though.  Here is the first:
I notice this pattern or equivalent in a lot of OSL shaders (for example this one):
Ci = (Kr * Cs) * microfacet("default", N, U, xalpha, yalpha, eta, 1);

I believe that the microfacet shader itself already has dielectric Fresnel built-in, hence the eta term.  Fresnel should go to 1 when dot( N, I ) = 0.  Basically there should be full reflection at grazing angles.  But the above pattern doesn't allow that because even if microfacet() returns 1.0, it is modulated by the specular color.
Basically this means that at anything approaching grazing angles, if you use a non monochromatic specular color / coefficient, you are getting physically incorrect results.  Almost every shader I see in OSL that has a specular component has what I believe is this "error."
I believe that either eta needs to be a color, which you can compute based on the specular color... something like this for each channel:
// from solve( f0 = (( 1-n ) / (1+n))^2, n ), assuming other medium is air.
float f0ToDielectricRefractionIndex( float f0 ) {
	return ( sqrt( f0 ) + 1 ) / ( 1 - sqrt( f0 ) );
}

Or you need to incorporate specular color into the microfacet distribution call so that it can be modulated to white at the grazing angles.
Or you need to call the microfacet distribution 3 times, once for each of the eta values representative of the specular color.
Disclaimer: I am somewhat new to PBR so I could be wrong about this.

Maybe the issue is that there is no non-monochromatic specular with dielectric fresnel materials in physical reality?  I personally can not think of one off the top of my head.  But then why do so may of these models allow for that situation when it is incorrectly represented by these shaders?
BTW the Disney model uses uses a specular color instead of an eta, and uses Schlick's approximation to Fresnel, which always goes to 1, no matter what the F0 color is:
// use Schlick's approximation to calculate Fresnel curve using f0 as specular color
color dielectricFresnelReflectance( color f0, vector light, vector half ) {
	return f0 + ( 1 - f0 ) * pow( 1 - dot( light, half ) ), 5 );
}

Hello Ben,
The lack of non-monochromatic /and/ complex fresnel term is indeed a big issue:  you can’t do metals as it is.
In our OSL shaders, we patch this by having our own fresnel outside of the closure. Note that this is wrong by design though: fresnel has to be applied to each sample, and not on a closure (which is sampled at some <wi,N,wo> at which point you need to apply the proper fresnel function).
In the near term, we will probably add a “complex_ior” parameter (or similar) to the micro-facet distributions.
Still, it would be good to have a standard way of achieving this,
—
Aghiles
www.3delight.com

On Oct 17, 2015, at 10:16 PM, Ben Houston<EMAIL_ADDRESS>wrote:
Maybe the issue is that there is no non-monochromatic specular with dielectric fresnel materials in physical reality? I personally can not think of one off the top of my head. But then why do so may of these models allow for that situation when it is incorrectly represented by these shaders?
BTW the Disney model uses uses a specular color instead of an eta, and uses Schlick's approximation to Fresnel, which always goes to 1, no matter what the F0 color is:
// use Schlick's approximation to calculate Fresnel curve using f0 as specular color
color dielectricFresnelReflectance( color f0, vector light, vector half ) {
return f0 + ( 1 - f0 ) * pow( 1 - dot( light, half ) ), 5 );
}
—
Reply to this email directly or view it on GitHub https://github.com/imageworks/OpenShadingLanguage/issues/567#issuecomment-148970402.

Hi Aghiles,
Nice to run into you again and to have my suspicions confirmed.  I'll do the Fresnel outside of the closure as well -- it is simple enough.  I do not know the resulting quality implications of this or speed.
Interestingly, the now classic PBRT book has a Fresnel class that is taken as an input to the Microfacet distribution, and they subclass the Fresnel class to represent both dielectics and conductors and the no-Fresnel case.  The class interface is a single function that takes a cosI and returns a spectrum.  I wonder if something like that is possible as an input to the OSL Microfacet closures?
https://github.com/mmp/pbrt-v3/blob/master/src/core/reflection.h#L245

Hi
We have our own "conductor" closure which is similar to microfacet but implements this complex ior feature. We haven't made it part of the "standard" because, to be honest, we don't seem to agree whether there should be a standard for things like this. Every render does things differently, so is not clear what the closure/protoypes should look like except for very obvious cases.
Alex

Everybody seems to use their personal GitHub logins, sometimes I too need a roster to remember which userids correspond to which places.
In case it's not clear, Alex is at SPI. We have indeed phased out our direct use of microfacet, and in general the "combine separate BSDFs in one material". Two BSDFs that are energy preserving individually can be not so when combined if they are not carefully paired. Now we are phasing in more holistic BSDFs that are harder to get wrong no matter how you dial the parameters. Though when we are "layering" or truly mixing materials, we can still have aggregate BSDFs coming out of the shader, but that's a somewhat different situation.
I'm a part of the faction that wants a standard set of closures to be part of the OSL spec, but it is very difficult -- at SPI we have completely revamped our set of closures several times, and are in the midst of a new overhaul now. Every time we say "did we get finally it right this time?" And every time I'm glad we didn't enshrine the last one in the spec and get a little embarrassed about the mismatch between the spec and what we are doing in practice. (In contrast, the language itself has been incredibly stable, as have been the compatibility of all shaders except the "root" node that creates and assembles the closures.)
It seems to me that all the groups and renderers are for the most part converging on a similar set of ideas about BSDFs (and for that matter, methods for integration / light transport). I don't think there are any real secrets here, so I'd certainly be in favor of everybody just laying cards on the table and saying what BSDFs they are using and exactly what the parameters are, and maybe there is a common core subset that we all think are worth making the written spec conform to. If there was consensus, I would be happy to scrap the set of BSDFs described in the spec (and testrender) in favor of a modern set that we all feel is better. That section is by far the weakest part of the document.

From an analysis done in collaboration with @anderslanglands, this is what I generally see for raw BSDFs in VRay PRMan, MDL/IRay, and mostly PBRT.  The below is what I consider to be an idealized version that doesn't present all parameter variants that are equivalent:
function LambertianReflection(diffuse) {
  return new OrenNayarReflection( diffuse, 0 )
}
function GGXDistribution( roughness, anisotropyU, anisotropyV ) {
  return new GeneralizedTrowbridgeReitzDistribution( roughness, 2, anisotropyU, anisotropyV );
}

{
  // Diffuse:
  bsdfs.add( new OrenNayarReflection( diffuse, roughnessSigma ) );
}

{
  // Glass:
  var microfacetDistribution = new GeneralizedTrowbridgeReitzDistribution( roughness, gamma, anisotropyU, anisotropyV );
  var fresnel = new FresnelDielectric( 1, ior, abbeNumber );
  bsdfs.add( new MicrofacetReflection( reflectance, microfacetDistribution, fresnel ) )
  bsdfs.add( new MicrofacetTransmission( transmittance, microfacetDistribution, 1, ior ) )
}

{
  // Mirror
  var fresnel = new FresnelNoOp();
  bsdfs.add( new SpecularReflection( reflectance, fresnel ) );
}

{
  // Glossy:
  var microfacetDistribution = new BeckmannDistribution( roughness, anisotropyU, anisotropyV );
  var fresnel = new FresnelDielectric( 1, ior, abbeNumber );
  bsdfs.add( new MicrofacetReflection( reflectanceColor, microfacetDistribution, fresnel ) )
  bsdfs.add( new MicrofacetTransmission( transmittanceColor, microfacetDistribution, 1, ior ) )
}

{
  // Metal
  var microfacetDistribution = new GeneralizedTrowbridgeReitzDistribution( roughness, gamma, anisotropyU, anisotropyV );
  var fresnel = new FresnelConductorNK( 1, ior, kappa );
  bsdfs.add( new MicrofacetReflection( 1, microfacetDistribution, fresnel ) );
}

{
  // Subsurface Uniform (the name TabulatedBSSRDF is taken from PBRT)
  bsdfs.add( new TabulatedBSSRDF_FromUniform( emissionColor, absorptionColor, scatteringColor, phaseFunction ) );
}

{
  // Subsurface Multi-Lobe (similar alSurface, PxrSkin)
  var lobes = [
    new SSSLobe( distance1, weight1, color1 ),
    ...
  ];

  bsdfs.add( new TabulatedBSSRDF_FromMultiLobe( lobes ) );
}

{
  // Hair
  bsdfs.add( new KayijaKay( diffuse, specular, roughness ) );
}

BTW some of the key features of the above definition, mostly as a result of me being inspired by @mmp's PBRT v3, are the following:

The Reflection and Transmission BSDFs are aware of the reflection/transmission colors, thus they can blend to white at Dielectric Fresnel angles if desired or other neat color-specific things.  This is a major limitation of the current standardized OSL closures I am working with -- and the reason for this Github issue.
Fresnel is pluggable, thus allows for different parameterizations for different Fresnel effects.  Not sure if you need full customization here, or whether the three I proposed above are sufficient.
The Distribution for the Microfacet distribution is pluggable, thus allowing different parameterizations for different distributions.

The only thing I am not sure on is whether "abbeNumber" should be part of DielectricFresnel or whether it should always be paired with IOR as part of the Reflection/Transmission BSDFs.

How do you envision pluggable fresnel and microfacet distributions working in the language?
Currently we have used a string enum for the latter and dedicated closures for the former (though as Alex and Larry have said -- the spec is not in sync with our best practices at SPI).

@fpsunflower I do not know how OSL should be modified, I am a novice at OSL (but this is also probably why limitations standout so much as I haven't yet learned to work around them.)
I can point to @mmp's PBRT implementation of the Fresnel class here as sort of a best practice as he implements both Conductor and Dielectric based on it:
https://github.com/mmp/pbrt-v3/blob/master/src/core/reflection.h#L245
It is basically a class that returns a Spectrum/Color given dot( N, I ).

I am not sure truly pluggable fresnels are required, there really are only 3 types I have ever seen, thus enums work.  I am not beholden to the design I gave in terms of pluggable Fresnels.  But the Fresnel options should be supported with color parameters within the Microfacet BRDF for physical correctness.  So an enum for Fresnel along with optional parameters to specify their inputs is sufficient.
Same with the Distributions, an enum with optional parameters to specify their inputs is sufficient.  There are only a fixed number of interesting Distributions that I know of, so full programmability here is not needed.
V-Ray is using the optional distribution parameters, and while not elegant, it is functional.

Those are all very reasonable ways of doing it. I'm sure we could agree on something like what you describe. I'd be happy to put forward what we currently use at SPI which I think covers the cases you are interested in.
Where the design space really explodes is when we start considering layering of these basic closures. That's where I think it might be harder to gain consensus, unless someone really wanted to champion a particular approach.

Layering should be left to the renderer, IMO. As long as the language
allows you to specify that this BSDF goes over this other one, it's up to
the renderer what that means.
The alternative is to try and codify all the different ways this could be
interpreted, which as you say would be an explosion.
On Mon, Oct 19, 2015 at 16:21 Christopher Kulla<EMAIL_ADDRESS>wrote:

Those are all very reasonable ways of doing it. I'm sure we could agree on
something like what you describe. I'd be happy to put forward what we
currently use at SPI which I think covers the cases you are interested in.
Where the design space really explodes is when we start considering
layering of these basic closures. That's where I think it might be harder
to gain consensus, unless someone really wanted to champion a particular
approach.
—
Reply to this email directly or view it on GitHub
https://github.com/imageworks/OpenShadingLanguage/issues/567#issuecomment-149334943
.

The language itself currently doesn't offer a way of saying "over". But you could define something like:
closure color over(closure color top, closure color bot);
It would then be the renderer's problem of deciding how to parse/evaluate/sample a tree of closures that used this primitive.
We'd be very happy if someone were to give this a go inside testrender.

In my analysis I found these types of layering functions -- I :
// Layered
var layers = [
	new Layer( bsdf1, weight1 ),
	new Layer( bsdf2, weight2 ),
	...
	];
bsdfs.add( new WeightedBlend( layers, blendMode = [ additive, normalized, clamped ] ) );

// Scaled
bsdfs.add( new Scaled( bsdf, color ) );

// FresnelBlend
bsdfs.add( new FresnelBlend( ior, weight, overBSDF, baseBSDF ) );

BTW the most interesting thing I've noticed in my studies was the MDL material class, it is basically a template you can plug BSDFs into.  I think it is fairly well though out:
struct material_emission {
	edf emission = edf(); // Light emission DF
	color intensity = color(0.0);
	intensity_mode mode = intensity_radiant_exitance;
};

struct material_surface {
	bsdf scattering = bsdf(); // Bidirectional scattering DF
	material_emission emission = material_emission();
};

struct material_volume {
	vdf scattering = vdf();
	color absorption_coefficient = 0.0;
	color scattering_coefficient = 0.0;
};

struct material_geometry {
	float3 displacement = float3(0.0);
	float cutout_opacity = 1.0;
	float3 normal = state::normal();
};

struct material {
	uniform bool thin_walled = false;
	material_surface surface = material_surface();
	material_surface backface = material_surface();
	uniform color ior = color(1.0);
	material_volume volume = material_volume();
	material_geometry geometry = material_geometry();
};

Weighted and Scaled are natively supported by OSL (since you can multiply closures by colors and add them).
You can emulate FresnelBlend if your renderer provides the vector I and then you can just weight your closures by a fresnel based view dependent term. But of course this will break reciprocity of the resulting BRDF.
There are more "correct" ways of doing the fresnel blend, but they require handling the operation on the renderer side. This points back to either an uber-closure model like we are trying to do at SPI, or a more general closure combination mechanism via modifiers like over().

Old discussion that seems to have died off (and maybe answered all the original questions?). Closing.


All in one updates

PR Description
All in one updates

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Vertex AI SDK authentication problem

Thanks for stopping by to let us know something could be better!
PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Please run down the following list and make sure you've tried the usual "quick fixes":

Search the issues already opened: https://github.com/googleapis/google-cloud-java/issues
Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform

If you are still having issues, please include as much information as possible:
Environment details

Specify the API at the beginning of the title. For example, "[vision]: ...").
General, Core, and Other are also allowed as types
OS type and version: windows 10
Java version: GraalVM 21.0.1
Version(s): vertex 0.1.0

Steps to reproduce

Use example here

It can't authenticate.
I set GOOGLE_APPLICATION_CREDENTIALS env var pointing to the service account json.

Code example
https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/send-multimodal-prompts#gemini-send-multimodal-samples-java
Stack trace
Exception in thread "main" com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:116)
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
	at com.google.api.gax.grpc.ExceptionResponseObserver.onErrorImpl(ExceptionResponseObserver.java:82)
	at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:84)
	at com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onClose(GrpcDirectStreamController.java:148)
	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
	at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:570)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.RuntimeException: Asynchronous task failed
		at com.google.api.gax.rpc.ServerStreamIterator.hasNext(ServerStreamIterator.java:105)
		at com.google.cloud.vertexai.generativeai.preview.ResponseStreamIteratorWithHistory.hasNext(ResponseStreamIteratorWithHistory.java:37)
		at com.google.cloud.vertexai.generativeai.preview.ResponseHandler.aggregateStreamIntoResponse(ResponseHandler.java:104)
		at com.google.cloud.vertexai.generativeai.preview.GenerativeModel.generateContent(GenerativeModel.java:396)
		at com.google.cloud.vertexai.generativeai.preview.GenerativeModel.generateContent(GenerativeModel.java:457)
		at com.google.cloud.vertexai.generativeai.preview.GenerativeModel.generateContent(GenerativeModel.java:408)
		at org.example.MultimodalMultiImage.multimodalMultiImage(MultimodalMultiImage.java:47)
		at org.example.MultimodalMultiImage.main(MultimodalMultiImage.java:24)
Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
	at io.grpc.Status.asRuntimeException(Status.java:537)
	... 17 more

External references such as API reference guides

https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/send-multimodal-prompts#gemini-send-multimodal-samples-java

Any additional information below
Following these steps guarantees the quickest resolution possible.
Thanks!

Are you able to verify that the credentials pointed to by GOOGLE_APPLICATION_CREDENTIALS are actually valid and have permissions to access the API?
cc: @ZhenyiQ

I confirmed. The ADC is working. But the GOOGLE_APPLICATION_CREDENTIALS not. I double check with others as well. Others can reproduce this with GOOGLE_APPLICATION_CREDENTIALS way.

@ZhenyiQ has been able to reproduce this issue.
@blakeli0 PTAL.

I also encountered the same issue, and in the end, I was able to temporarily bypass this issue using Transport.REST. Hope this helps @SetoKaiba

I also encountered the same issue, and in the end, I was able to temporarily bypass this issue using Transport.REST. Hope this helps @SetoKaiba

You can use ADC for authentication. You can still use the sdk.

Thanks @SetoKaiba ! This is indeed a bug in the vertexai sdk. We will push a new version to resolve this. Thanks for bringing this to us!

@blakeli0 @ZhenyiQ @meltsufin Should the defaultScopes be applied to the method public VertexAI(String projectId, String location, GoogleCredentials credentials)  as well?


tmpop: deliver events via abci query

The FireEvent method on Tendermint Client was in fact not meant to be used by the ABCI.
After discussing this with the Tendermint team, an ok solution is to store pending events in tmpop and deliver them through an abci query.
Tendermint will maybe support ways for us to notify connected clients directly in the future, but meanwhile this is a good solution (even though I don't find it very elegant).

This change is

Reviewed 9 of 9 files at r1.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

tmstore/tmstore.go, line 147 at r1 (raw file):

	}

	if len(pendingEvents) > 0 {

do you have to check this?

Comments from Reviewable

Review status: all files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable

Review status: all files reviewed at latest revision, 1 unresolved discussion.

tmstore/tmstore.go, line 147 at r1 (raw file):
Previously, such (Adrien Montfort) wrote…
do you have to check this?

Since I have the range loop beneath I don't, thanks !

Comments from Reviewable

Review status: 8 of 9 files reviewed at latest revision, 1 unresolved discussion.

tmstore/tmstore.go, line 147 at r1 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
Since I have the range loop beneath I don't, thanks !

Done.

Comments from Reviewable

Reviewed 1 of 1 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Codecov Report

Merging #291 into master will decrease coverage by 0.02%.
The diff coverage is 69.23%.

@@            Coverage Diff             @@
##           master     #291      +/-   ##
==========================================
- Coverage   77.43%   77.41%   -0.03%
==========================================
  Files          58       58
  Lines        4037     4042       +5
==========================================
+ Hits         3126     3129       +3
- Misses        695      696       +1
- Partials      216      217       +1

Impacted Files
Coverage Δ

tmstore/tmstore.go
63.88% <69.23%> (-0.14%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f2bbf43...2df3134. Read the comment docs.


Processing data

#Processing data
Checklist before requesting a review

[x] I have performed a self-review of my code
[x] Create model classes for all entities
[x] Set up associations between models.
[x] Create at least one user
[x] Create at least 4 posts written by one of the users
[x] Create at least 6 posts comments for one of the posts

@EricMbouwe Thank you so much for your kind review mate!


Update layout for conclusion section in report_generator.py

🗣 Description
Update layout for conclusion section in report_generator.py

💭 Motivation and context
Ensure that the conclusion section's header will not be separated by a line break from its content.

🧪 Testing
Ran the report multiple times to ensure it would not break apart.

✅ Pre-approval checklist

[ ] This PR has an informative and human-readable title.
[ ] Changes are limited to a single goal - eschew scope creep!
[ ] All future TODOs are captured in issues, which are referenced
in code comments.
[ ] All relevant type-of-change labels have been added.
[ ] I have read the CONTRIBUTING document.
[ ] These code changes follow cisagov code standards.
[ ] All relevant repo and/or project documentation has been updated
to reflect the changes in this PR.
[ ] Tests have been added and/or modified to cover the changes in this PR.
[ ] All new and existing tests pass.

✅ Pre-merge checklist

[ ] Revert dependencies to default branches.
[ ] Finalize version.

✅ Post-merge checklist

[ ] Create a release.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 0.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
0

Relevant Lines:
0

💛  - Coveralls


Task: PIE, when we rename queries, jsObjects , and widgets ensure that bindings are taken care of

We fixed duplicate naming here: https://github.com/appsmithorg/appsmith/issues/30291

We have to ensure that bindings get updated according to renamings.

This should be solved via the recent changes done to support the import block API


Rancher nil pointer

Problem description: If clouds.yaml has no credentials auth will fail with nil.
Tool version: 0.1.2-linux-amd64
OS Version: Windows 10.0.18363 N/A Build 18363

Caused with https://github.com/gophercloud/utils/issues/115


Basic UI for tracing actions

Flowtrace allows retroactive debugging of FBP runtimes. We've specced a capability for starting,stopping,resetting tracing and dumping a trace in the FBP runtime protocol.
It would be nice to have basic UI for these actions in NoFlo UI, to avoid having to use commandline tools to do it.

Will this add "green trace" like https://www.flyde.dev/playground/react-counter/ ?
How to use this?


fix snapshot check

problems with #294

I added a stray " into the regex
the Group match has to handle 3 extra subgroups that the regex change brings in

for 1.1.0-M0+22-4f570ea2-SNAPSHOT, the 3 extra subgroups are -M0, -M, 0
if there is no -M0 (or -RC0), the extra subgroups are all null

the sorting will do for now but it doesn't take into account whether the tag is M or RC and there are other problems
I did play around with sbt.librarymanagement.VersionNumber but it doesn't have a comparator (or scala.Ordering implementation).

I will try this again to see if I can create my own comparator for sbt.librarymanagement.VersionNumber
using this PR to get the build working for now

I have tested these changes on my laptop

I will try this again to see if I can create my own comparator for sbt.librarymanagement.VersionNumber

Having an ordering number for a VersionNumber doesn't make sense in a library like sbt because its completely bespoke, different orgs have the option of making their own release strategies.
In any case I would recommend how we approach this problem in general because there are issues in just using the latest snapshot as is done now, i.e. the builds are not deterministic which can be especially problematic if you are checking out past versions of pekko-http (i.e. trying to disect/figure out a bug) and instead of resolving the pekko version at that time you are instead resolving a completely different modern version.


Update limitations.md

Issue https://github.com/prisma/prisma2/issues/420 has been solved and available in the preview016 release.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks a lot for thinking along @VehpuS 😃🙏


select2:unselect doesn't work properly

(since the previous issue [where I've said in the comments about not working unselect event] has been closed, opening a new issue)
Here is the demo - http://jsfiddle.net/and7ey/p2uyc1ej/1/
Do the following to repeat the problem:

choose the value from dropdown by mouse click;
click on the cross sign with mouse (please note: allowClear is true).
Expected result - select2:unselect should be triggered. But it doesn't happen. Even if you do additional step:
click on the arrow to hide dropdown.

But, if you press Enter at the search field with empty value, then it is triggered, but (!) the chosen before value remains chosen.
Tested on OSX 10.10.1, Google Chrome 39.0.2171.95 (64-bit) and Safari 8.0.2 (10600.2.5).

This was fixed in https://github.com/select2/select2/commit/d81886e5def277df52e8510d97ae9aedadf0cf90, you can expect to see it working in 4.0.0-beta.3.
http://jsfiddle.net/p2uyc1ej/2/

But, if you press Enter at the search field with empty value, then it is triggered, but (!) the chosen before value remains chosen.

Ah, right. Because it is trying to remove the already selected value... even though it can't. This is a bug, reopening this for now.


Fragment example code throws "Warning: validateDOMNesting(...)"

What?
The sample Component code for Table in Fragments/Motivation throws follow warning.
class Table extends React.Component {
  render() {
    return (

          <Columns />

    );
  }
}

Warning: validateDOMNesting(...):
 cannot appear as a child of
. Add a
 to your code to match the DOM tree generated by the browser.
How to reproduce the warning
You can either try it out on CodeSandbox or

Use following code to test
import React from "react";
import ReactDOM from "react-dom";

import "./styles.css";

class Columns extends React.Component {
  render() {
    return (
      <>
        Hello
        World
      </>
    );
  }
}

class Table extends React.Component {
  render() {
    return (

          <Columns />

    );
  }
}

function App() {
  return
;
}

const rootElement = document.getElementById("root");
ReactDOM.render(<App />, rootElement);

How to fix the warning
One needs to wrap
 with
 as shown below.
class Table extends React.Component {
  render() {
    return (

+

            <Columns />

+

    );
  }
}

Hello @dance2die, thank you for reporting and sorry for the delay!
From what I can see the latest of docs has this warning code gone, so I will close this issue as outdated.


Chore (Navbar): migrate to RTL

[ ] Bug fix
[ ] New feature
[x] Chore
[ ] Breaking change
[ ] There is an open issue which this change addresses
[ ] I have read the CONTRIBUTING document.
[ ] My commits follow the Git Commit Guidelines
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.

[ ] I have updated the documentation accordingly.

[ ] My change requires a change to Typescript typings.

[ ] I have updated the typings accordingly.

[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

LGTM.
@davidacevedo, can you look into the prerelease failing?


croc: 9.5.1 -> 9.5.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/schollz/croc/releases.
meta.description for croc is: "Easily and securely send things from one computer to another"
meta.homepage for croc is: "https://github.com/schollz/croc"
meta.changelog for croc is: ""
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
Warning: no invocation of /nix/store/vmxawv69x5zra5ld8gwnxc5rdy7jczrm-croc-9.5.2/bin/croc had a zero exit code or showed the expected version
0 of 1 passed binary check by having a zero exit code.
0 of 1 passed binary check by having the new version present in output.
found 9.5.2 with grep in /nix/store/vmxawv69x5zra5ld8gwnxc5rdy7jczrm-croc-9.5.2
found 9.5.2 in filename of file in /nix/store/vmxawv69x5zra5ld8gwnxc5rdy7jczrm-croc-9.5.2

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
croc

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/vmxawv69x5zra5ld8gwnxc5rdy7jczrm-croc-9.5.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A croc https://github.com/r-ryantm/nixpkgs/archive/10aafd51375ddc451dddf3deb5c4a6bd19de4094.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/vmxawv69x5zra5ld8gwnxc5rdy7jczrm-croc-9.5.2
ls -la /nix/store/vmxawv69x5zra5ld8gwnxc5rdy7jczrm-croc-9.5.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

croc

Maintainer pings
cc @hugoreeves @equirosa @SuperSandro2000 @kalbasit for testing.

@ofborg eval


static Pool

I am implementing a connection pool using bb8. I need this because our connections are async. I need to make a static version of the pool, to do that I am using tokio::sync::OnceCell. The OnceCell requires a static like this:
static ONCE: OnceCell<bb8::Pool<MyDbConnectionManager>> = OnceCell::const_new();

The global accessor fn has an API like this:
pub async fn get_db_conn() -> bb8::PooledConnection<'static, &'static MyDbConnectionManager> {

Because OnceCell requires the &'static ref, I don't know how to get my value back in that type. bb8 has a builder that creates the type as bb8::PooledConnection<MyDbConnectionManager> (this is mandated by the trait), is there any way I can convert that to: bb8::PooledConnection<'static, &'static MyDbConnectionManager>?
If there is a better way to do this let me know, my goal is to have a static connection pool that I can access.

I worked out a way to do this, I need to get the pool first from a OnceCell and then get the connection from that. This keeps the scope smaller. It is some boilerplate but it works fine.


Implemented various improvements

Added a builder instead of child
Separated preview widget from the full screen widget

I like that the builders are separated 👍 Would directly solve my current issue.


chore(main): release 0.40.7

:robot: I have created a release beep boop
0.40.7 (2024-10-30)
Bug Fixes

bump laravel/framework from 11.29.0 to 11.30.0 (#925) (aaed9b4)
bump open-telemetry/gen-otlp-protobuf from 1.2.0 to 1.2.1 in the open-telemetry group (#924) (fd6bb61)
bump sebastian/comparator from 6.1.1 to 6.2.0 (#926) (7bc03ae)
bump zenstruck/assert from 1.5.0 to 1.5.1 (#923) (9d7894a)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

0.40.7
:sunflower:


Core: Implement write API

Summary :memo:
Write an overview about it.
Details
Describe more what you did on changes.

(...)
(...)

Bugfixes :bug: (delete if dind't have any)

Checks

[ ] Closed #798
[ ] Tested Changes
[ ] Stakeholder Approval

Note: append mode will support in a subsequence single PR to implement.


Prototype/Propose MVP Api endpoints required

Given the functionality we require what endpoints do we need, in terms of:

url patterns (roughly - these don't need to be finalised yet, see note)
methods
params
document expected behaviour, pseudo code if necessary.
take pains to future proof.
classify as:

private (needs auth)
public (no auth)
dynamic auth (some response data will need auth)

Of the endpoints, which are MVP which are post-MVP?
What do these patterns tell us about the data model we'll need beneath the api?
Note: this is more concerned with mapping out functionality than exact url patterns, there are some open questions at time of writing (15/1/2022) around patterns for versioning.
Acceptance Criteria

[ ] Document proposal of the above
[ ] Host a design session to discuss with wider team, updating proposal appropriately.
[ ] Feed information into initial requirements for underlying data model/database.
[ ] Create new tasks for sketching BDD per endpoint that enforce the functionality as highlighted by this task (once the BDD tasks are are peer reviewed and in, we then create the appropriate "go build it" tasks).

wrong repo :(


WV-716 Ballot Measures - Allow the ability to delete - Improvement

Modified the POST data to use values from current URL rather than measure data. This allows for preservation of original search filters.

👍  Looks good.


python3Packages.qiskit: 0.26.2 -> 0.32.0

python3Packages.qiskit-terra: 0.17.4 -> 0.18.3
python3Packages.qiskit-aer: 0.8.2 -> 0.9.1
python3Packages.qiskit-aqua: 0.9.1 -> 0.9.5
python3Packages.qiskit-ibmq-provider: 0.13.1 -> 0.18.0
python3Packages.tweedledum: 1.0.0 -> 1.1.1
python3Packages.qiskit-finance: init at 0.2.1
python3Packages.qiskit-optimization: init at 0.2.3
python3Packages.qiskit-machine-learning: init at 0.2.1
python3Packages.qiskit-nature: init at 0.2.2
python3Packages.qiskit: 0.26.2 -> 0.32.0

Motivation for this change
Consolidate several updates to qiskit, and update/init sub-packages.
ZHF: #144627
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

nixpkgs-review is successful on macos 10.15, nixos aarch64 & x86_64. Though of course only limited packages are available on all but the latter.

Result of nixpkgs-review pr 145580 run on x86_64-linux 1

22 packages built:

python38Packages.qiskit
python38Packages.qiskit-aer
python38Packages.qiskit-aqua
python38Packages.qiskit-finance
python38Packages.qiskit-ibmq-provider
python38Packages.qiskit-ignis
python38Packages.qiskit-machine-learning
python38Packages.qiskit-nature
python38Packages.qiskit-optimization
python38Packages.qiskit-terra
python38Packages.tweedledum
python39Packages.qiskit
python39Packages.qiskit-aer
python39Packages.qiskit-aqua
python39Packages.qiskit-finance
python39Packages.qiskit-ibmq-provider
python39Packages.qiskit-ignis
python39Packages.qiskit-machine-learning
python39Packages.qiskit-nature
python39Packages.qiskit-optimization
python39Packages.qiskit-terra
python39Packages.tweedledum

Tested with:
nix shell --expr "((builtins.getFlake "github:drewrisinger/nixpkgs/dr-pr-qiskit-bump").legacyPackages.x86_64-linux.python38.withPackages (ps: with ps; [  qiskit])).override (args: {ignoreCollisions=true;})" --impure --command python -c 'import qiskit; print(qiskit.version.VERSION)'

and python39


COUNT statement

Hi,
I posted another issue #48 where you asked me to split my features in different issues, so this is the first one.
I wonder if you could find a way to easily make a select count statement like below :).

Also please add support for other functions, like SUM() and AVG().


[SUPPORT] Small files (423KB) generated after running delete query

I ran a delete query on my hudi table and found out there are a lot newly generated small files (423KB) (see screenshot). Is there a way to control the size of these small files when running delete query? Basically I am trying to avoid many small files after running delete query.

This is my delete query:
val deleteExistingRecords = spark.read.format("org.apache.hudi").load(basePath + "/*/*").where(col("device_id").startsWith("D"))

deleteExistingRecords.
  write.format("org.apache.hudi").
  option("hoodie.datasource.write.operation", "delete").
  option("hoodie.parquet.max.file.size",<PHONE_NUMBER>).
  option("hoodie.parquet.block.size",<PHONE_NUMBER>).
  option("hoodie.parquet.small.file.limit", 512000000).
  option(TABLE_NAME, tableName).
  option(TABLE_TYPE_OPT_KEY, "COPY_ON_WRITE").
  option(RECORDKEY_FIELD_OPT_KEY, "device_id").
  option(PRECOMBINE_FIELD_OPT_KEY, "device_id").
  option(PARTITIONPATH_FIELD_OPT_KEY, "date_key").
  option(HIVE_SYNC_ENABLED_OPT_KEY, "true").
  option(HIVE_DATABASE_OPT_KEY, "default").
  option(HIVE_TABLE_OPT_KEY, "hudi_fact_device_logs").
  option(HIVE_USER_OPT_KEY, "hadoop").
  option(HIVE_PARTITION_FIELDS_OPT_KEY, "date_key").
  option(HIVE_PARTITION_EXTRACTOR_CLASS_OPT_KEY, classOf[MultiPartKeysValueExtractor].getName).
  mode(Append).
  save(basePath)

@jiegzhan : Delete operations do not create new files. They only create new versions of the same files. My guess is the earlier write operation could have created smaller files due to misconfigurations.  If this is the case, please take a look at https://cwiki.apache.org/confluence/display/HUDI/FAQ#FAQ-HowdoItoavoidcreatingtonsofsmallfiles

cc @nsivabalan

@bvaradar What is the size of new version of the same files after running delete query? For me, they are 423KB.
Step 1: ran bulk_insert query:
df.
  write.format("org.apache.hudi").
  option("hoodie.datasource.write.operation", "bulk_insert").
  option("hoodie.bulkinsert.shuffle.parallelism", 5120).
  option("hoodie.parquet.max.file.size",<PHONE_NUMBER>).
  option("hoodie.parquet.block.size",<PHONE_NUMBER>).
  option("hoodie.parquet.small.file.limit", 512000000).
  option("hoodie.combine.before.insert", "false").
  option("hoodie.combine.before.upsert", "false").
  option(TABLE_NAME, tableName).
  option(TABLE_TYPE_OPT_KEY, "COPY_ON_WRITE").
  option(RECORDKEY_FIELD_OPT_KEY, "device_id").
  option(PRECOMBINE_FIELD_OPT_KEY, "device_id").
  option(PARTITIONPATH_FIELD_OPT_KEY, "date_key").
  option(HIVE_SYNC_ENABLED_OPT_KEY, "true").
  option(HIVE_DATABASE_OPT_KEY, "default").
  option(HIVE_TABLE_OPT_KEY, "hudi_fact_device_logs").
  option(HIVE_USER_OPT_KEY, "hadoop").
  option(HIVE_PARTITION_FIELDS_OPT_KEY, "date_key").
  option(HIVE_PARTITION_EXTRACTOR_CLASS_OPT_KEY, classOf[MultiPartKeysValueExtractor].getName).
  mode(Append).
  save(basePath)

Got 5120 parquet files in S3, they are about 300MB - 700MB each.
Step 2: ran delete query:
val deleteExistingRecords = spark.read.format("org.apache.hudi").load(basePath + "/*/*").where(col("device_id").startsWith("D"))

deleteExistingRecords.
  write.format("org.apache.hudi").
  option("hoodie.datasource.write.operation", "delete").
  option("hoodie.parquet.max.file.size",<PHONE_NUMBER>).
  option("hoodie.parquet.block.size",<PHONE_NUMBER>).
  option("hoodie.parquet.small.file.limit", 512000000).
  option(TABLE_NAME, tableName).
  option(TABLE_TYPE_OPT_KEY, "COPY_ON_WRITE").
  option(RECORDKEY_FIELD_OPT_KEY, "device_id").
  option(PRECOMBINE_FIELD_OPT_KEY, "device_id").
  option(PARTITIONPATH_FIELD_OPT_KEY, "date_key").
  option(HIVE_SYNC_ENABLED_OPT_KEY, "true").
  option(HIVE_DATABASE_OPT_KEY, "default").
  option(HIVE_TABLE_OPT_KEY, "hudi_fact_device_logs").
  option(HIVE_USER_OPT_KEY, "hadoop").
  option(HIVE_PARTITION_FIELDS_OPT_KEY, "date_key").
  option(HIVE_PARTITION_EXTRACTOR_CLASS_OPT_KEY, classOf[MultiPartKeysValueExtractor].getName).
  mode(Append).
  save(basePath)

Got some newly generated small files (423KB) (see screenshot on the top).

@jiegzhan : This can happen if you are deleting all/most of records in your dataset. Even if all the records in a file is deleted, Hudi creates a new version of the file - an empty parquet file in order to ensure queries do not read deleted records accidentally.
The commit metadata should have information about the number of records you deleted.   Let me know if this makes sense.

@bvaradar that makes sense, thanks. After ran many delete queries, I got a lot small files in S3. Is there a way to merge these small files? Basically I am trying to clean up S3 folder after running many GDPR delete requests. Thanks.

@jiegzhan : This is an upcoming feature to re-cluster records in files (https://cwiki.apache.org/confluence/display/HUDI/RFC+-+19+Clustering+data+for+speed+and+query+performance). We are currently working on the 0.6.0 release (Release candidate already out) and will be targeting this feature in  the next major release.

@bvaradar, before re-clustering is available, I tested hoodie.cleaner.commits.retained.
I set option("hoodie.cleaner.commits.retained", 1), then issued a few delete queries. For each parquet file in S3, the latest version and 1 older version (sometimes, not always) got kept in S3, all other versions are gone from S3. Is this how it works?

Yes, this is expected. We retain the penultimate version of the file to prevent a running query from failing. In this case, you might see only one version  of some file which did not see any deletes. That is expected.

Thanks for your explanation, @bvaradar. Closed this ticket.


Add specs for Surrealist::Utils

Coverage increased (+4.5%) to 99.756% when pulling 52beb9fe1b84424e7305e58421afdf0ca670ed7d on specs into ea91b1980158a3f157da908ff5ab4ca2b5dc7ee3 on master.

Coverage increased (+4.5%) to 99.756% when pulling 52beb9fe1b84424e7305e58421afdf0ca670ed7d on specs into ea91b1980158a3f157da908ff5ab4ca2b5dc7ee3 on master.

Coverage increased (+4.5%) to 99.756% when pulling 52beb9fe1b84424e7305e58421afdf0ca670ed7d on specs into ea91b1980158a3f157da908ff5ab4ca2b5dc7ee3 on master.

Coverage increased (+4.5%) to 99.756% when pulling 52beb9fe1b84424e7305e58421afdf0ca670ed7d on specs into ea91b1980158a3f157da908ff5ab4ca2b5dc7ee3 on master.

Coverage increased (+4.5%) to 99.756% when pulling 52beb9fe1b84424e7305e58421afdf0ca670ed7d on specs into ea91b1980158a3f157da908ff5ab4ca2b5dc7ee3 on master.


Function Pointer not implements in C# 9?

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6240d621-9b08-a9b9-ceed-5e4fd76a73a0
Version Independent ID: 336fb56d-9797-e1e2-b623-a31a804cfecd
Content: Function pointers - C# 9.0 specification proposals
Content Source: proposals/csharp-9.0/function-pointers.md
Product: dotnet-csharp
Technology: csharp-spec
GitHub Login: @dotnet-bot
Microsoft Alias: wiwagn

@GF-Huang I just verified that function pointers are implemented in C# 9.0.
You'll need to make sure you enable unsafe code (/unsafe) and that you are targeting .net5.0. Then, it should work.
Let us know if that addresses your concern. If not, can you add more information on what's not working for you?

@GF-Huang I just verified that function pointers are implemented in C# 9.0.
You'll need to make sure you enable unsafe code (/unsafe) and that you are targeting .net5.0. Then, it should work.
Let us know if that addresses your concern. If not, can you add more information on what's not working for you?


Link custom user forms with campaign creation flow

closing as not needed anymore - parent ticket was solved long ago


feat: include reqwest::Error in RecaptchaError

This will allow users of this crate to match on the underlying reqwest error and probe it for more information via its methods.

Looks great! Might be introducing breaking changes, but I will describe the change in the README and let people modify their client code appropriately.


Fix generate-manifest.sh for release mode

The cp command was failing because patches/release/ has no YAML files at
the moment. I ran into this issue as I was generating the YAML manifest
for v0.2.0.

/skip-all


Bitrange symbols are not handle

This is known issue.

I'm currently doing a small rework to address this and #30

Great, will wait for that before trying to search and fix new bugs.


Names and glossary of terms

Prompted by #377 and naming discussions, let's define the terms we'll be using for Cloudstate going forward.
Discussions have led to renaming stateless function -> action. And using workflow for coordinating actions.

Could we use a Github WikiPage for this?

Naming Failures
I link this contributors' call outcome where it was discussed how to handle failures and especially for this issue how to call failures. We might still find consistent names for them here.

I am working on trying to understand the naming right now. Here are the names that I am currently using.

Proxy - This is the sidecar proxy that is deployed alongside Cloudstate. It consists of the following components:

Actors - Manage the flow of Commands/Events to/from the Entities defined in the Cloudstate Service.
Event Store - Essentially the database, but avoiding the use of that term because the capabilities of the database are intentionally limited.

Service - This is the deployable unit that the user creates in their language of choice. It consists of the following components:

Entities - These map 1-1 to the Actors that live in the Proxy. The user code is executed inside the Entity.

Client - This is the external system that calls into the Cloudstate system.
Commands - These are the messages that flow from the Client into the Actor in the  Proxy, and then get passed through to the Service to be processed by the Entity.
Events - These are the messages that are emitted by the Entity as a result of processing a Command. They are persisted in the Event Store by the Actor associated with that Entity.

This is based on my incomplete understanding of Cloudstate, so feel free to correct any errors.

Hi @WadeWaldron, thanks for the contributions. I just have a few points to mention.
First, I think that when referring to Actors we make it clear that we are only talking about the Akka implementation of Cloudstate and not more generally the Cloudstate protocol, which can be implemented on top of other technologies in the future. I don't know if this is what we expect for the glossary of terms in this context. It might be worth having it separately, or making it clear that we are only referring to the Akka implementation.
Another small point is that at least I found an ambiguous definition when saying "the Entities defined in the Cloudstate Service" at this point I do not know if I am talking about the gRPC Service defined via Protobuf, the entity registration in Discover (via support language), or to the LBCS. I think that specific sentence could be more elaborated.
What do you think?

@sleipnir Thanks for the feedback. I actually wasn't aware that Cloudstate could be implemented on top of other technologies. That's very interesting. Does the protocol define something that acts as a more generic term than "Actor"? I.E. Does the Actor fill a particular role in the Cloudstate protocol?
I will admit the Service term is definitely overloaded. I just don't know what else to call it. If there's a better term, I would be happy to use it. I am using the term Service to refer to the deployed unit written in the user's language of choice.

@WadeWaldron Yes, indeed. Cloudstate is both a protocol and a reference implementation and in that sense I think it is always good to define the scope of each nomenclature well.
@pvlugter  / @marcellanz , about the nomenclature of the service, I think we have discussed something similar before somewhere? Can you suggest something to help here?

I think this message https://groups.google.com/g/cloudstate/c/hALcsXiCpKk/m/XR-tl47cAQAJ started a thread where we iterated over service/entity/controller and, after this longer thread
https://github.com/cloudstateio/cloudstate/pull/377
we went with action instead of naming controllers controllers and this last message by @pvlugter  https://github.com/cloudstateio/cloudstate/pull/377#issuecomment-662158293 sums ups the current understanding I think. There is still some wording left to be defined as Peter asked in this last post.

Thanks @marcellanz

Interestingly, I found right from the beginning looking solely from the spec, one has not to mention its underlying technology stack to understand. I found the spec itself enough to implement the user language support (at least for event sourcing) and I think when claiming by the project it rightfully should.
Also, the spec is a Draft PR since nearly a year, and not much fundamentally has changed. There are some questions to be clarified and I think once things settled, it will get more attention.
Over time, and if one gets to know Akka-(Cluster) I think, it seems clear that the reference implementation is based on it and also ideas from it are capture in the spec, but to understand or learn Cloudstate I think this is a detail not to be known for the user. I'm also looking forward to another implementation not based on Akka I think.

I absolutely agree


Unable to execute inference.py

Executing the command mentioned in README.
python3 inference.py --model=mobilenet --imgpath=<path-to-image>
Gives this error in the end.
Failed to get matching files on ./models\trained/mobilenet_368x368/model-release: Not found
And there's nothing about Trained folder in README.md

I get this error too.

Me too :-(

Download the mobilenet 368x368 weights/checkpoint files (as linked in the readme)
Adapt the paths in networks.py so that the downloaded files get restored (loader.restore should be called with something like '[PATH_TO_DOWNLOADED_CHECKPOINT_FOLDER]/model-388003' without any file extensions)

when i downloaded the models and modify the code, it still have errors;
2017-11-15 10:01:55.643782: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:193] kernel reported version is: 375.66.0
2017-11-15 10:01:55.643788: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:300] kernel version seems to match DSO: 375.66.0
INFO:tensorflow:Restoring parameters from ./models/mobilenet_368x368/model-388003
2017-11-15 10:01:57,600 INFO Restoring parameters from ./models/mobilenet_368x368/model-388003
2017-11-15 10:01:57.856058: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.860577: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L2_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.863187: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L2_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.864063: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L2_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.865679: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L1_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.867032: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L1_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.868632: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L1_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.869931: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L1_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.871450: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L1_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.873282: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L2_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.874794: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.878593: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L2_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.879861: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L2_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.882751: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage6_L2_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.885738: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_0/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.887116: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_10_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.888944: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L2_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.889782: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_11_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.892116: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.893473: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.895517: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.898718: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.901610: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.902991: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_6_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.904798: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_7_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.906265: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L1_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.908498: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_8_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.909199: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key MobilenetV1/Conv2d_9_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.915773: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L1_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.919393: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L1_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.920945: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L1_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.921516: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L1_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.922258: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L1_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.923465: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L2_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.925081: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L1_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.925573: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.926912: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L2_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.927975: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L2_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.930758: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage1_L2_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.933026: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L1_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.935328: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L1_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.935596: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L1_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.937661: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L1_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.940021: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L1_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.942464: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L1_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.943362: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L2_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.947879: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.952309: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L2_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.956860: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L1_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.958723: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L2_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.960879: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage2_L2_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.962403: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L1_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.964658: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L1_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.966254: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L1_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.966800: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage5_L1_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.966993: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L1_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.969448: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L1_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.970581: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L2_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.973933: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.973958: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L2_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.975185: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L2_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.975773: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L2_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.977009: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage3_L2_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.979628: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L1_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.982459: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L1_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.984104: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L1_3_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.985895: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L1_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.987194: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L1_5_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.992043: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L2_1_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.992582: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L2_4_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.994649: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
2017-11-15 10:01:57.995790: W tensorflow/core/framework/op_kernel.cc:1158] Not found: Key Openpose/MConv_Stage4_L2_3_pointwise/BatchNorm/gamma not found in checkpoint
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 1139, in _do_call
return fn(*args)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 1121, in _run_fn
status, run_metadata)
File "/usr/lib/python3.4/contextlib.py", line 66, in exit
next(self.gen)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.NotFoundError: Key Openpose/MConv_Stage5_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
[[Node: save/RestoreV2_349 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_349/tensor_names, save/RestoreV2_349/shape_and_slices)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "inference.py", line 35, in
net, _, last_layer = get_network(args.model, input_node, sess)
File "/dev/tf-openpose/networks.py", line 46, in get_network
loader.restore(sess_for_load, os.path.join(_get_base_path(), ckpts[type]))
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/training/saver.py", line 1548, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key Openpose/MConv_Stage5_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
[[Node: save/RestoreV2_349 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_349/tensor_names, save/RestoreV2_349/shape_and_slices)]]
Caused by op 'save/RestoreV2_349', defined at:
File "inference.py", line 35, in
net, _, last_layer = get_network(args.model, input_node, sess)
File "/dev/tf-openpose/networks.py", line 45, in get_network
loader = tf.train.Saver()
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/training/saver.py", line 1139, in init
self.build()
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/training/saver.py", line 1170, in build
restore_sequentially=self._restore_sequentially)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/training/saver.py", line 691, in build
restore_sequentially, reshape)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/training/saver.py", line 407, in _AddRestoreOps
tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/training/saver.py", line 247, in restore_op
[spec.tensor.dtype])[0])
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 640, in restore_v2
dtypes=dtypes, name=name)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 1269, in init
self._traceback = _extract_stack()
NotFoundError (see above for traceback): Key Openpose/MConv_Stage5_L2_2_pointwise/BatchNorm/gamma not found in checkpoint
[[Node: save/RestoreV2_349 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_349/tensor_names, save/RestoreV2_349/shape_and_slices)]]

Yes it is the problem related to the model file. you should download and move the file correctly.

Hi,
I have followed the steps, but when I exec
$ python3 inference.py --model=mobilenet --imgpath=...
I get the next error:
DataLossError (see above for traceback): Unable to open table file /home/Downloads/mobilenet/model-388003: Failed precondition: /home/Downloads/mobilenet/model-388003: perhaps your file is in a different file format and you need to use a different restore operator?
How can I fix this error?
Thank you!

Any updates on the issue ?

same problem ...

I have followed the steps,  and modify the run_checkpoint.py file
input_node = tf.placeholder(tf.float32, shape=(None, None, None, 3), name='image')
to
input_node = tf.placeholder(tf.float32, shape=(None, 368, 432, 3), name='image')
then run:
$ python3 run_checkpoint.py --model=mobilenet
and I got en error:
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1,1,128,19] rhs shape= [1,1,64,19]
[[Node: save/Assign_293 = Assign[T=DT_FLOAT, _class=["loc:@Openpose/MConv_Stage5_L2_5_pointwise/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](Openpose/MConv_Stage5_L2_5_pointwise/weights, save/RestoreV2/_587)]]
[[Node: save/RestoreV2/_32 = _SendT=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_38_save/RestoreV2", _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
how to fix this error?


Task: Manage LoRa Concentrator on the portal

Is your feature request related to a problem? Please describe.
A lora concentrator device type should be added in the solution.
Describe the solution you'd like
Add screens to manage LoRa Concentrators on the portal.
Distinguish LoRa Concentrator and other devices, by a specific tag on the concentrator twin.
Describe alternatives you've considered
Manage it manually.
Additional context
N/A

Lora concentrator screen should be present in a separate menu.

The default Concentrator router configuration could be foudn at: https://github.com/Azure/iotedge-lorawan-starterkit/tree/dev/Tools/Cli-LoRa-Device-Provisioning/DefaultRouterConfig

Please add a new Menu item on the portal:

LoRa Device Management

Basic Stations (aka: Concentrators)

Fields usefull for concentrators:

Device Id (deviceId) Required, Readonly
Device friendly name (tag) Required
LoRa Region (Select item), Required -> properties.desired.routerConfig (downloaded from https://github.com/Azure/iotedge-lorawan-starterkit/tree/dev/Tools/Cli-LoRa-Device-Provisioning/DefaultRouterConfig

EU863,
US902,
CN470RP1,
CN470RP2

client certificate thumbprint (Added by user, String) -> properties.desired.clientThumbprint

Other fields from device are not used and should be removed.
Concentrator is not based on a model.
Region value might be stored in the device tags.


Update documentation URL

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thank you for this catch @t3hcr


add d3 tree dataviz to "examples" on ipfs.io

would be good to have it on the readme of this repo, too

I'm wanting to circle back to this repo and do this and some other issues.  If anyone is interested in helping out that would be great too.  Maybe @Dignifiedquire?
The viz is very cool and works well at a basic level... Screenshot in #1
Thanks for thinking of this project yesterday @jbenet, I was too!

I'm always a big fan of visualizations. Though I'm also sceptical about d3 performance as for example the globe in the webui kicks in all fans on my macbook.
In terms of help, what exactly needs to be done, the title of this issues seems to suggest that the visualization is already finished?

Great! Hosting Thanksgiving, will reply soon : )

My "reply" was #19, in case that wasn't clear 😉


Pin three js package to 0.136.0 same as Simulator and fix running demos and fix CI build

This change pin three JavaScript library to the same version which Simulator project is using.
Build of SimulatorCore is failing with version 0.144.0 but not with 0.136.0
It seems the failure start to happen since https://github.com/FRUK-Simulator/SimulatorCore/pull/188 was merged.
This also fixes the ability to run demos in SimulatorCore.

Bump the version in packge.json?
Yes, Netlify will need fix but I'm not sure I can fix it without more permissions for Netlify.


Redeploy template after failed attempt

In the GUI, when a template fails this is shown in the log for the reason mentioned. It would be great to have the option here to be able to get filled in template back and start solving the issue instead of refilling each time after a failure.
Used versions
FAST version: 1.3.0
TMOS version: <IP_ADDRESS>

Hi @gwolfis , thanks for your feature request.
It turns out that this feature is being worked on right now. There will be a button in the deploy log to re-load the template with the previously filled form values.
It will be available in the next release of FAST, which is currently planned for October 13th. This date could change, but shouldn't drift far.
Thanks,
Matt

This feature is available as of FAST 1.4.0. A new button has been added to the Deploy Logs tab to "Modify and Resubmit Application". This button is available for failed and for successful deploy actions.


Plugins management

As discussed in https://github.com/progrium/dokku/issues/1205, I tried a way to manage plugins and be able to do a dokku plugins-install only on core ones.
This PR is here to discuss the features and the details of the implementation.
It’s pretty basic for now but it works as intended :)
So we have three subdirectories in /var/lib/dokku/plugins, core, custom and enabled. Plugins are now divided in two categories: core ones and other ones.
The enabled directory only links to plugins (either in core or enabled directories). The core plugins are enabled automatically when installing dokku and cannot be disabled.
There are two new commands:

dokku plugins:enable <name> to enable a third party plugin.
dokku plugins:disable <name> to disable a third party plugin. Core plugins cannot be disabled by this command.

plugins-install and plugins-install-dependencies now take an optional argument --core which when set will instruct those commands to only refer to the core plugins. This should resolve the issue #1205.
This happens by setting PLUGIN_PATH to /var/lib/dokku/plugins/core instead of the default /var/lib/dokku/plugins/enabled when --core is set.
/cc @josegonzalez @michaelshobbs @jimeh

I did not look at the tests at all since I don’t know where we’re going exactly :grin:

Looks like a good start to my dokku-n00b eyes :)
And not that you've gotten there yet, but just so it's mentioned here, the upgrade process would obviously need to delete existing core plugins in the old plugins path, and probably move any non-core plugins into the new custom path.

@michaelshobbs how much of this would be affected by the plugn pr?

There is significant overlap. One of you guys should really take a look at plugn and that branch before going too far down this path.  I’m pretty sure (but I didn’t check) that we could replicate what I do here with plugn instead of pluginhook. Plugn seems to take an env variable for its plugins directory so the mechanism is pretty similar to the one used here.
Also the goal of this PR isn’t the same than yours. Here we try to give a solution (that seems pretty clean to me) to the problem in issue #1205. Another important point I think is that plugin compatibility is not affected at all by this PR.

Just found a bug: source "$(dirname "$0")/../common/functions" won’t work with symlinks :confused:
Anyone knows why source "$PLUGIN_PATH/common/functions" isn’t used instead?

Really? Shouldn't source follow symlinks? Or does dirname  $0 return the real path rather than the symlinked one?

well it won’t work with ls too. The thing that works is when you move around with cd because your shell keeps an history from where you’ve “entered” the symlinked directory. And since here there is no context, .. refers to the “real” parent directory.

So we’ll need to use $PLUGIN_PATH instead in all plugins (but compatibility will be broken :confused:) or we’ll have to drop symlinks and copy the plugins between directories. Other ideas?

Ah, yeah you're right. The .. does indeed screw it up. Sorry, seemed weird while I was guessing off the top of my head.
Seems symlinks for custom plugins isn't going to work then. But I just did some basic testing and seems custom plugins will work if they simply live in the folder relative to the common symlink, rather than being symlinked in themselves.
I tested a folder structure like this locally with some mock plugins:
/var
 └── lib
     └── dokku
         ├── core-plugins
         |   ├── common
         |   ├── checks
         |   └── domains
         ├── disabled-plugins
         |   └── mongodb
         └── plugins
             ├── common -> /var/lib/dokku/core-plugins/common
             ├── checks -> /var/lib/dokku/core-plugins/checks
             ├── domains -> /var/lib/dokku/core-plugins/domains
             ├── forego
             └── redis

It's mostly just the enable and disable commands that need to simply move the custom plugin folder back and forth, while refusing to operate on symlinks. It might not be the absolutely cleanest solution, but probably the most decent one to avoid breaking backwards compatibility.
And on a side-note, the example I have above would mean even less backwards compatibility issues, as even installation instructions in existing plugins would still be correct when they instruct you to copy/clone/extract the plugin to /var/lib/dokku/plugins.
As for why dirname "$0" is used, seems silly, but I guess someone started the convention and others followed... lol

Just convention. If you guys want to make a separate pr that uses $PLUGIN_PATH, that would work.

@jimeh yes I think your solution will work and if we want to keep plugin compatibility we don’t really have a choice.
@josegonzalez we could be then we’ll have to update (almost) all community plugins too :grin: I’m not sure it’s worth the effort.

I think it's worth to change core plugins to use $PLUGIN_PATH in an effort to start setting a new and better convention :)... I'd be happy to submit such a PR.

Ok since you seem to be motivated, I’ll wait for you PR. Meanwhile, I’ll patch my plugins and forks :wink:

PR is in, but I don't think we can rely on or assume all 3rd party plugins will be updated any time soon. So plugin management is still probably best done via moving them back and forth. But at least new plugins will hopefully follow the new convention and make life easier for everyone in the future.

Yes but it’s sad.
We can update and submit PR to a lot of them. There’s also a lot of plugins which aren’t active/maintained anymore :confused:
Since we’ll need an upgrade script with the new plugins management, we could check for source $(dirname $0)/../common/functions in installed ones and print a warning (and not enable them).

or better yet, replace automatically the line with the new one but still print a warning message to update plugin (or report the error to the author).

That's one solution, although when/how would that check be executed? I'm assuming as part of plugins:enable right?
And come to think of it, the enable/disable thing is nice, but if we use the move solution the benefit is that there's ZERO difference in how things work now. Plugins are enabled by default, they don't run as part of dokku installation, and you can disable plugins via a simple command instead of manually deleting or moving the directory on the server.
The symlink solution feels nicer to me from a architectural point of view, but does alter how the end user installs plugins. An extra step isn't much, but it's still different.
Don't know which one you prefer @Flink, nor am I fully sure which one I prefer, just wanted to point things out for clarity's sake :)
P.S. As a nod towards moving plugins around, OSX actually handles user-installed fonts that way. User fonts live in ~/Library/Fonts, and if you disable one, it's moved to ~/Library/Fonts (Disabled).

Well I think I’m like you on this one, the symlink solution seems “better” but what you say is also very true. Moving plugins instead of symlinking them and changing the directory structure will be better from a user point of view.

Ok I think this PR is done now and should work as expected. I used the approach suggested by @jimeh with directories and mv instead of symlinks. So we don’t need an upgrade script or something like that, it will work out of the box with an existing installation. You just have to upgrade dokku normally :smile:
/cc @josegonzalez @michaelshobbs

If we're good with this I'd like to merge it into the plugn branch and merge them both into master via that PR; unless we want this in before 0.4.0. Though I didn't think we were releasing another 0.3.x.

Well it fixes the issue with debian installation so it can be merged in a 0.3.x branch. But I personally don’t have a preference :)

I don't mind either :)... Personally I worked around the installation issue by having the chef cookbook disable/enable plugins itself before installation/upgrade, so I don't mind waiting till 0.4.x for a fix.

We are not doing another 0.3.x release.

Ok so it settled ;) @michaelshobbs Why not merge this in master and then rebase your PR?

We're not rebasing plugn anymore.

@Flink 6 of one have a dozen of the other. Hence me saying I'd like to do it this way. Just a preference.

Ok I was just asking :sweat_smile: If it’s easier for you like that then there’s no problem (I don’t really mind :grin:)

Yeah no worries at all. It's probably all the same but my thinking was just that if we're going to overhaul plugins then let's just do it in one go.

If you want to merge this into the plugn branch first, be my guest :) . Just close and rebase it onto that branch.

Okay I'm going to close this PR. Please redo the PR and target the mh-switch-to-plugn branch. We can merge it into there and then see how we like/dislike the experience :)


email template is not centered in Apple Mail app

MacOS Ventura 13.1

Fixed by adding these styles:
@media all {
    body {
        margin: 0;
        padding: 0;
    }
}

Interesting, thanks for sharing the solution. I wonder why it requires @media and why this isn't picked up.

I'm not able to recreate this. I tested using Postdrop and all looks fine.
Confirming you used the inline version?

Copy-pasted inline version that you've mentioned, sending a screen. Haven't tried Postdrop, sent email by copy-pasting html in Thunderbird


Add the option to play the animation or just return the child in the Animate class

Hi, not much changes, just a new property called isAnimationEnabled
tell me if I should rename it to fit your needs
sometimes when I don't want to play the animation for some reason or disable them all instead of using conditions or create helper class, function to do that I just add that built-in in the Animate class
Example
After:

Animate(
            effects: const [
              SlideEffect(),
            ],
            enabled: false,
            child: TextButton(
              onPressed: () async {
                print(jsonEncode(_controller.document.toDelta().toJson()));
              },
              child: const Icon(Icons.help_outline),
            ),
          ),

Before:

    Builder(
            builder: (context) {
              final child = TextButton(
                onPressed: () async {
                  print(jsonEncode(_controller.document.toDelta().toJson()));
                },
                child: const Icon(Icons.help_outline),
              );
              if (isAnimationEnabled) {
                return Animate(
                  effects: const [
                    SlideEffect(),
                  ],
                  child: child,
                );
              }
              return child;
            },
          )

That just an example

Thanks for the PR!
I'll give this a bit of thought. I can definitely see the point of it, but it's also a bit against normal Flutter conventions — normally you'd just rebuild omitting the Animate entirely.
Other feedback:

I'd likely also just go with enabled, I think there's sufficient context.
don't forget to add it to the .animate() extension
I try to make sure that animations / effects don't alter the widget tree as much as possible. In this case I think we should break this rule, but it might be worth calling that out in the docs, since reparenting widgets can cause them to lose state unless you take care to avoid it.

Thanks for the PR!
I'll give this a bit of thought. I can definitely see the point of it, but it's also a bit against normal Flutter conventions — normally you'd just rebuild omitting the Animate entirely.
Other feedback:

I'd likely also just go with enabled, I think there's sufficient context.
don't forget to add it to the .animate() extension
I try to make sure that animations / effects don't alter the widget tree as much as possible. In this case I think we should break this rule, but it might be worth calling that out in the docs, since reparenting widgets can cause them to lose state unless you take care to avoid it.

Hi, thank you for the reply. I will make the request changes as soon as I can.

I try to make sure that animations / effects don't alter the widget tree as much as possible. In this case I think we should break this rule, but it might be worth calling that out in the docs, since reparenting widgets can cause them to lose state unless you take care to avoid it.

The rebuilt in case the animation state set to enabled in the app settings so we don't want the animation effects to be working

Thanks for the PR!
I'll give this a bit of thought. I can definitely see the point of it, but it's also a bit against normal Flutter conventions — normally you'd just rebuild omitting the Animate entirely.
Other feedback:

I'd likely also just go with enabled, I think there's sufficient context.
don't forget to add it to the .animate() extension
I try to make sure that animations / effects don't alter the widget tree as much as possible. In this case I think we should break this rule, but it might be worth calling that out in the docs, since reparenting widgets can cause them to lose state unless you take care to avoid it.

If there are any actions required from my side, feel free to tell me when you can

So, I merged this in and tweaked it a bit, but in the process I developed some hesitation and backed it out for now.
Concerns:

It really isn't enabling/disabling (the animation logic still runs), just "hiding"
Widgets like ImageFiltered use enabled specifically to mean the effect will be disabled, but the widget tree will be maintained.

Ultimately, this feels like it's just a convenience method for a ternary operator in your build function.
animationEnabled ? myWidget : myWidget.animate().etc()

Open to more discussion on this.


Multiplexing support

In Beaker's implementation of PeerSocket, we need each app to operate separately from each other. They need to join/leave swarm topics separately, and use the sockets separately.
In practice, I think this means

Beaker needs to reference count the lobby joins and exit swarm topics when all tabs have left a lobby.
The messaging protocol needs multiplexing support so that Beaker can parcel out the sub-sockets to apps.

The wire protocol that Beaker is currently constructing for PeerSockets is two layers.

Messaging framing with length-prefixes
Protobuf schemas for individual message types (currently PeerSocketMessage and PeerSocketSessionData)

I think we could handle multiplexing by adding channels to the protobuf schemas, in which case this doesn't need to be implemented in Hyperswarm itself.

we multiplex


Add "Notes" section

Allow for users to write more detail about content in the group

This is handled by #98