Feature request - Add endpoint option to Builder.

Currently the Builder's initializer accepts a host param which assumes https. I like to use minio for local development and it isn't ideal to run it with https.
The normal Aws::S3::Client gem accepts an :endpoint option to by-pass this issue.
Here is a section of the docs that explains this option:
:endpoint (String, URI::HTTPS, URI::HTTP) — Normally you should not configure the :endpoint option directly. This is normally constructed from the :region option. Configuring :endpoint is normally reserved for connecting to test or custom endpoints. The endpoint should be a URI formatted like:
'http://example.com'
'https://example.com'
'http://example.com:123'

https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html

Makes sense, are you up to sending a PR with specs to fix?  Thank you if so!

I hope to send a PR soon.

Thank you!!

I need to stop for today, but I thought I would leave a comment with my progress just in case you wanted to give me any feedback.
I cloned the repo and made this commit on my clone:
https://github.com/BenKanouse/faster_s3_url/commit/f76883c84206c52455590381c4ad5dbcf5ec8209
The commit message summarizes the progress:

This is working for my use case but the logic is a bit confusing to
support both endpoint and host options, since we need to fetch the
host from the endpoint when the endpoint option is supplied. So
there are 3 ways to get a host now.

Supply a host
Use the default Host
Get the host from endpoint

The other thing that is confusing is that sometimes S3 seems to put the
bucket in the host name and sometimes it is the first item in the path.
I have it working for my use case, but it might not be working for other
edge cases.

Thanks!
I won't have time to look at this until next week! Do you want to just make a PR here with your work so far? That'd be a convenient way for me to look at it! You can mark it draft or whatever.

The other thing that is confusing is that sometimes S3 seems to put the
bucket in the host name and sometimes it is the first item in the path.

Yes, I seem to remember writing some logic (copied from AWS SDK) for determining which was which, that's in there somewhere?
This sounds like it should be do-able!

since we need to fetch the host from the endpoint when the endpoint option is supplied.

Do you mean you need to make an API request to find the host, or just like parse it from the endpoint?  API request to get it is a bit annoying, performance wise, and needing network access on boot or whatever!

I created a pull request https://github.com/jrochkind/faster_s3_url/pull/10
I did find a few interesting things to point out.

I made the judgement call of trying to abstract a class from the Builder class. I put it in a separate commit so it can be removed easily if you don't like it.
Amazon's logic for formatting the url is pretty complicated and I suspect this gem doesn't want the complexity of supporting every use case: https://github.com/aws/aws-sdk-ruby/blob/version-3/gems/aws-sdk-s3/lib/aws-sdk-s3/endpoint_provider.rb#L30
I don't think the current host param works as intended and I wonder if it would be better to remove the host option in favor of this endpoint option. I put more details into that issue here: https://github.com/jrochkind/faster_s3_url/issues/11

Thanks for all the feedback! I really enjoy the gem!


Response from commands

The document shows how to create a command for invoking a direct method on the device, https://docs.microsoft.com/en-us/azure/iot-central/tutorial-define-device-type#use-commands
But, there is missing a description about the response payload from the device direct method either its showing on the dashboard or mapping to the iotc app properties.
Does this feature is supported in the current Azure IoT Central?
Thanks
Roman

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f802444e-a296-ca21-aa86-1ead1e6f3716
Version Independent ID: 06a9fe60-1da7-5327-eb33-e066e58e8509
Content: Define a new device type in Azure IoT Central
Content Source: articles/iot-central/tutorial-define-device-type.md
Service: iot-central
GitHub Login: @dominicbetts
Microsoft Alias: dobett

@romankiss Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

Hi @romankiss - the IoT Central UI shows any payload returned from the device when a command is invoked. For more information about direct methods (this is the mechanism used by IoT Central commands), see https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-direct-methods.
Would it be useful to include a screenshot in this article showing how the UI displays return values from a command?

Thanks @dominicbetts  for your prompt reply.
I have not a problem receiving a response from the device direct method invoking by HTTPs and/or  Device Explorer tool. Everything is working well including my device which is using a direct MQTT protocol.
When my device is connected to the Azure IoT Central App, I can successfully invoked my device direct method and sent a simple or empty payload back to the invoker.
The topic for response is the following:
$iothub/methods/res/200/?$rid=1
the body is a JSON formatted text.
Note, that the Azure IoT Central App accepted my response, but didn't show up it. Interesting is, when I have published an error, for example:
$iothub/methods/res/500/?$rid=1
still, the AIoTC App also accepted this response like OK, no error message.
Also, I have added to the topic a content-type and content-encoding properties:
$iothub/methods/res/200/?$rid=1&$.ct=application%2Fjson&$.ce=utf-8
but the problem stayed the same, no displaying a response payload on the command dashboard.
Note, that using the HTTP postman and/or Device Explorer tool everything is working well.
It looks like, there is probably a bug during the deserializing a response message, because the status property also doesn't working.
Please, could you verify if the response from the device direct method is working with the Azure IoT Central, including a status property?
I have tried to help with this issue here.

Hi @romankiss - please see my response on Stackoverflow here.
I'll create a workitem to update the documentation to include this information.
Thanks for getting in touch,
Dominic

Hi Dominic,
Thanks for your example on the Stackoverflow.
Based on this example, it looks like the IoT Central App doesn't handle a response and status from the device direct method.
I think, this is a design bug, I have expected mapping the response from the device direct method to the IoTC application properties and showing it in the command dashboard.
Using a "hardcoded" reported property with the name of the direct method (command) and updating this reported property by the device direct method code for showing as a result is "breaking" a request/response message exchange pattern between the invoker and device direct method to the fire&forget pattern.
Thanks
Roman

Hi Roman,
I'll make sure this information is passed on to the product team. In the meantime, I'm going to close this issue, but please feel free to reopen it if you have any further questions.
Thanks,
Dominic

#please-close


Bug with updating store in IE11

When I use IE 11, and try receive data from server, store is not updating, it updating only first time.
try use polyfill for reducer, but it not helps.
const objectAssign = require('object-assign');

const BackupReducer = function(state = initialUserState, action) {
   //console.log('actiondata in reducer:' + action.data + action.type);

  switch(action.type) {

  case 'GET_BACK_LIST':
		 return objectAssign({}, state, { backups: action.data });
 break;
}}

@zolcman For usage question, please ask on Stack Overflow.

sorry for truble, was problem with axios, it cache data. need use Pragma: 'no-cache'


Typo "**B. ～button **"→"B. ～button"

https://docs.microsoft.com/en-us/azure/devops/repos/tfvc/view-branch-hierarchy-team-project?view=azure-devops

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Build issue with testing

I randomly get a failure when testing the Hello program as part of the build process. The output of launching build.cmd when it fails is:
Time elapsed 00:00:02.5220348
Input Assembly: C:\work\Microsoft\corert\tests\src\Simple\Hello\bin\Debug\dnxcore50\Hello.dll
Unable to remove directory: C:\work\Microsoft\corert\tests\src\Simple\Hello\bin\Debug\dnxcore50\win7-x64\native
Access to the path 'Hello.exe' is denied.
cl : Command line warning D9025 : overriding '/MT' with '/MTd'
Hello.cpp
LINK : fatal error LNK1104: cannot open file 'C:\work\Microsoft\corert\tests\src\Simple\Hello\bin\Debug\dnxcore50\win7-x64\native\Hello.exe'
Linking of intermediate files failed.

I got it with Add1 too

Found out that my anti-virus software is the culprit. I can mitigate the issue by re-issuing the bin removal directory command in runtest.cmd:
    if exist "!__SourceFolder!\bin" rmdir /s /q !__SourceFolder!\bin
    REM Some antivirus program are keeping a handle on the .exe, a second attempt usually works.
    if exist "!__SourceFolder!\bin" rmdir /s /q !__SourceFolder!\bin

Is this something we can add in runtest.cmd?

We don't really have a guarantee the second attempt will work. We would at least need to put a delay between them. It depends on how busy your machine is, etc. We would also have to sprinkle retry logic all over the place to make it work on different people's machines (e.g. someone else might hit it while we clean out the obj directory, someone else within MSBuild, etc.). I would rather not set the precedent.
What I would suggest is either using an antivirus where developers know how to use opportunistic locks, or putting the bin directory on a list of directories that are excluded from realtime scanning...

I've hit this problem in the past for other projects and the solution was to actually retry infinitely or after a few attempts and bail out. This is especially true if you are running on top of a network share.
Adding the exclusion to the antivirus is not so great.

Although I haven't changed my setup, I'm not hitting this issue anymore.


UnnecessaryAbstractClass false positive

Expected Behavior
Don't report following code:
abstract class Fake : Base() // UnnecessaryAbstractClass
abstract class Base {
	open val order: Int get() = 0
	abstract fun String.foo()
}

Observed Behavior

"An abstract class without a concrete member can be refactored to an interface."

No, it can't:

Fake interface couldn't extend a class.
it has concrete members inherited from Base
I get a detection even if I add an actual class using the "abstractness":

class Foo : Fake() {
	override fun String.foo() = TODO("not implemented")
}

Note: String. might be a red herring, you can ignore that part, just wanted to report what I had. More minimal repro might exist.
Steps to Reproduce
Just copy above code into a test or code checked by Detekt 1.20.0.
Context
Real context is creating fakes for a test:

The abstract method is never invoked in the test, so there's no need to implement it, hence using abstract class. The class must be unique because it's ::class object is used.
Your Environment

Version of detekt used: 1.20.0
Version of Gradle used (if applicable): N/A
Gradle scan link (add --scan option when running the gradle task): N/A
Operating System and version: all
Link to your project (if it's a public repository): https://github.com/TWiStErRob/net.twisterrob.cinema/pull/114/commits/0a43c8e914bcd46fd9db4a80baeba0470a7e07df in https://github.com/TWiStErRob/net.twisterrob.cinema/pull/114

Isn't this an edge case? I mean, in general a class that only extends another class and does nothing is unnecessary, you could use the base class. In this case the code uses the type but that seems to me like an edge case.
I'm not against this change but I'm worried that by allowing this we will add a lot of false negatives.

If you build an expression hierarchy, or similar, you might introduce an intermediate base class for grouping part of the class hierarchy.

We face the same issue when wrapping a third party library classes so it's not an edge case.

I agree, maybe we could add a configuration here but I don't think that it could be even needed. Do you have bandwith to provide a PR?

Me, not for a month probably.
@dawidhyzy-ifolor can you please share your use case, it might help with unit tests.

Sure!
We have DataSource defined in the domain layer
abstract class PhotosMediaItemPagedDataSource : PagingSource<NextPageToken, MediaItem>() {

    interface Factory {
        fun create(albumId: AlbumId?, pageSize: Int?): PhotosMediaItemPagedDataSource
    }
}

As androidx.paging.PagingSource is an abstract class we can't refactor PhotosMediaItemPagedDataSource to an interface - interface can't extend abstract class.
Please, let me know if you need more information.

I think the rule of thumb is because Interface can't extend abstract class. I agree that we should at least provide a configuration to this rule.

@BraisGabin May i take this?

Go ahead

I'm fine with or without configuration. I suppose that in that case without is better because it keeps the api cleaner but that's your call. I think that no one here have a hard opinion about this topic.

Ask yourself if there's a possibility for the case you're flagging. It it possible to get code that passes/fails the rule? And are both of them valid engineering choices? (i.e. due to decisions on preferences or restricted by constraints)

This issue is closed, but it still happens in 1.21.0 (gradle plugin). Am I doing something wrong?
Example run: https://github.com/EdwarDDay/detekt-abstract-class-unnecessary/runs/7410220388?check_suite_focus=true
Example code:
abstract class Foo {
    abstract val i: Int
    val j = 1
}

abstract class Bar : Foo() // UnnecessaryAbstractClass issue

That is very strange since that almost exact example is in a unit test:
https://github.com/detekt/detekt/pull/4399/files#diff-28879f691cfd6348ba8a90a837260b8c317e548295c9c400b1199de7bafdc8f4R75-R85

I can confirm that the fix did not fix the reported problem:
https://github.com/TWiStErRob/net.twisterrob.cinema/pull/173/files

@BraisGabin ca  you please reopen?

Hi, @EdwarDDay the TC given by you is passing when you run detekt with type resolution on but if you run w/o type resolution then it is failing. Can you confirm in which mode you are running the detekt?
Hi @TWiStErRob can you confirm in the PR you run detekt /w type resolution or w/o it? cc @BraisGabin

Yes, I just ran the detekt task and not detektMain. When I read the docs right, this means, that the task runs w/o type resolution.

@atulgpt If you meant https://github.com/detekt/detekt/issues/4753#issuecomment-1191312692, the linked PR https://github.com/TWiStErRob/net.twisterrob.cinema/pull/173 has a commit whose build found this. Navigating the build you can find the workflow file which refers to this hacky w/o type resolution task (since then I've moved on to per-module type resolution)

Hi @TWiStErRob yup talking bout https://github.com/detekt/detekt/issues/4753#issuecomment-1191312692
if this is resolved we can close this issue


Status endpoint only enabled if status.user and status.password specified

It also appears that status.user and status.password are not actually used to get status anymore.

router.status.user and router.status.password should be required properties to secure the /routes endpoint. If that endpoint is no longer secured it may have been a regression introduced when we add the /health endpoint, which is meant to be open.
Prioritizing for engineering review.

Ah, I don't know anything about /routes.  So, it may still work to secure /routes.  In that case the question may be why do I need to specify status.user and status.password in order for gorouter to listen on /health?

I just confirmed /routes does work with the user and password provided.  I didn't know /routes existed and was confused that I needed to provide a user and password to use it.  So, you can probably close this.

Got it.
See https://github.com/cloudfoundry/gorouter/#instrumentation
status is a poor property name when the endpoints it secures are /varz and /routes. I see the job spec descriptions for the status properties could be improved. Will do so.
If you have suggestions for what could be improved in docs, etc to have made the solution more self-discoverable, let us know.


Output JSON schema during build process

It would be great if Serde could optionally produce a JSON schema as a side-effect of the build process. AFAIK it has all the information it needs to write one. You just need to translate the structs/enums to their appropriate schema representations (read: matching JSON type).
Additional:
While the above is an awesome starting block, it would also be really nice if you could compile-time check that Serde's JSON will validate against an externally provided schema. This isn't totally necessary, as you could do this after the fact with a tool like ajv. It would just provide stronger guarantees if it was compile-time checked.
Motivation

Compatibility: Presently there is no way to guarantee that JSON produced by Serde is compatible with another framework. We can only write tests against JSON samples and write code to match an API spec. We have no way of knowing if either of them is up-to-date or correct.
Extendability: Having a portable artifact of your data-representation is an enormously useful tool. In many dynamic languages, you can auto generate data bindings and UIs provided a schema. This allows devs to quickly develop across platforms and languages while maintaining integrity of their data.

Anticipated Questions:

Why Serde? - Serde already has all of the user-facing hardware necessary to produce a schema. Using attributes and types already in the user's code makes adding this feature "free" and to existing libraries.
Why at compile-time? - Validating against a schema at compile-time enables devs to "Hack without fear", because they will know that they are properly encoding their data types. It allows devs to easily update their code and immediately know if their schema/data-bindings are out of date.

It would be great if Serde could optionally produce a JSON schema as a side-effect of the build process. AFAIK it has all the information it needs to write one. You just need to translate the structs/enums to their appropriate schema representations (read: matching JSON type).

This is tricky. It seems like Serde has all that information but in reality we don't. At compile time, we have type information about only one struct/enum at a time. For example we might know that your struct S has a field whose type is a::b::c::D<T> but there is no way to use that to find out more about D's type or what types T is used as. Serde's job is to stamp out some code that implements Serialize/Deserialize, and for that we blindly assume that a::b::c::D already implements Serialize/Deserialize without knowing anything else about it.
Conversely at runtime Serde deals with values only, not types. So there would be no way for Serde to come up with a JSON schema for a particular type unless we can get a concrete instance of that type from somewhere and try serializing it. And even then we would be blind to any shenanigans the type might try to do, like serializing itself as an integer if the day of the month is prime and serializing itself as a string otherwise.
https://github.com/serde-rs/serde/issues/345 is tracking a similar request.
If this is a feature you need (rather than "wouldn't it be great if..."), I think a more promising place to start would be implementing this as a compiler plugin, similar to how Clippy works. I think they have a lot more type information at that stage than what we have in Serde.

@dtolnay

It seems like Serde has all that information but in reality we don't.

I'm not really familiar with the implementation of Serde, but from your description it sounds like you have enough information, you just don't have it all at once. There is a set of unconnected (from the point of view of Serde) points of analysis:

When Serde derives struct S, it knows the names of it's fields (what the schema calls properties), and their types (but not if they're templates?).
When Serde derives a::b::c::D<T>, and assuming it isn't also a struct, it must determine what JSON type it serializes to, or what named values it serializes to if it's a string enum.
Even if these points aren't connected, they can still be defined individually, and combined after the fact. Even if the type information is missing for a sub-field, when the struct is added to the schema, you can reference it by name and it will work later. Since all types used in a struct must must be serializable (i.e. implement or derive Serialize) in order to be able to serialize it later, we can know that at least by the end of the compilation, the schema will be complete.

If this is a feature you need...

This isn't something I need in the long run, it's more something I want for Rust in the long run. Having something like this would really help make it even more compelling for web devs and server architects (not exclusively, just especially). If this is something that's really, truly, impossible for Serde, my next recommendation (if anyone is watching) is to use external mixins to call one of the JS, Python, or Ruby libraries that could get this done. But that would at best be a hack, and not really the most appropriate thing for Rust long term. I agree Clippy would be a good choice for linting (validating) against the schema.

Referring to what @oli-obk said on  serde-rs/serde#345:

Nope. Serde works with values of concrete types. You want to work with concrete types directly.

Could you elaborate?

Nope. Serde works with values of concrete types. You want to work with concrete types directly.

Could you elaborate?

This is the same thing I was getting at above:

Conversely at runtime Serde deals with values only, not types. So there would be no way for Serde to come up with a JSON schema for a particular type unless we can get a concrete instance of that type from somewhere and try serializing it.

In other languages like Java or Go, data serialization is typically built on runtime reflection which lets you do things like this at runtime:

@lylemoffitt: Language, give me information about the type S.
Language: S is a struct type with 6 fields.
@lylemoffitt: Give me information about the first field of S.
Language: It is called x and the type is com.enterprise.Foo, which itself is a struct.
@lylemoffitt: Tell me more about the type com.enterprise.Foo.
Language: Sure, no problem.

Rust does not have runtime reflection.

@lylemoffitt: Rust, give me information about the type S.
Rust: lol you wish :nerd_face:

Instead Serde serializes structs by generating code at compile time to serialize them.

@lylemoffitt: Please tell me any time someone writes #[derive(Serialize)]
Rust: bro, someone just wrote that on "pub struct S { x: Bar }"
@lylemoffitt: Cool so the first field is called x? What type is Bar?
Rust: lol I have no idea
@lylemoffitt: Is it in the same crate as S? Does it implement Serialize?
Rust: it's called Bar bro, your guess is as good as mine
@lylemoffitt: Is it a real type though? Like are you going to be able to compile it later?
Rust: maybe? chill out bro, we'll find out
@lylemoffitt: I guess "impl Serialize for S { /* ... */ }"
Rust: someone just wrote #[derive(Serialize)] on "pub enum Bar { /* ... */ }"
@lylemoffitt: Cool, that's the same Bar as before right?
Rust: I can't tell you

Hopefully that clarifies the limitations of putting together a complete JSON schema at Serde's position in the stack. I didn't know about "$ref", thanks for pointing that out. Possibly we could do something like generate an individual schema for each struct that we process that uses "$ref" to refer to all of the fields, but there is no guarantee that you would be able to wire those together later. For example if someone does use A as B we would generate a ref to B and a schema for A with no indication that those are the same type.
In very simplified form, compilation works like this:

Compile any other crates that are transitive dependencies of the current crate.
Parse the current crate. This produces very basic syntactic information. Is this Bar the same as that Bar? Don't know.
Do macro expansion, which includes the Serde code generation personified above.
Resolve what each identifier refers to, do type inference, type checking, borrow checking and a lot of other magic.
Run compiler plugins like Clippy. These get to take advantage of a lot more type information that Serde does, but are also much more unstable and break more often.
Make an executable.

That is why I suggested Clippy as a more promising starting point for this. Not that JSON schema should be built into Clippy, but that something at that stage would have all of the relevant information available and would be able to do the job better. It may make sense for the Serde team to own this functionality but basically none of what we already have is going to be helpful, so we would be starting from scratch whether it goes in Serde or into a separate library.

Excellent explanation.  Love your personification of Rust.
To summarize:

Serde has the names of types, but not the actual types.
Serde exists only during the macro expansion phase and not during the type resolution phase.

This leads me to some questions:

Doesn't Serde have a compiler plugin (on nightly)? I understand that you're working towards Macros 2.0, but aren't quite there yet. Does this change what phase Serde runs in, or just add an extra phase?
Would it be possible to have Serde generate a partial, intermediate, representation of a schema that could later be fixed and updated by a compiler plugin?
I found an implementation of RTTI for Rust. Is this any different than what you're doing with Serde (on stable)? Looking at the source, it looks like mostly macros.
Is type info for other, dependent crates available during macro expansion?

Thank you for bearing with me by the way.

It would be possible to add a static method on the Serialize trait that has a Sized bound. This method would then return some enum describing the type (which might require calling the method on the field's type). You'd basically get a runtime representation of the type, barring any information only available if you have a concrete instance. At that point we're replicating the rtti crate but with macros 1.1. Thus I think this should be outside the serialize/deserialize traits, but I could see it being in serde.
Even then, you can't do compile time checks, but you can simply write a unit test validating the format against a schema

Doesn't Serde have a compiler plugin (on nightly)? I understand that you're working towards Macros 2.0, but aren't quite there yet. Does this change what phase Serde runs in, or just add an extra phase?

Serde has a "procedural macro" for nightly (will be stabilized in Rust 1.15 in February). This is the same phase as what used to be the serde_macros "compiler plugin," just the mechanics have changed in order to stabilize a part of it. My talk at the most recent SF meetup discusses how these work (start at 3:00).

Would it be possible to have Serde generate a partial, intermediate, representation of a schema that could later be fixed and updated by a compiler plugin?

@oli-obk can speak more to how compiler plugins work since he has contributed extensively to Clippy. My current understanding is that it would be better to just do it all in a compiler plugin. It should have all the same syntactic information plus also type information.

I found an implementation of RTTI for Rust. Is this any different than what you're doing with Serde (on stable)? Looking at the source, it looks like mostly macros.

As @oli-obk responded above, this is equivalent to what Serde is doing and has all the same limitations.

Is type info for other, dependent crates available during macro expansion?

No, just syntax-level information about the current crate. Think of it as nothing more than the textual source code of the current crate.

Using a compiler plugin to spit out a schema for a type is almost trivial once you get a handle of the type. Clippy actually has a very rudimentary inspection lint that almost does that. I'd be happy to mentor any extensions to it

@dtolnay Thank you for referring me to that video! I was looking for something more recent than the RustConf2016 I had seen. Was this video on TWiR? I hadn't seen it there...
If it's ok, I have some questions about your talk:

Do you have to use and AST in the middle, or will any old string manipulation work (assuming the result is valid rust code)?
Can I redirect the generated code to another file?
Does the derived source code replace the written code, or just append it?
Can quote! generate further derive attributes or other macro calls that will later be expanded?

@oli-obk Could you refer me to specific lint you're talking about?

@dtolnay Thanks for pointing out Valico! I totally didn't know they had a schema building & validation. That really helps solve a chunk of this problem. I don't know if a direct port is the best thing though. If you're adding it to Serde, it's probably best to try and generalize to try and support schema for other languages. Is Serde's existing par/gen infrastructure something that would be helpful here?

If it's ok, I have some questions about your talk

I moved it to https://github.com/dtolnay/talks/issues/1 to not derail the discussion here:balloon:.

If you're adding it to Serde, it's probably best to try and generalize to try and support schema for other languages. Is Serde's existing par/gen infrastructure something that would be helpful here?

Good call, but whether we generate JSON schema directly or a higher-level broadly applicable Serde schema, eventually we will need a way to get a JSON schema so I would rather reuse an existing high-quality implementation of that.

The inspector lint is implemented here: https://github.com/Manishearth/rust-clippy/blob/master/clippy_lints/src/utils/inspector.rs

@lylemofitt @dtolnay FYI, I'm working on a crate for generating MongoDB-flavored JSON schemas: https://github.com/H2CO3/magnet — it's not generating 100% standard-compliant JSON schema because MongoDB's spec is more precise and powerful, but it's close, and I think my approach is pretty reasonable should someone want to extend/build upon it.

For future readers that seek a solution, I found this repository:
https://github.com/GREsau/schemars


Turning on, hue ambient white lights go green in transition

Latest version of HA and AL and hue firmwares
When turning on the lights and they are warm color temp and it's now daytime, they turn green on their way to white cool temp. It's a very unpleasant green effect that doesn't happen when using the sliders, only when AL adjusts when turning on going from one extreme of color temp to the other.

Can you enable debugging logging (https://github.com/basnijholt/adaptive-lighting#having-problems) and tell me the service_data associated with setting the color of the lights?

Hi, I have just updated, and the issue doesn't seem to be happening anymore. Was this possible due to the change to kelvin vs mired or whatever was fixed? Thanks!

That's possible 😄
Thanks for reporting back, let's close this issue.


PP: Do we need to add some content about how to fine people?

Northumberland - Its part of the terms of conditions of the permit that states where the badge must be displayed otherwise they will get a fine, this is supplied when the permit is issued

Closing this issue as I don't think there's anything useful that's not out of scope.


Vi keybindings for up/down reversed

Issue #12 adds vi keybindings, but the bindings for j/k up/down are reversed. j should move the cursor down. k should move the cursor up (similar for related modifiers). man vi

Fixed in v0.9: https://github.com/mjrusso/scoot/releases/tag/v0.9


Provide exemplary GUI tests for Corpus Structure Editor

For the same reasons as stated in #59, we should set up exemplary GUI tests, e.g., for the corpus structure editor, using RCPTT, and perhaps SWTBot where it makes sense.
Again I think that this should be part of the Dev Platform Milestone, but I'd be happy to push a fix to after the project meeting.
@thomaskrause Perhaps we should create a project meeting milestone to prioritize issues?

We should not add issues to milestones without discussing them first. Its now impossible to distinguish the issues we already agreed on and the ones you added. This is seriously not OK. We did discuss the issue of having GUI tests for the first developer release and as far as I remember we decided against them because of time issues.
In my opinion, milestones are not a good way of prioritizing issues. Milestones define a specific feature/bug set with a given deadline. Moving a lot of issues from one milestone to another because the deadline was missed causes additional management overhead. In the end you just put every issue to the milestone, and moving these "not so important" issues from milestone to milestone until there is no distinction between the "All issues" and "what we have to do for this milestone". Also, it is not good for the morale if a milestone just keeps piling up issues and never actually converges to 100%. It is also bad for project planning if you can't know if you are on track implementing the milestone in time.
I would propose to add issues only to a milestone after the planning phase if

it turns out we missed something very important or something is a pre-condition for another issue already in the milestone: this must be done in coordination with the participating parties of a milestone
we finish all issues for a milestone before the deadline and have time to include new issues

I also think that having an actual developer releases before our project meeting is still important for our progress. How we call this release is not as important ("Development Platform Preview"?) but having performed an actual release with all the automated steps etc. is an important experience which outcome we have to discuss in the meeting. Not the planning, but how it went.

We should not add issues to milestones without discussing them first. Its now impossible to distinguish the issues we already agreed on and the ones you added.

No, the new ones are #57, #59, #60.
#57 is minor, but an omission.
Why I put in #59 and #60 is explained below.

This is seriously not OK. We did discuss the issue of having GUI tests for the first developer release and as far as I remember we decided against them because of time issues.

I think it is okay with respect to the definition of milestone, cf. below, and not okay as it shows that we lack a better process for planning our work. I don't see this as a huge problem right now, as we haven't had much hands-on experience yet in terms of working along the process. It's part of the learning curve to figure out a better process. Part of this is a) using an agreed-upon definition of milestone (i.e., the deadline is not part of the actual milestone), and b) defining milestones and the work towards them before starting the work (which we haven't done really in this case).

In my opinion, milestones are not a good way of prioritizing issues. Milestones define a specific feature/bug set with a given deadline. Moving a lot of issues from one milestone to another because the deadline was missed causes additional management overhead. In the end you just put every issue to the milestone, and moving these "not so important" issues from milestone to milestone until there is no distinction between the "All issues" and "what we have to do for this milestone". Also, it is not good for the morale if a milestone just keeps piling up issues and never actually converges to 100%. It is also bad for project planning if you can't know if you are on track implementing the milestone in time.

Agree. But the PM/SE definition of milestone places importance on the result, not the duration, and not the deadline. A milestone is the completion and approval of a deliverable, for example, not the fact that it has happened by a specific date. I think the problem we've run into here is that we haven't carved out a well-defined milestone of "Dev Platform" before starting the work. IMHO, providing the framework for the basics of good practice (here: GUI and unit tests) is part of the deliverable, the completion of which is defined by the milestone. If we overrun, we overrun, but the milestone stays the same. Also I think mixing (rudiments of) agile and plan-based methods hasn't worked well so far.
I, too, think that pushing "not so important" or in fact any issues down a series of milestone is useless and detrimental. But so is cutting down deliverables in order to meet a deadline.

I would propose to add issues only to a milestone after the planning phase if

Yes, stressing that we do actually need a planning phase. And smaller milestones (this one's been cut out way too big).

* it turns out we missed something very important or something is a pre-condition for another issue already in the milestone: this must be done in coordination with the participating parties of a milestone

* we finish all issues for a milestone before the deadline and have time to include new issues

Agree.

I also think that having an actual developer releases before our project meeting is still important for our progress. How we call this release is not as important ("Development Platform Preview"?) but having performed an actual release with all the automated steps etc. is an important experience which outcome we have to discuss in the meeting. Not the planning, but how it went.

Sure. How about we start tomorrow morning with 1) defining the Platform milestone and what should be in it, and 2) defining a Preview milestone to collect relevant issues for the project meeting, then divide the assigned issues over the two. This will maybe cost us an hour but will have the effect that we have a defined (and more motivating) goal for the meeting release, but don't have to neglect important parts of the "platform" release due to time pressure.
For the future, we should implement a separate planning phase more clearly.
Night night.

So, as a way of going forward with this, I propose to create another milestone "Preview Platform"
and assigned it the following issues:
#16, #23, #29, #32, #33, #35, #39, #41, #42, #44, #47, #57
This is the set of issues originally assigned to the Dev Platform Milestone.
To the Dev Platform Milestone, I'd start with assigning #59, #60.
At a point in the future (I suggest the project meeting itself, or directly after it), we should then do the work to define this milestone properly.
I think the project meeting would be a good place because we will likely discuss further future milestones up until the February 2020 Beta release, which will have to build on the Dev Platform release.

It seems like RCPTT does not support pure E4 applications (https://bugs.eclipse.org/bugs/show_bug.cgi?id=441067). I try to go ahead with SWTBot, but also hitting some weird problems.

Jubula might be an alternative, although it is more complex AFAICR, and I've never tested on pure E4.


🐸Coqui for offline TTS and STT

Proposal
Hi Leon team,
Thx for this great project. I've been following your impressive progress almost from day one (newsletter, code, and so on.).
As far as I can see in your road map, there is an inquiry of offline TTS and STT options, and Coqui gracefully listed there :)
I think 🐸 Coqui projects have all the bells and whistles for real-time offline or even online TTS and STT inference, and, the best part, everything is open-source (models and source code).
I am also personally ready to help along the way if you are interested in adding 🐸  to the Leon stack.
I  drop some links below and feel free to reach out or continue the discussion here however you prefer.
🐸TTS:  https://github.com/coqui-ai/TTS
🐸 STT: https://github.com/coqui-ai/STT
Gitter channels: https://gitter.im/coqui-ai

:wave: Hi, thanks for reaching out and for willing to help with this integration.
Indeed, Coqui will definitely be part of Leon in the future. Let's keep in touch and see how we can make this happens. Awesome work with Coqui BTW!

 ✨ Take a look to implement a new offline TTS

 ✨ Take a look to implement a new offline STT

Thanks to @johannbarbie, Coqui STT is now the default STT solution for Leon 🎉

Docs have been updated https://github.com/leon-ai/docs.getleon.ai/commit/cca3a13cd7f2e0ecbf5c5096d9d443b7a3a192b5.
Coqui STT solution appears on the landing page https://github.com/leon-ai/getleon.ai/commit/10c7922d930e36b456979eeb5e38a51030c40061

Looking forward to implement Coqui TTS! Will keep this issue open as long as it is not there.
cc @JRMeyer @reuben

Hi there what an impressive job!
Do you have any rough timeframe plans/wishes 😄 for the TTS part?

Hi there what an impressive job!
Do you have any rough timeframe plans/wishes smile for the TTS part?

Thanks!
I don't have time frame for this but it will happen before the official release. If anyone is willing to contribute in adding Coqui TTS, feel free to do it. Otherwise I will focus on it a bit later.

Just to note, we've added a basic Python API for TTS if it helps for better integration. https://github.com/coqui-ai/TTS#-python-api


Require-ing vendor libraries

I really want to be able to require libraries that I have installed in my vendor location (ie. bower_components). I have tried a solution I found on SO, but it doesn't seem to work.
Is what I'm trying to do a really bad idea? Is there an easier way? I want to be able to require all of my libraries (vendor or not) so that I can write better/more isolated tests.

Just realised that #988 sums up my underlying promblem, and was opened by @paulmillr.


Cannot enter Polish characters (diacritics) from keyboard

I can't enter any special character (Polish) with keyboard, using right ALT + letter: ęółśńćżź. Just no reaction - nothing is showing.
I can enter this characters everywhere else (including cmd.exe) - except PowerShell Console.
I have no problem with printing any of the Unicode characters in any console.
I have Polish programmers keyboard input set.
Please help.
PSVersion                      6.2.0
PSEdition                      Core
GitCommitId                    6.2.0
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Remove PSReadline and check the issue again.

Yes, It was PSReadLine. I've reinstalled it and now everything it is working. Thanks!


Dedupe gatsby transformer sharp

the most recent push has updated the demo site sharp plugins to match the ones in the theme package, and now we have

one gatsby-transformer-sharp
one gatsby-plugin-sharp
two sharp but they are both 0.30.x instead of also including a 0.29.x version

the best i can guess the reason for this might be is because we have two gatsby versions. the theme package is at 4.8.2 and the demo is at 4.4.0  - i tried bumping the gatsby version and this did not resolve it.

so hopefully this helps!

can we bump the dev dependency for gatsby to 4.19.0 while we're in here


style: change finished ressource icon to success

Related to #4127
Change finished executable succeed icon from warning to checkmark
Microsoft Reviewers: Open in CodeFlow

I think this broke "Failed to start" with a null exit code:

Filed an issue here https://github.com/dotnet/aspire/issues/4498

@adamint It's totally mine, I didn't check this case. Do you want me to fix it?

@pierrebelin PR is most welcome :)


[Feature Request] Remove Flask/FastAPI from promptflow-core dependencies?

Is your feature request related to a problem? Please describe.
We would like to bring promptflow-core into our web apps, but we're concerned about the Flask/FastAPI dependency as it may end up conflicting with our own dependencies, plus we can't afford to bring in a lot of dependencies, as our pip install is already too slow.
Could those dependencies be optionals or in another package? Why must they be in promptflow-core? Is there a more minimal package if we just want to use Prompty?

@pamelafox could you share some details about why you want to build your own web app instead of leveraging the PF serving app which already has good support for different engine and deployment? That will be very helpful for us to understand customer's requirement.

We are not using a microservices model, which I guess is what you're imagining? Instead, we're deploying a single monolithic app and would like to use promptflow as part of the web app calls. Microservices are popular, but so are monolithic apps, so we need to support both.


[helm3] Urgent issue regarding HPA managed deployments

Output of helm version:
version.BuildInfo{Version:"v3.0.0", GitCommit:"e29ce2a54e96cd02ccfce88bee4f58bb6e2a28b6", GitTreeState:"clean", GoVersion:"go1.13.4"}

Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.3", GitCommit:"b3cbbae08ec52a7fc73d334838e18d17e8512749", GitTreeState:"clean", BuildDate:"2019-11-14T04:24:34Z", GoVersion:"go1.12.13", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.7-gke.23", GitCommit:"81c87c699557fed991e292cd328b2129c2f242a2", GitTreeState:"clean", BuildDate:"2019-11-07T19:23:23Z", GoVersion:"go1.12.11b4", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.):  GKE
Please try the following:
helm install static-replicas-app stable/nginx-ingress --set controller.replicaCount=5
helm install autoscaled-app stable/nginx-ingress --set controller.autoscaling.enabled=true,controller.autoscaling.minReplicas=5,controller.autoscaling.maxReplicas=10

When all pods are up and running perform:
helm upgrade static-replicas-app stable/nginx-ingress
helm upgrade autoscaled-app stable/nginx-ingress

And observe all but one of the autoscaled-app pods get recreated!! This is a major issue as I kind of took a leap of faith and am using helm3 in production, we can't redeploy apps without getting downtime (most are hpa managed deployments)

I've confirmed that helm2 does NOT have this behaviour:
helm2 install --name static-replicas-app-2 stable/nginx-ingress --set controller.replicaCount=5

helm2 install --name autoscaled-app-2 stable/nginx-ingress --set controller.autoscaling.enabled=true,controller.autoscaling.minReplicas=5,controller.autoscaling.maxReplicas=10

And once all pods are up:
helm2 upgrade static-replicas-app-2 stable/nginx-ingress
helm2 upgrade autoscaled-app-2 stable/nginx-ingress

And no pods were recreated.

@naseemkullah Trying to get better understanding here. Some questions about your issue.
Once the install is complete, is there a reason for then doing the upgrade? How many of the autoscaled-app pods were created after install and then after upgrade? Can you show the output of this command helm upgrade autoscaled-app stable/nginx-ingress --debug --dry-run?

Hi @hickeyma, first off thanks and I really enjoyed the helm 3 deep dive, it prompted me to go ahead and upgrade!
Alright so:

Reasons for upgrades vary, sometimes we want a new feature, or to change a value (but this generally triggers a proper kubernetes rolling update). And what makes matters worse is our CI runs a helm upgrade on all charts, which normally does nothing, especially for charts whose values did not change. But on every try, HPA managed deployments'  majority of pods get re-created and NOT in a proper kubernetes rolling update manner.

In the provided example 5 replicas were created, and then once the upgrade was applied, 4/5 pods were re-created at the same time (for no reason!)

Release "autoscaled-app" has been upgraded. Happy Helming!
NAME: autoscaled-app
LAST DEPLOYED: Wed Nov 27 10:06:30 2019
NAMESPACE: test
STATUS: pending-upgrade
REVISION: 4
TEST SUITE: None
USER-SUPPLIED VALUES:
controller:
  autoscaling:
    enabled: true
    maxReplicas: 10
    minReplicas: 5

COMPUTED VALUES:
controller:
  addHeaders: {}
  admissionWebhooks:
    enabled: false
    failurePolicy: Fail
    patch:
      enabled: true
      image:
        pullPolicy: IfNotPresent
        repository: jettech/kube-webhook-certgen
        tag: v1.0.0
      nodeSelector: {}
      podAnnotations: {}
      priorityClassName: ""
    port: 8443
    service:
      annotations: {}
      externalIPs: []
      loadBalancerIP: ""
      loadBalancerSourceRanges: []
      omitClusterIP: false
      servicePort: 443
      type: ClusterIP
  affinity: {}
  autoscaling:
    enabled: true
    maxReplicas: 10
    minReplicas: 5
    targetCPUUtilizationPercentage: 50
    targetMemoryUtilizationPercentage: 50
  config: {}
  configMapNamespace: ""
  containerPort:
    http: 80
    https: 443
  customTemplate:
    configMapKey: ""
    configMapName: ""
  daemonset:
    hostPorts:
      http: 80
      https: 443
    useHostPort: false
  defaultBackendService: ""
  dnsPolicy: ClusterFirst
  electionID: ingress-controller-leader
  extraArgs: {}
  extraContainers: []
  extraEnvs: []
  extraInitContainers: []
  extraVolumeMounts: []
  extraVolumes: []
  hostNetwork: false
  image:
    allowPrivilegeEscalation: true
    pullPolicy: IfNotPresent
    repository: quay.io/kubernetes-ingress-controller/nginx-ingress-controller
    runAsUser: 33
    tag: 0.26.1
  ingressClass: nginx
  kind: Deployment
  lifecycle: {}
  livenessProbe:
    failureThreshold: 3
    initialDelaySeconds: 10
    periodSeconds: 10
    port: 10254
    successThreshold: 1
    timeoutSeconds: 1
  metrics:
    enabled: false
    port: 10254
    prometheusRule:
      additionalLabels: {}
      enabled: false
      namespace: ""
      rules: []
    service:
      annotations: {}
      externalIPs: []
      loadBalancerIP: ""
      loadBalancerSourceRanges: []
      omitClusterIP: false
      servicePort: 9913
      type: ClusterIP
    serviceMonitor:
      additionalLabels: {}
      enabled: false
      namespace: ""
      namespaceSelector: {}
      scrapeInterval: 30s
  minAvailable: 1
  minReadySeconds: 0
  name: controller
  nodeSelector: {}
  podAnnotations: {}
  podLabels: {}
  podSecurityContext: {}
  priorityClassName: ""
  proxySetHeaders: {}
  publishService:
    enabled: false
    pathOverride: ""
  readinessProbe:
    failureThreshold: 3
    initialDelaySeconds: 10
    periodSeconds: 10
    port: 10254
    successThreshold: 1
    timeoutSeconds: 1
  replicaCount: 1
  reportNodeInternalIp: false
  resources: {}
  scope:
    enabled: false
    namespace: ""
  service:
    annotations: {}
    enableHttp: true
    enableHttps: true
    enabled: true
    externalIPs: []
    externalTrafficPolicy: ""
    healthCheckNodePort: 0
    labels: {}
    loadBalancerIP: ""
    loadBalancerSourceRanges: []
    nodePorts:
      http: ""
      https: ""
      tcp: {}
      udp: {}
    omitClusterIP: false
    ports:
      http: 80
      https: 443
    targetPorts:
      http: http
      https: https
    type: LoadBalancer
  tcp:
    configMapNamespace: ""
  terminationGracePeriodSeconds: 60
  tolerations: []
  udp:
    configMapNamespace: ""
  updateStrategy: {}
defaultBackend:
  affinity: {}
  enabled: true
  extraArgs: {}
  extraEnvs: []
  image:
    pullPolicy: IfNotPresent
    repository: k8s.gcr.io/defaultbackend-amd64
    runAsUser: 65534
    tag: "1.5"
  livenessProbe:
    failureThreshold: 3
    initialDelaySeconds: 30
    periodSeconds: 10
    successThreshold: 1
    timeoutSeconds: 5
  minAvailable: 1
  name: default-backend
  nodeSelector: {}
  podAnnotations: {}
  podLabels: {}
  podSecurityContext: {}
  port: 8080
  priorityClassName: ""
  readinessProbe:
    failureThreshold: 6
    initialDelaySeconds: 0
    periodSeconds: 5
    successThreshold: 1
    timeoutSeconds: 5
  replicaCount: 1
  resources: {}
  service:
    annotations: {}
    externalIPs: []
    loadBalancerIP: ""
    loadBalancerSourceRanges: []
    omitClusterIP: false
    servicePort: 80
    type: ClusterIP
  serviceAccount:
    create: true
    name: null
  tolerations: []
imagePullSecrets: []
podSecurityPolicy:
  enabled: false
rbac:
  create: true
revisionHistoryLimit: 10
serviceAccount:
  create: true
  name: null
tcp: {}
udp: {}

HOOKS:
MANIFEST:
---
# Source: nginx-ingress/templates/controller-poddisruptionbudget.yaml
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    component: "controller"
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress-controller
spec:
  selector:
    matchLabels:
      app: nginx-ingress
      release: autoscaled-app
      component: "controller"
  minAvailable: 1
---
# Source: nginx-ingress/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress
---
# Source: nginx-ingress/templates/default-backend-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress-backend
---
# Source: nginx-ingress/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - update
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
---
# Source: nginx-ingress/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: autoscaled-app-nginx-ingress
subjects:
  - kind: ServiceAccount
    name: autoscaled-app-nginx-ingress
    namespace: test
---
# Source: nginx-ingress/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress
rules:
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - update
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      - ingress-controller-leader-nginx
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - create
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
---
# Source: nginx-ingress/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: autoscaled-app-nginx-ingress
subjects:
  - kind: ServiceAccount
    name: autoscaled-app-nginx-ingress
    namespace: test
---
# Source: nginx-ingress/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    component: "controller"
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress-controller
spec:

  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
  selector:
    app: nginx-ingress
    component: "controller"
    release: autoscaled-app
  type: "LoadBalancer"
---
# Source: nginx-ingress/templates/default-backend-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    component: "default-backend"
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress-default-backend
spec:

  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
  selector:
    app: nginx-ingress
    component: "default-backend"
    release: autoscaled-app
  type: "ClusterIP"
---
# Source: nginx-ingress/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    component: "controller"
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress-controller
spec:
  selector:
    matchLabels:
      app: nginx-ingress
      release: autoscaled-app
  replicas: 1
  revisionHistoryLimit: 10
  strategy:
    {}
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app: nginx-ingress
        component: "controller"
        release: autoscaled-app
    spec:
      dnsPolicy: ClusterFirst
      containers:
        - name: nginx-ingress-controller
          image: "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1"
          imagePullPolicy: "IfNotPresent"
          args:
            - /nginx-ingress-controller
            - --default-backend-service=test/autoscaled-app-nginx-ingress-default-backend
            - --election-id=ingress-controller-leader
            - --ingress-class=nginx
            - --configmap=test/autoscaled-app-nginx-ingress-controller
          securityContext:
            capabilities:
                drop:
                - ALL
                add:
                - NET_BIND_SERVICE
            runAsUser: 33
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          livenessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          readinessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
          resources:
            {}
      hostNetwork: false
      serviceAccountName: autoscaled-app-nginx-ingress
      terminationGracePeriodSeconds: 60
---
# Source: nginx-ingress/templates/default-backend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    component: "default-backend"
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress-default-backend
spec:
  selector:
    matchLabels:
      app: nginx-ingress
      release: autoscaled-app
  replicas: 1
  revisionHistoryLimit: 10
  template:
    metadata:
      labels:
        app: nginx-ingress
        component: "default-backend"
        release: autoscaled-app
    spec:
      containers:
        - name: nginx-ingress-default-backend
          image: "k8s.gcr.io/defaultbackend-amd64:1.5"
          imagePullPolicy: "IfNotPresent"
          args:
          securityContext:
            runAsUser: 65534
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 0
            periodSeconds: 5
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 6
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          resources:
            {}
      serviceAccountName: autoscaled-app-nginx-ingress-backend
      terminationGracePeriodSeconds: 60
---
# Source: nginx-ingress/templates/controller-hpa.yaml
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.0
    component: "controller"
    heritage: Helm
    release: autoscaled-app
  name: autoscaled-app-nginx-ingress-controller
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: autoscaled-app-nginx-ingress-controller
  minReplicas: 5
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: 50
    - type: Resource
      resource:
        name: memory
        targetAverageUtilization: 50

NOTES:
The nginx-ingress controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace test get services -o wide -w autoscaled-app-nginx-ingress-controller'

An example Ingress that makes use of the controller:

  apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class: nginx
    name: example
    namespace: foo
  spec:
    rules:
      - host: www.example.com
        http:
          paths:
            - backend:
                serviceName: exampleService
                servicePort: 80
              path: /
    # This section is only required if TLS is to be enabled for the Ingress
    tls:
        - hosts:
            - www.example.com
          secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

  apiVersion: v1
  kind: Secret
  metadata:
    name: example-tls
    namespace: foo
  data:
    tls.crt: <base64 encoded cert>
    tls.key: <base64 encoded key>
  type: kubernetes.io/tls

And just to re-iterate, I've tested with helm2 and this issue does not occur. If there is nothing to change, no pods are re-created, and if there is something to change, they are re-created in a rollingly updatingly manner.
Bug with helm3?
Something to do with HPA object and 3 way merge? I have no idea

Very important finding observed by @rimusz
In helm3, the HPA's currentReplicas is not respected, and upon helm upgrade, the deployment goes to the set replicaCount which should be ignored if an HPA is being used.
In the above case, the autoscaled-app has spec.replicas set to 1 so despite the fact that the HPA says 5 replicas should be running, helm upgrade ignores that, sets the spec.replicas value, and then the HPA does it's thing so to speak.
In helm2 the behaviour was favorable, .spec.replicas value was not applied, rather the HPA's current replicas were respected upon helm upgrade.

When I test with Helm 3 doing upgrades on autoscaled-app example, it recreates 4 out of 5 of the pods as follows:
default       pod/autoscaled-app-nginx-ingress-controller-5d45b7cbfc-bb4qf       1/1     Running   0          108s
default       pod/autoscaled-app-nginx-ingress-controller-5d45b7cbfc-f6vnb       1/1     Running   0          108s
default       pod/autoscaled-app-nginx-ingress-controller-5d45b7cbfc-mvk2r       1/1     Running   0          108s
default       pod/autoscaled-app-nginx-ingress-controller-5d45b7cbfc-tjhdr       1/1     Running   0          18m
default       pod/autoscaled-app-nginx-ingress-controller-5d45b7cbfc-z4x9k       1/1     Running   0          108s
default       pod/autoscaled-app-nginx-ingress-default-backend-9895584b9-dwz4m   1/1     Running   0          18m

Correct, this is because regardless of the HPA wanting to have 5 replicas at that time, it looks at the value for the Deployment's spec.replicas and goes to that (which is set to 1), but ideally it should not look at that, it should respect the HPA's desired replica count, just like it was in helm2

Sorry that was slightly misleading:
So our template says the deployment replica is 1
Our hpa says it should be between 5 and 10.
k8s sets the deployment according to what the hpa wants.
helm3 sees that replicas of deployment is now 5, but should be 1, so puts it back to 1.
HPA eventually fixes this, but there is a lack of pods in the meanwhile.

If you can point out where in the codebase it doesn't respect the desired replica count, please do so. As far as I'm aware, Helm supplies a patch to Kubernetes which performs the updates in the backend. I don't believe Helm has any influence here on how a statefulset (or any object for that matter) is updated.

This does not affect stateful sets(or any object other than HPA managed deployments) @bacongobbler , this is HPA managed deployments that are being affected.

Sorry, was a typo on my end. Point still stands.

"problem is on your end", classic

helm v3 doesn't respect replicas set in Deployment by HPA, but helm v2 does.
So there for sure things not working as expected.

I never once said that the problem's on your end. I'm asking you to point out where is the issue in the codebase causing this issue. That's all.

Where is the issue in the codebase causing this issue?
I am now also asking you to point out where the issue is in the codebase.

In the end, there is no issue with the codebase, and it was mentioned that

Helm supplies a patch to Kubernetes which performs the updates in the backend.

I've went ahead and tested with raw YAML files, where a deployment manifest has a replica count set to 1, and it's HPA ranges from 5 to 10, and the observed behaviour was the same as helm3.
The deployment scales down to 1 irrespective of the HPA's settings.
@bacongobbler 1
@naseemkullah 0

Action point:
All charts that offer an HPA, need to add a condition in the HPA managed deployment to omit the replica count of the deployment when rendering the deployment template.
Thanks you all for participating.

interesting results, so helm v2 has a bug and v3 does not, but that can be easily fixed in the chart :-)

@naseemkullah , i just hit the same issue, i am using helm 3.1.2, i couldn't understand from the thread how to avoid or what is the workaround for this behavior,  on every deploy when i am using HPA to pod count go down to defined replica set in the deployment manifest, or go down to 1 if i omit it from the manifest.

Hi @DanOfir, so the workaround is to add an if statement around the deployment's spec.replicas to not template it if HPA is enabled as done in this PR https://github.com/rimusz/charts/commit/dc1b5f64282e1df3dc7d096f3f9a3ac5d6494af4#diff-5f759fc4e425d98b09bce05ac44d0c78

got it! thanks @naseemkullah

I would doubt the workaround can really work,  if you omit spec.replicas, actually k8s will set it to 1.
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#deploymentspec-v1-apps

replicas integer
Number of desired pods. This is a pointer to distinguish between explicit zero and not specified. Defaults to 1.

I experimented omitting spec.replicas in a statefulset, the value is changed to 1, just as the document said.

I would doubt the workaround can really work, if you omit spec.replicas, actually k8s will set it to 1.
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#deploymentspec-v1-apps

replicas integer	Number of desired pods. This is a pointer to distinguish between explicit zero and not specified. Defaults to 1.
I experimented omitting spec.replicas in a statefulset, the value is changed to 1, just as the document said.
@naseemkullah does the workaround work on your side?

It does work @tianchengli . Try it with an HPA, it won't be 1 it will (try to) be the HPA's desired replica count.

just verified that the statefulsets works with your way. @naseemkullah
I remained replicas field but no value like below, it resulted in setting replicas to 1.
replicas:

If I remove replicas field completely, it will respect the replicas number managed by HPA.

What about custom autoscaler operations? How do you handle that? When omitting the replicas - it will re-set them to 1.

If I want to use help upgrade to upgrade from a chart with replicas field to a chart without replicas field, there will be always a time window that Kubernetes will set replicas to be 1.
Solution provided by Kubernetes can't work for helm:
https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#migrating-deployments-and-statefulsets-to-horizontal-autoscaling
The only way that I can think is to hack helm release secret.
Is there a better way to solve it?

Yes the only way I found to work around this behavior with Helm is to do surgery on the Helm release secret. That way, you can avoid mass replica slaughter in production (30 replicas in my case; 29 terminated instantaneously, leaving only one). I had to automate this process to fix dozens of production releases in dozens of K8s clusters before actually removing the replicas field from the respective templates in Git and deploying them with helm upgrade.
The script flow was basically (error-handling removed):

Get the release secret name:

HELM_RELEASE_SECRET_NAME=$(
    kubectl get secret -n "${HELM_NS}" \
        -l "owner=helm,status=deployed,name=${CHART}" \
        -o jsonpath='{..metadata.name}'
)

Back up the secret locally before modifying it (just in case):

kubectl get secret "${HELM_RELEASE_SECRET_NAME}" -n "${HELM_NS}" -o yaml \
    >/tmp/${CHART}-backup.$$.yaml

Extract the release using the "double-b64-gunzip dance":

kubectl get secret "${HELM_RELEASE_SECRET_NAME}" -n "${HELM_NS}" -o json \
    |jq -r '.data.release' \
    |base64 -d \
    |base64 -d \
    |gunzip \
    |jq >"${HELM_RELEASE_JSON}"

Edit the rendered manifest in the release object:

sed -i.orig -r 's/  replicas: [0-9]+\\n//g' "${HELM_RELEASE_JSON}"

Also extract and edit the b64-encoded templates from the release object:

for tmpl in mytemplate1 mytemplate2; do  # XXX use your template basenames
    HELM_TEMPL="/tmp/${CHART}-${tmpl}.yaml"

    # extract and decode
    jq --arg tmpl "${tmpl}" -r \
        '.chart.templates[] | select(.name=="templates/\($tmpl).yaml") | .data' \
        <"${HELM_RELEASE_JSON}" \
        |base64 -d >"${HELM_TEMPL}"

    # edit
    sed -i.orig -r '/^ +replicas: .*/d' "${HELM_TEMPL}"

    # encode and inject
    HELM_TEMPL_B64="$(base64 -w0 <"${HELM_TEMPL}")"
    jq --arg tmpl "${tmpl}" --arg b64 "${HELM_TEMPL_B64}" \
        '(.chart.templates[] | select(.name=="templates/\($tmpl).yaml") | .data) |= $b64' \
            "${HELM_RELEASE_JSON}" \
                >"${HELM_RELEASE_JSON}".updated

    mv -v "${HELM_RELEASE_JSON}".updated "${HELM_RELEASE_JSON}"
done

Repackage the Helm release and update the Secret:

SECRET_PATCH_FILE=/tmp/"${CHART}"-secret-patch.yaml
cat >"${SECRET_PATCH_FILE}" <<__EOT__
data:
  release: $(gzip <"${HELM_RELEASE_JSON}" |base64 -w0 |base64 -w0)
__EOT__

if [ "$(stat -c %s "${SECRET_PATCH_FILE}")" -lt 512 ]; then
    >&2 echo "$0: Aborting patch: Something wrong with ${SECRET_PATCH_FILE}"
    exit 1
fi

kubectl patch secret -n "${HELM_NS}" "${HELM_RELEASE_SECRET_NAME}" \
    --patch-file "${SECRET_PATCH_FILE}"

Obviously you would need to operationalize these statements if you intend to safely automate it like I did, but otherwise, you have an idea of the steps involved for protecting yourself from K8s when Helm submits the resource patch (i.e., "I see you've removed the replicas field from your template(s), so let's remove it from the Deployment! :boom:) and you get the default of 1 replica.

@invsblduck , Thanks for your detail solution.
Has this secret patch solution been used in production? :-)

Thanks @brc — we just used used your patch (with adjustments) at @modal-labs and it fixed all of our HPA managed deployment issues :)

@invsblduck , Thanks for your detail solution.
Has this secret patch solution been used in production? :-)

Yep, I mentioned that I fixed "dozens of production releases in dozens of K8s clusters," but I used those commands in a bash script with set -e and sanity checks at every step, and I was running the script under GNU Parallel ;-)
This "Release object hacking" technique is documented at the Helm website here: https://helm.sh/docs/topics/kubernetes_apis/#updating-api-versions-of-a-release-manifest
PS: I changed my GitHub nick to match my unix login!


encryptor has not main source file

Hi,
Mortar is a awesome project! But, I have some trouble that i get a error with "encryptor has not main source file" when i use kail to build the encryptor project.  I don't know why.  Can you provide a encryptor file  and  a calc.enc file ? Thank you please ~

thanks, can you please provide more logs to dig into the problem, I am glad to fix the problem, I will keep this issue ticket open in the case released the encryptor as Linux version

Yes.And this is the first time that I used this program(Lazarus). I installed it in kali , then i choice encryptor  project and build it,i get this:

I was install  Lazarus in windows too. I copied encryptor.lpr code to deliver.lpr and build successful........

But when I run deliver.exe -d -f bin.enc I get a EReadError:
Exception at<PHONE_NUMBER>010930: EReadError:
Stream read error.

thanks for your testing, there was a bug in the encryptor and encryption key, I have pushed the fix please clone the new repo and have a test, make sure to be root to avoid permission locking, I have added also an encrypted version of mimikatz for testing. let me know your testing results again.
https://github.com/0xsp-SRD/mortar/commit/18736a408b53ad0ff3d62c97a4d08b976e14518d

Nice! The encryptor It's working(win and kali). And thank you for your enthusiastic help. I greatly appreciate it.


Allow Codenarc Plugin to generate and exclude baseline violations

Expected Behavior
The codenarc plugin supports baseline reports to allow legacy codebases to ignore old violations and look for new ones. The gradle configuration should allow:

report format of baseline which produces the standard codenarc baseline report
a config property of excludeBaseline which accepts a baseline report and passes it to the codenarc runner

Current Behavior
Currently the only output formats are Xml, Html, Text, and Console. It is not possible to ignore prior codenarc violations
Context
We are re-evaluating our code standards and moving them into a common repository. Cleaning up failures will take a significant amount of time, and we plan to establish current baselines so we can fix them as we modify the surround code.
We use codenarc in grails 2 and grails 3 projects. With the grails 2 plugin we've been able to set-up the baseline reports, but using gradle for the grails 3 builds is preventing us from taking this approach.

Hello, did you find any solution?


deno: change process.env to Deno.env

process.env.NODE_ENV works, but VSCode unnecessarily complains and it also doesn't feel Deno-y enough so I would propose changing it to the appropriate Deno.env.get("NODE_ENV").
Please correct me if I'm mistaken 🙏
Also so happy to finally get Deno Remix just working, with the latest Deno and all 🥳 Thanks so much @pcattori  👏

I'm sorry I have no idea what this is, I just edited one file in the GitHub UI and switched to the dev branch when comparing before creating the PR 😳

Since this is a change to the templates, this can go to main

Thought about it but want sure 😇
Thank you 🙏


Item Mod doesn't work

When I'm modding the durability of any weapon, after a few uses, they'll break, even though it says that it has high durability... I tried saving, and reloading, but it won't fix the problem

This works for sure so you must be using a bad value?
6553500 is the max value, just use the check-boxes in the codes section for infinite durability of current weapon / shield / bow


Locale default should be current locale instead of American English

Hi there,
Locale as default set as american english. But isn't it appropriate to set as current locale.
/// date locale (language displayed), default to American English
public var locale = Locale(identifier: "en_US") {
    didSet {
        configureView()
    }
}

/// date locale (language displayed), default to Current
public var locale = Locale.current {
    didSet {
        configureView()
    }
}

Thanks, this will be fixed in version 2.0.1


[Question] No language slugs for default language

Is it possible to have the default language without any slugs?
"/home"       >> Default (to be used if no language slug has been specified"
"/en/home" >> English
"/it/home"   >> Italian
"/fr/home"   >> French

Hi @keithazzopardi, you can choose to either have slugs always:
en/home >> default
it/home >> Italian
...
Or to have default language ONLY without slugs:
/home >> Default
en/home X this will fail
it/home >> Italian
This is configurable via LocalizeRouterConfig:
    LocalizeRouterModule.forRoot(routes, {
      parser: {
        provide: LocalizeParser,
        useFactory: ManualLoaderFactory,
        deps: [TranslateService, Location, LocalizeRouterSettings]
      },
      alwaysSetPrefix: false,
      useCachedLang: false
    })

@meeroslav  Thanks for this. I have some difficulty managing to go around my requirement without useCachedLang.
My requirement is the following:
The first time the user goes to the site it will redirect him to the browser's language. Otherwise, it will redirect him last language.
If the user clicks on en site, it will redirect the  user to the default page.

The reason why you should choose only one prefixing option - default with or without language prefix, is because otherwise localize-router would not know how to transform following:
it/buon/giorno -> en/good/morning ?? /good/morning


Add grouping for containers by stacks in /containers

Optional displaying containers in a tree component grouped by stacks.
Currently all containers are shown in the /containers route. If you have multiple Stacks with a lot containers it can get messy. While you could sort by "stack name", you still end up with a long list of containers. Stacks are shown grouped in the /stacks route, but docker container which are not in a stack are missing in this view.
A toggle (or comparable) for enabling grouping containers belonging to a stack in a tree component would enhance the user experience.

(I use the current version of portainer, but due to customer regulations i used this screenshot)
Possible implementation of this feature:

collapse and expand tree view for each stack
(possible) redirect to stack page by clicking on the tree name
(possible) disable grouping and show all containers in one list

Example
For example, Docker Desktop shows all containers, however stacks (docker-compose) are shown in a tree component.

I was just looking to see if this was raise already, lol. I think this makes the layout a lot smoother if they can be collapsed as well. hopefully cookie to remember which are collapsed or opened too for those with long list of stacks.

This would be kind of a merge between the current Containers and Stacks pages. On the Containers page, you can sort by the Stack by clicking the column name, and while this doesn't give the full expandable/collapsible view you're suggesting it might be helpful as an intermediary step.
A clickable stack name has been raised before (#7617) and is with our UX team for consideration in future releases.


Add contains method on Directory to check if a File or other Directory is inside it

The title pretty much says everything.

@leisim If you don't have time, I'd be happy to implement this.


Add test alias for it

This adds the test alias for it and the other variants. This is similar to facebook's Jest. I think there are some cases where test can be more suiting fo the test description.
Specifically, I've created a package which generates markdown documentation from your specs, and I've found that using it can make the wording weird, but test is more natural.
Btw, thank you so much for making this package, it's such a great tool!
Signed-off-by: RJ Garcia<EMAIL_ADDRESS>

Codecov Report

Merging #210 into master will decrease coverage by 0.36%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #210      +/-   ##
==========================================
- Coverage   97.67%   97.31%   -0.37%
==========================================
  Files          22       22
  Lines         905      781     -124
==========================================
- Hits          884      760     -124
  Misses         21       21

Impacted Files
Coverage Δ

src/Console/Application.php
61.11% <0%> (-3.89%)
:arrow_down:

src/Configuration.php
100% <0%> (ø)
:arrow_up:

src/Reporter/AbstractBaseReporter.php
100% <0%> (ø)
:arrow_up:

src/Core/Suite.php
100% <0%> (ø)
:arrow_up:

src/Console/Command.php
100% <0%> (ø)
:arrow_up:

src/Console/InputDefinition.php
100% <0%> (ø)
:arrow_up:

src/Runner/Context.php
100% <0%> (ø)
:arrow_up:

src/Runner/SuiteLoader.php
100% <0%> (ø)
:arrow_up:

src/Console/CliOptionParser.php
100% <0%> (ø)
:arrow_up:

src/AssertException.php
100% <0%> (ø)
:arrow_up:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e785c17...236ed93. Read the comment docs.

Wow! Very cool on all fronts. I should probably get around to updating the README - but peridot is no longer actively maintained. I think kahlan has become the defacto tool for this style of testing. Hopefully is extensible enough to port your work over without too much effort.

Ah, that breaks my heart a little bit, I LOVE peridot and Leo, I’ll check out Kahlan though...

Hmm, I feel like Kahlan almost does too much. I love the simplicity of peridot, and I definitely don’t like their monkey patching. It encourages bad code if you need that much flexibility.
Any chance I could help out with maintaining peridot?

I’m up for it. I have a lot of vested interest in peridot. I use it a ton in my libraries and work. In addition, for platesphp v4 I rewrote all to peridot. Do you want to continue this discussion over email?

Sounds good :) - I have my email address listed on my profile.


MissingPluginException(No implementation found for method getApplicationDocumentsDirectory on channel plugins.flutter.io/path_provider)

I'm trying to test some functionality in my app and when updating flutter to version 3.7.0 this code snippet started giving an error
main() async {
 TestWidgetsFlutterBinding.ensureInitialized();
  SharedPreferences.setMockInitialValues({});

setUpAll(() async {
    await initHive();
  });
test("exemple", () async {
});
}

Future<void> initHive() async {
  try {
    await Hive.initFlutter();

  } on Exception catch (_, e) {
    Groveman.error(_.toString(), error: e);
  }

this is the exception returned when running Hive.initFlutter():
MissingPluginException(No implementation found for method getApplicationDocumentsDirectory on channel plugins.flutter.io/path_provider)

@RafaelRVargas
I see that you are using hive package which is third party. Does the same error occur without using it and just using path_provider ?
See also similar issues in dedicated repo for your reference: https://github.com/hivedb/hive/issues

thanks for helping me, i found something that solved my problems in this issue https://github.com/hivedb/hive/issues/881


Hi,
I downloaded the latest 1.9 beta version and I wanted to build a fat binary for iOS and the simulator. What's the proper way to do that from within Xcode? I tried building and archive, but I don't get a fat binary.

Also, when I use the Archive command in Xcode and then export the product, I get libminizip.a as the only output. That's not the full UnzipKit.framework.

I pushed a change that should hopefully take care of libminizip.a showing up, but I'm not clear on what you're trying to accomplish. I would recommend using Carthage if you want a binary of the framework, or you can download a copy of the built framework from the release's page on GitHub (an example).
Build and Archive, as far as I know, is intended for app targets, not frameworks.

I was just attempting to get a build that had bitcode support and a fat binary that I could deploy to the App Store. I knew that Archive would normally build with bitcode support.
 Took a while to figure it all out. I just hadn't run my app in the simulator in a while and a few frameworks I was using (including UnzipKit) wouldn't run in the simulator for me because I didn't have fat binaries of them.
I didn't realize you had pre-built binaries. That's great!
I don't use Carthage right now, so I just use Xcode to build frameworks.

Cool, it sounds like you're all set?
Also, that's what Carthage basically does, is checks out libraries from GitHub and builds with Xcode for you to manually integrate with your project (in contrast with CocoaPods, which integrates itself into your workspace and builds static libraries it integrates. FYI, the binaries attached to the GitHub releases are produced by Carthage as part of the automated build process.

Yup, I'm all set. Thanks!
I'll have to look into Carthage.


Update Google Maps SDK for iOS 1.10.4 - October 2015

Hello Hirbod,
I noticed the Google has updated Google Maps SDK for iOS.
https://developers.google.com/maps/documentation/ios-sdk/releases?hl=en
The update contains fixes, which sometimes happens in my cordova google maps application.
Could you please say, the optimistic date, when you can update the repository?https://github.com/nightstomp/googlemaps-ios-sdk-for-phonegap
Thanks a lot!

Hi @dukhanov,
optimistic I would say, I've already updated it :P
Just reinstall plugin.

You are awesome.
Thanks again!


onError callback doesn't provide props

According to the documentation the onError callback has a props parameter in case of succes, but this is never set in sound.js. Is somebody working on this? If not, I'll submit a pull request.

Not that I know of. If you don't mind, a PR would be great.


The change of SecurityStamp in ASP.NET Core 2.1

What is the main reason for changing SecurityStamp generating method? I originally parse it and store as a GUID type variable in my custom mongo identity provider and now it changes.
2.0: https://github.com/aspnet/Identity/blob/rel/2.0.0/src/Microsoft.Extensions.Identity.Core/UserManager.cs#L2342
Current: https://github.com/aspnet/Identity/blob/dev/src/Core/UserManager.cs#L2414

Found it. https://github.com/aspnet/Identity/pull/1714


Much faster FormatStringEncoding

Description

Loading a large pose estimation data file with a simple "{id}–{label}" format (using the napari-deeplabcut plugin; resulting feature table with >6M rows and four columns) lasted 3+ min.
While I first thought it had something to do with rendering the keypoints, a bit of profiling (see pink lines below) indicated that 92% of the time it took to load the annotations (177 s!) was spent in the TextManager, and specifically in _get_feature_row.

I substituted df.iloc with df.itertuples, and loading now takes only roughly 5 s (~35x speedup).
Type of change

[x] New feature (non-breaking change which adds functionality)

References

How has this been tested?

[ ] example: the test suite for my feature covers cases x, y, and z
[x] example: all tests pass with my change
[x] example: I check if my changes works with both PySide and PyQt backends
as there are small differences between the two Qt bindings.

Final checklist:

[x] My PR is the minimum possible work for the desired functionality
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x]  I have added tests that prove my fix is effective or that my feature works
[x] If I included new strings, I have used trans. to make them localizable.
For more information see our translations guide.

Also, if you have the profile data or even just a screenshot of snakeviz for the optimized version, that would be ideal. I'm curious what takes up the rest of that 5s.

I was unclear above; the 5 s are spent in the TextManager. The ColorManager asks for 9 other seconds, but this may be something I have to improve at the level of the plugin; I'll give it a look.

I was unclear above; the 5 s are spent in the TextManager. The ColorManager asks for 9 other seconds, but this may be something I have to improve at the level of the plugin; I'll give it a look.

Thanks for clarifying. The optimization here should just go in as is. There was an effort to replace ColorManager which is currently on hold, but if we pick it back up, we might be able to look for an optimization. You should also feel free to look for one too.

Also, FYI, I ran the relevant ASV benchmark for the format string case and we also see a big speedup there. At least 10x on my machine, though we only go up to 65536 (2^16) elements and timings are not linear.
(napari-dev) ➜  napari git:(pr/jeylau/5315) ✗ asv run --python=same --bench "TextManagerSuite.time_create"
· Discovering benchmarks
· Running 1 total benchmarks (1 commits * 1 environments * 1 benchmarks)
[  0.00%] ·· Benchmarking existing-py_Users_asweet_software_miniconda3_envs_napari-dev_bin_python
[ 50.00%] ··· Running (benchmark_text_manager.TextManagerSuite.time_create--).
[100.00%] ··· benchmark_text_manager.TextManagerSuite.time_create                                                                                ok
[100.00%] ··· ======= =========================================
              --                        string
              ------- -----------------------------------------
                 n     {string_property}: {float_property:.2f}
              ======= =========================================
                 16                    829±50μs
                 64                    855±20μs
                256                  1.14±0.06ms
                1024                 2.23±0.05ms
                4096                 6.15±0.06ms
               16384                  21.5±0.2ms
               65536                  84.9±0.9ms
              ======= =========================================

(napari-dev) ➜  napari git:(pr/jeylau/5315) ✗ git switch main
M	napari/benchmarks/benchmark_text_manager.py
Switched to branch 'main'
Your branch is ahead of 'origin/main' by 4 commits.
  (use "git push" to publish your local commits)
(napari-dev) ➜  napari git:(main) ✗ asv run --python=same --bench "TextManagerSuite.time_create"
· Discovering benchmarks
· Running 1 total benchmarks (1 commits * 1 environments * 1 benchmarks)
[  0.00%] ·· Benchmarking existing-py_Users_asweet_software_miniconda3_envs_napari-dev_bin_python
[ 50.00%] ··· Running (benchmark_text_manager.TextManagerSuite.time_create--).
[100.00%] ··· benchmark_text_manager.TextManagerSuite.time_create                                                                                ok
[100.00%] ··· ======= =========================================
              --                        string
              ------- -----------------------------------------
                 n     {string_property}: {float_property:.2f}
              ======= =========================================
                 16                   1.04±0.1ms
                 64                  1.87±0.08ms
                256                   5.81±0.6ms
                1024                   19.9±2ms
                4096                   78.9±8ms
               16384                   290±7ms
               65536                  1.15±0.01s
              ======= =========================================

Sweet!

I'll merge this after 48 hours unless anyone objects.

The functionality is very similar to pd.Dataframe.to_dict("records"), it does essentially the same thing with a few extra checks.
This PR is still 2x faster than to_dict("records").
This PR:
[100.00%] ··· ======= =========================================
              --
              ------- -----------------------------------------
                 n     {string_property}: {float_property:.2f}
              ======= =========================================
                 16                    676±40μs
                 64                    864±80μs
                256                  1.02±0.05ms
                1024                  2.00±0.1ms
                4096                  5.67±0.1ms
               16384                  19.6±0.2ms
               65536                   78.6±2ms
              ======= =========================================

with values = [self.format.format(**row) for row in features.to_dict("records")]
[100.00%] ··· ======= =========================================
             --
             ------- -----------------------------------------
                n     {string_property}: {float_property:.2f}
             ======= =========================================
                16                    759±30μs
                64                    803±30μs
               256                  1.17±0.05ms
               1024                 2.90±0.03ms
               4096                  9.54±0.3ms
              16384                  34.4±0.4ms
              65536                   144±4ms
             ======= =========================================

@JoOkuma, and df.itertuples memory footprint will also be very low, as it returns an iterator.

Hi! I came across this PR due to the cuDF mention. Performance gains look fantastic with this change!
I wanted to add some context about cuDF and itertuples:

Previously we needed the dataframe to define iloc and support len, but now we need itertuples. I checked dask's and cudf's dataframe APIs and they both provide itertuples, so there's no problem here.

cuDF doesn't support this kind of row based iteration via itertuples or itterrows. Iterating one row at a time from raw Python will cause a GPU->CPU transfer of one row at time, which is very inefficient due to the transfer overhead. If there's interest in cuDF support in the future, it may be fine to jump back and forth between cuDF and pandas here, as one bulk transfer of pdf = gdf.to_pandas(); ...; gdf = cudf.from_pandas(...) will not be as slow.
I also wanted to mention that cuDF is adding support for DataFrame.to_dict, which will essentially do the pandas conversion for you and then call pd.DataFrame.to_dict. to_dict provides a similar "dictionary of every row in a list" output.
import cudf

df = cudf.datasets.randomdata(nrows=3)
pdf = df.to_pandas()
print(pdf.to_dict(orient="records"))

# from the function in the PR
feature_names = pdf.columns.to_list()
print([dict(zip(feature_names, row)) for row in pdf.itertuples(index=False, name=None)])
[{'id': 1023, 'x': -0.3437607026238143, 'y': 0.4419788553645101}, {'id': 1012, 'x': 0.2760312523846038, 'y': 0.8273451449034162}, {'id': 1013, 'x': -0.35755297004454434, 'y': -0.13542889747873632}]
[{'id': 1023, 'x': -0.3437607026238143, 'y': 0.4419788553645101}, {'id': 1012, 'x': 0.2760312523846038, 'y': 0.8273451449034162}, {'id': 1013, 'x': -0.35755297004454434, 'y': -0.13542889747873632}]

Happy to chat further if helpful.

cuDF doesn't support this kind of row based iteration via itertuples or itterrows. Iterating one row at a time from raw Python will cause a GPU->CPU transfer of one row at time, which is very inefficient due to the transfer overhead. If there's interest in cuDF support in the future, it may be fine to jump back and forth between cuDF and pandas here, as one bulk transfer of pdf = gdf.to_pandas(); ...; gdf = cudf.from_pandas(...) will not be as slow.

Thanks for the very useful information!
napari doesn't yet support different types of tables/dataframes, but we can at least imagine that it could in the not too distant future. Will definitely reference this information then and may pick your brains more. I imagine support would look something like defining a core functional protocol and maybe do some specialized implementation for specific types (e.g. here doing something special if it's a cuDF dataframe).

FYI RAPIDS viz works closely with HoloViews and have done a similar cuDF implementations with success, for  example.

Merge after 24 but before 48 hours since we have multiple approvals here.


Allow consumers to specify a host when using knStartServer()

It would be nice if a value could be passed for the hostname parameter to Restify's .listen() to specify what address to bind to. If nothing is passed, it should default to the same value Restify uses, ::, the unspecified address.

Integrated.


[Fix How to Use Section] - add a "How to Use" section to the README.md

This PR fixes #325

Suggestions on how to improve formatting, content included, excluded in both changed files and how to enhance if need be are welcome!

Have you ran pre-commit run --all
There is a linting issue that is likely a minor markdown issue.

You can run pre-commit run --all-files

@kjaymiller @oleksis Thank you both for the feedback, having WiFi connectivity issues, will fix as soon when bandwidth becomes consistent.