[ENH] MultiplexTransformer Discussion

During the community meetup today @aiwalter and @indinewton suggested it would be useful to create this issue to discuss the implementation of a  MultiplexTransformer.  After getting a chance to check out OptionalPassthrough I think we might not need build this thing at all (I agree it could be a bad idea to have multiple areas of the code base with such overlapping/similar functionality).  But - it did leave me with some questions/ideas about some potential use cases that might be worth expanding the current model to support:
Some Background:

The original idea  to make a MultiplexTransformer came from a suggestion in #2459 that it would be nice to create a MultiplexTransformer to complement the MultiplexForecaster.
As of #2612 we have implemented a _DelegatedTransformer to facilitate implementation of MultiplexTransformer in a similar way to how MultiplexForecaster has been written.
As was mentioned in the community meeting however - it seems like the OptionalPassthrough transformer can already do most of what we would want to do with the MultiplexTransformer though performing a grid search over its passthrough variable.  (A solution which I was not aware of, thank you for bringing it up!)

Some remaining questions/ideas I had:

Question 1 - What if we want to use exactly 1 Transformer from a list of n transformers?.  It seems to me with the current system if we did something like this (borrowed from the docstring example for OptionalPassthrough):

pipe = TransformedTargetForecaster(steps=[
    ("deseasonalizer", OptionalPassthrough(Deseasonalizer())),
    ("scaler", OptionalPassthrough(TabularToSeriesAdaptor(StandardScaler()))),
    ("forecaster", NaiveForecaster())])
# putting it all together in a grid search
cv = SlidingWindowSplitter(
    initial_window=60,
    window_length=24,
    start_with_window=True,
    step_length=48)
param_grid = {
    "deseasonalizer__passthrough" : [True, False],
    "scaler__transformer__transformer__with_mean": [True, False],
    "scaler__passthrough" : [True, False],
    "forecaster__strategy": ["drift", "mean", "last"]}
gscv = ForecastingGridSearchCV(
    forecaster=pipe,
    param_grid=param_grid,
    cv=cv,
    n_jobs=-1)
gscv_fitted = gscv.fit(load_airline())

Then we could actually end up with a situation where we either skip both deseasonalizer and scaler or include them both.  One might imagine it could be nice in some situations to specify that we want exactly one transformer from a list of transformers. Having a MultiplexTransformer is one way to achieve this (I didn't see a good way to do this with the current system), though I imagine it could also be supported by making some modifications to param_grid (ie could we specify pairs of variables simultaneously).  Personally I like the second idea, but I am not sure that it is the "best solution"

Question 2: What is the best way to support "multiplexing" over pairs of transformers/forecasters simultaneously?  (eg, maybe some transformations are needed for certain forecasters).  The way that springs to mind would be to create a separate ForecastingPipeline or a TransformedTargetForecaster for each pair, and then put all the pairs into a MultiplexForecaster?   Would it be worth building a cleaner work around? (a similar question/solution springs to mind in the case where we had a list of known transformations to use but we wanted to select the optimal ordering ie trans_a -> trans_b vs trans_b -> trans_a)

What should be resolved:

[ ] Identify any meaningful use cases that either a) something like a MultiplexTransformer could uniquely support or b) that are worth expanding the current OptionalPassthrough approach to support as well?
[ ] Should we clean up (ie delete) the DelegatedTransformer code if we don't need it?

Lastly - I wonder if there is some way we can more easily inform people of the existence of OptionalPassthrough?  (Clearly people know about it!!  But I had missed it (which may of course just be on me)!  Thoughts on a Documentation page which has all the various Pipeline/model selection components? (and related dunders?))

Tagging related issue: https://github.com/alan-turing-institute/sktime/issues/2603

As was mentioned in the community meeting however - it seems like the OptionalPassthrough transformer can already do most of what we would want to do with the MultiplexTransformer though performing a grid search over its passthrough variable. (A solution which I was not aware of, thank you for bringing it up!)

I was aware! I faintly remember that I may have said, written, or thought that OptionalPassthrough is, content-wise, equivalent to a MultiplexTransformer with the transformer that is the parameter, and an IdentityTransformer. So, the OptionalPassthrough would be the special case, the other way round I think it does not work.
Although I´m not sure whether implementing it that way the best solution, it seems to me the two transformers - multiplexer, and OptionalPassthrough - are not obvious special cases of each other, interface-wise. It might still be good to have them implemented separately.

it seems like the OptionalPassthrough transformer can already do most of what we would want to do with the MultiplexTransformer though performing a grid search over its passthrough variable. (A solution which I was not aware of, thank you for bringing it up!)

@miraep8, @aiwalter, can you explain how this would look like mathematically?
This does not seem correct to me. I do not think we can easily replace an instance of the MultiplexTransformer with OptionalPassthrough.
How would this work, formally? Can you give an example, let´s say, how we would replace a multiplexer over three transformers, A, B, C?

The way that springs to mind would be to create a separate ForecastingPipeline or a TransformedTargetForecaster for each pair, and then put all the pairs into a MultiplexForecaster?

That's how I would do it. I.e., (A * B) | (C * D), where A, C are transformers, and B, D are forecasters. The dunder syntax would invoke the MultiplexForecaster, not the MultiplexTransformer, since the pipelines A * B and C * D are forecasters.

Identify any meaningful use cases that either a) something like a MultiplexTransformer could uniquely support or b) that are worth expanding the current OptionalPassthrough approach to support as well?

I think that already the simplest case, i.e., A | B , cannot be captured equivalently by a composition that involves only OptionalPassthrough.
(except when A or B is trivial)

Should we clean up (ie delete) the DelegatedTransformer code if we don't need it?

Yes, but I doubt that we do not need it.

Lastly - I wonder if there is some way we can more easily inform people of the existence of OptionalPassthrough? (Clearly people know about it!! But I had missed it (which may of course just be on me)! Thoughts on a Documentation page which has all the various Pipeline/model selection components? (and related dunders?))

Yes! A "Pipeline" notebook. examples/05_pipelines.ipynb where all the neat facts about pipeliens are collected in one place, and which has a list and examples with all the components and dunders!

... and all this was discussed in the 10min when I wasn't there?? You were busy, apparently...

How would this work, formally? Can you give an example, let´s say, how we would replace a multiplexer over three transformers, A, B, C?
Let's take an example (below is not the actual code but more of a pythonic way of explaining the current concepts of sktime)

transformers = [
 ("log", OptionalPassthrough(LogTransformer())),
 ("box", OptionalPassthrough(BoxCoxTransformer())),
]
# you can bundle up even upto 20 transformers like this

# lets say you have a model list
model_list = [
 ("arima", AutoArima()),
 ("ets", AutoETS()),
]
# you can also add many more forecasters in this model_list

From here, we challenged that the outcome and functionality of MultiplexTransformer + MultiplexForecaster
will be same as what is achieved below
best_forecaster = []
for model in model_list:
    pipe = transformers + model
    gscv_model = ForecastingGridSearchCV(forecaster = pipe, cv=cv, param_grid=param_grid)
    gscv_model.fit(y, fh)
    best_forecaster.append(gscv_model.best_forecaster_)

multiplex = MultiplexForecasters(best_forecasters, fh)
multiplex.fit(y, fh)

multiplex.predict(fh)

So having a class of MultiplexTransformer allows user to save this 5 to 6 lines of code. Which is why we were wondering what is the additional value of this besides convenience for the user.

@indinewton, can you kindly be more precise, and specify two estimators that would be equivalent?
You are outlining a manual process and not an estimator, and it seems you are not entirely on one page with @miraep8 on how the MultiplexForecaster would be used (or what it formally does). You would be doing ForecastingGridSearchCV(MultiplexForecaster), not the other way round.

a similar question/solution springs to mind in the case where we had a list of known transformations to use but we wanted to select the optimal ordering ie trans_a -> trans_b vs trans_b -> trans_a

Hm, depends on how many, and which combinations we want to select over.
If only some combinations, I would still use the multiplexer.
If all combinations, one could create a PermutationPipeline, which takes as parameters the transformers in the chain, and an element of the permutation group over the set of transformers. Although that feels impractical for anything except a small number of elements, due to combinatorial explosion.

Question 1 - What if we want to use exactly 1 Transformer from a list of n transformers?. It seems to me with the current system if we did something like this (borrowed from the docstring example for OptionalPassthrough)

@miraep8 Actually this is currently not possible in sktime. Adding MultiplexTransformer will probably solve this, you are right.

Then we could actually end up with a situation where we either skip both deseasonalizer and scaler or include them both. One might imagine it could be nice in some situations to specify that we want exactly one transformer from a list of transformers. Having a MultiplexTransformer is one way to achieve this...

Generally when we are building forecasting applications, we expect GridSearchCV to find answers for us. So if the answer would be deaseasonlizer true and scaler false, then that would be detected through GSCV. But what if user just wants to pick only one transformer and not more? That's not possible to control as of current sktime classes. So MultiplexTransformer will be helpful there.
Additionally after reading through rapidfire comments from @fkiraly, I also agree with below -

This claim does not seem correct to me. I do not think we can easily replace an instance of the MultiplexTransformer with OptionalPassthrough.

MultiplexTransformer will have more control over returned Transformers by controlling some parameter like return_n_best_transformers = 2. With OptionalPassthrough you can only benefit it when you wrap it in ForecastingGridSearchCV. So @fkiraly is right, actually. It seems it makes to add this in Sktime. So, the conceptual example I commented earlier, seems a bit out of context.

a similar question/solution springs to mind in the case where we had a list of known transformations to use but we wanted to select the optimal ordering ie trans_a -> trans_b vs trans_b -> trans_a

Hm, depends on how many, and which combinations we want to select over. If only some combinations, I would still use the multiplexer.
If all combinations, one could create a PermutationPipeline, which takes as parameters the transformers in the chain, and an element of the permutation group over the set of transformers. Although that feels impractical for anything except a small number of elements, due to combinatorial explosion.

I also agree with what has been suggested here. Additionally transformers can get really messed up if orders are randomly permuted. For example apply MinMaxScaler before a transformer which might be doing divisions or logs (LogTransformer), because minmax guarantees presence of 0 in the transformed data. Additionally, in pipeline that we have built for customers, the order is carefully decided after reading through all the transformers in sktime and sklearn. An example of such a order is: HampelFilter() should be first -> immediately followed by Imputer() ->  other transformers -> (at the end) MinMaxScaler (Obviously passed to TabularToSeriesAdaptor)

can you explain how this would look like mathematically?
This claim does not seem correct to me. I do not think we can easily replace an instance of the MultiplexTransformer with OptionalPassthrough.

@fkiraly - I 100% agree that the OptionalPassthrough cannot fully replace what MultiplexTransformer could do in its current state- Question 1 above was meant to be one such highlight of this issue (which I think is the more general case of the A | B example that you mentioned).
My point was really - should we either a) change how we handle OptionalPassthrough (which I think we could do) so that it can cover more of the general cases or b) just write the MultiplexTransformer.
I do think we could make OptionalPassthrough behave more like MultiplexTransformer if we changed how we did the grid search. As I mentioned above briefly one could imagine specifying pairs of variables simultaneously (ie we do the grid search over (deseasonalizer, scaler) = [(True, False), (False, True)] (not that the syntax would necessarily look like that, but hopefully you get the gist).
Would MultiplexTransformer read better than that^.......  I think so. But I guess I was trying to strike a balance between creating the best solution and one that works within the code base we already have.
Perhaps in the end we can just have both and that's ok/my concerns that it will be confusing are unmerited.

Yes! A "Pipeline" notebook. examples/05_pipelines.ipynb where all the neat facts about pipelines are collected in one place, and which has a list and examples with all the components and dunders!

Sounds like a good idea!  Haha, I actually went to look for it first without seeing your note it was not written yet and was disappointed. Might I suggest that when we get around to it we also write a tutorial or similar on the docs site??  It just seems like too important a feature in sktime for users to struggle to have to piece together on their own.

(Question 2 was meant to demonstrate another case that would be easier to solve with MultiplexTransformer than with OptionalPassthrough I realize now perhaps I should have called them cases not questions)

However - I don't want to discount my suggestion about generalizing the GridSearchCV param_grid entirely as I think it could also unlock some interesting functionality/abilities to pair/have more control over what space we are searching over.  Which might be really powerful! :) But also might be really confusing/more of a mess than its worth.
Just to reiterate, the real question hidden in the block of text above is:

can we make OptionalPassthrough sufficiently general by making the param_grid argument of GridSearchCV more flexible (and is that worth it even if we could do that) or should we just write MultiplexTransformer and be ok with having both.

Might I suggest that when we get around to it we also write a tutorial or similar on the docs site?? It just seems like too important a feature in sktime for users to struggle to have to piece together on their own.

Depends on whom you mean with "we", is that that the Royal "we"? 😁

I do think we could make OptionalPassthrough behave more like MultiplexTransformer if we changed how we did the grid search.

That does not sound like a good idea, since I feel it would make both the compositors and the grid search more specific to each other, hence less generic. This sounds like an instance of "conceptual diffusion", where you blur the line between multiple otherwise very clear concepts. You would essentially get objects that are "both" passthrough and multiplexer and a bit of grid search maybe.
I agree with you that this would not be pretty, and I would advise to go with the design that is conceptually clearer (the original one). The "formal" design principle is separation of concerns: https://en.wikipedia.org/wiki/Separation_of_concerns
If possible, an object should do passthrough or multiplexing or grid search but not all of it, since these are concerns that are clearly conceptually separated on the formal level (in my opinion).

entirely as I think it could also unlock some interesting functionality/abilities to pair/have more control over what space we are searching over

In the grid search, you would still be specifying the same grid, no matter whether you merge the inner components or not? Or am I not seeing this correctly?

can we make OptionalPassthrough sufficiently general by making the param_grid argument of GridSearchCV more flexible (and is that worth it even if we could do that) or should we just write MultiplexTransformer and be ok with having both.

I think we can, but I also think it´s a bad idea, since it scores lower on "separation of concerns" and "conceptual clarity" imo.

So - main takeaways - it seems we are all on board with going with implementing  MultiplexTransformer as originally planned.  Unless anyone voices additional concerns, I will proceed with the MultiplexTransformer implementation as planned 👍

entirely as I think it could also unlock some interesting functionality/abilities to pair/have more control over what space we are searching over

in the grid search, you would still be specifying the same grid, no matter whether you merge the inner components or not? Or am I not seeing this correctly?

If I understand your question - then no, the explored parameter space in the "merged" version would actually be smaller.   For example - in the example I gave above (ie the example I gave of  (deseasonalizer, scaler) = [(True, False), (False, True)])) the space of parameter combinations we are exploring (only looking at deseasonalizer and scaler) would be {(True, False), (False, True)} whereas in the original "unmerged" component we would evaluate {(True, True), (True, False), (False, True) (False, False)}.
In that^ example the artificial | we are effectively creating can of course be accomplished with the MultiplexTransformer instead, which as we have agreed is a "prettier solution".  The reason I said we might not want to discount it entirely though is that there might be other use cases where having the flexibility to manually reduce the space of combinations we are considering is that a) it might help reduce the computation costs of a potentially very complex pipeline with lots of tunable parameters and b) help us avoid exploring parameter combinations that we might expect to be a poor fit for the model.  We could even provide classes that could be passed to grid search cv to facilitate this smaller search space - eg the example above could be generalized to a "pick one of n" function which was provided with a list of transformers.
But thinking on it more the idea might end up having more complexity than its worth! :)

For example - in the example I gave above (ie the example I gave of (deseasonalizer, scaler) = [(True, False), (False, True)])) the space of parameter combinations we are exploring (only looking at deseasonalizer and scaler) would be {(True, False), (False, True)} whereas in the original "unmerged" component we would evaluate {(True, True), (True, False), (False, True) (False, False)}.

I don't see that, or I don't see why you are not comparing like with like.
Can we not specify both the Cartesian product grid [True, False] x [True, False] and the "exactly one" grid [(True, False), (False, True)] in both cases? It's just a matter of the param grid to pass, the parameter space seems to be the same?

Unless anyone voices additional concerns

No objections.

In my second example I was trying to suggest that we not explore full grid, but rather a reduced parameter space. (Which now that I say it like that does seem like it would be confusing for users to have in "gridSearchCV"  - but if we liked it would could make a more general "paramSearchCV" or something like that).
Hence the emphasis is on the set of parameter combinations one could explore for the two parameters deseasonalize and scaler.
In the original example, because grid search does enforce a Cartesian product we have:
{(True, True), (True, False) (False, True), (False, False)}
In the second I was suggesting we given the user the option to define a reduced parameter space to explore:
{(True, False) (False, True)}
Hence they could recreate the "effective or"

I am not necessarily arguing we need to do things this way, and I don't think GridSearchCV would be the place to do them if we did want to.  I am also not sure how much the user would benefit from this degree of control.  Obviously we plan to have a better way to achieve what this example was trying to do (MultiplexTransformer), but I was wondering if there might be other conditions where such a capability might be useful.

In the original example, because grid search does enforce a Cartesian product

Ah, I think that's where we are not on one page.
Grid search does not necessarily optimize over a Cartesian product, see for instance the example here:
https://scikit-learn.org/stable/modules/grid_search.html#exhaustive-grid-search
Arbitrary sums of grids (Carteisan products) are part of common formulations.

Nice!  Good to know!


TextInput secureTextEntry color bug

[x] Review the documentation: https://facebook.github.io/react-native
[x] Search for existing issues: https://github.com/facebook/react-native/issues
[ ] Use the latest React Native release: https://github.com/facebook/react-native/releases

Environment
React Native Environment Info:
System:
OS: macOS 10.14.2
CPU: x64 Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
Memory: 120.61 MB / 16.00 GB
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 10.13.0 - ~/.nvm/versions/node/v10.13.0/bin/node
Yarn: 1.9.4 - /usr/local/bin/yarn
npm: 6.4.1 - ~/.nvm/versions/node/v10.13.0/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
SDKs:
iOS SDK:
Platforms: iOS 12.1, macOS 10.14, tvOS 12.1, watchOS 5.1
IDEs:
Android Studio: 3.1 AI-173.4907809
Xcode: 10.1/10B61 - /usr/bin/xcodebuild
npmPackages:
react: 16.5.0 => 16.5.0
react-native: ^0.57.2 => 0.57.2
npmGlobalPackages:
react-native-cli: 2.0.1
Description
I have a form with many textinputs, onFocus they have one color and onBlur another. All work fine until I add property secureTextEntry={true} to the field. After that the color doesn't change anymore.
Reproducible Demo
<TextInput
ref={this.input}
secureTextEntry
style={this.state.focusStyle ? {color: '#fff'} : {color: '#aaa'}}
onFocus={() => {
this.setState({
focusStyle: true
})
// this.input.current.setNativeProps({
//   style: {
//     color: '#fff'
//   }
// })
}}
onBlur={() => {
this.setState({
focusStyle: false
})
// this.input.current.setNativeProps({
//   style: {
//     color: '#aaa'
//   }
// })
}}
/>

I ran into the same problem

Could someone please help with this issue? This is relevant one

i've gotten same issue. Could someone helps with that?
@react-native-bot

I had the same problem. Could someone please help with that?

Encountering this issue on RN-0.57.7

@zaporozhetsAnton Did this issue happen on both platforms? It happened on ios only for me on RN-0.57.7

@peterdev6 I did not check it on the android platform

Just encountered this bug.
Confirmed on iOS with RN 0.58.4

Fixed by #23524

Still an issue in RN 0.59.5

Still an issue in RN 59.10


Instantiated types for more schema flexibility

I've been working on trying to use more keywords from the JSON Schema dialect, such as minimum, maximum, and required. I wasn't able to get shale's JSON mapping to work which makes sense if it's just for serializing/deserializing, so as of now I've thought of three possible approaches
1. Custom classes for every possibility
I think this is what developers would have to do right now to use more of the dialect. To have an integer with a minimum value of 1 might look like this:
  class IntegerMinimumOneType < Shale::Type::Integer
  end

  class IntegerMinimumOneJSONType < Shale::Schema::JSONGenerator::Base
    def as_type
      { "type" => "integer", "minimum" => 1 }
    end
  end

  Shale::Schema::JSONGenerator.register_json_type(IntegerMinimumOneType, IntegerMinimumOneJSONType)

  class PersonMapper < Shale::Mapper
    model Person
    attribute :age, IntegerMinimumOneType
  end

This would be fine for common things, like PositiveInteger, but this would get increasingly difficult to manage if the schema needed multiple validations for properties, such as a minimum, maximum, and required all at once.
2. Instantiated types for more flexibility
This isn't possible with shale right now, but I have a local branch where I've gotten it to work and would be happy to make a PR if this is a direction you'd like to go in. To achieve the same schema as above the API could look like this:
  class BoundedIntegerType < Shale::Type::Integer
    attr_reader :min, :max

    def initialize(min = nil, max = nil)
      @min = min
      @max = max
    end
  end

  class BoundedIntegerJSONType < Shale::Schema::JSONGenerator::Base
    def as_type
      { "type" => "integer", "minimum" => instance.min, "maximum": instance.max }.compact
    end
  end

  Shale::Schema::JSONGenerator.register_json_type(BoundedIntegerType, BoundedIntegerJSONType)

  class PersonMapper < Shale::Mapper
    model Person
    attribute :age, BoundedIntegerType.new(1) # minimum of 1, no maximum
  end

The responsibility is still on the consumer of the gem to create the types needed for their schema, but having a generator that can receive an instance of a class can allow for a lot of flexibility.
3. Modify the API to include more options, similar to collection
Another option to allow more of the dialect to be used is to allow the attribute API to receive all the possible keywords, either individually or through some sort of hash.
  class PersonMapper < Shale::Mapper
    model Person
    attribute :age, Shale::Type::Integer, minimum: 1
  end

This might involve some more work on the shale side of things to ensure that the provided keywords are compatible with the type. For example this should be considered invalid since maxItems is for arrays:
    attribute :age, Shale::Type::Integer, max_items: 5

Apologies if I've overlooked something, but based on reading the docs and trying things out I think right now Option 1 is all I can do, so I wanted to explore Option 2 since it seemed less cumbersome. It also seemed like implementing Option 3 would add a lot more complexity to the gem. Should I go ahead and make a PR for this, or do you have guidance on how to achieve this another way?

Hey, I'd happily accept a PR from you. Option 3 is something I'd like to see, although with some modifications.
Since this change relates to Schema I'd prefer extra options be defined on schema definition instead of on attributes. This way we can have different options per schema. It will aslo be more consistent, since only JSON and XML support schemas.
This is an example I have in mind:
class Person < Shale::Mapper
  attribute :id, Shale::Type::Integer
  attribute :first_name, Shale::Type::String

  json do
    map 'id', to: :id, minimum: 1
    map 'first_name', to: :first_name, maximum: 5
  end

  xml do
    map_element 'Id', to: :id
    map_element 'FirstName', to: :first_name
  end
end


Overflow on PDFs with alternating page sizes

Attach (recommended) or Link to PDF file here:
testFile.pdf
Configuration:

Web browser and its version: Chrome: 111.0.5563.110
Operating system and its version: Windows 10 Pro 22H2
PDF.js version: 2.14.305 (via ng2-pdf-viewer 9.1.2)
Is a browser extension: ?? no

Steps to reproduce the problem:

Open dev tools, set width: 40% on
Upload given example File to https://mozilla.github.io/pdf.js/web/viewer.html
Scroll down

What is the expected behavior? (add screenshot)
Last PDF page is rendered correctly, zoom of other pages are adjusted accordingly

What went wrong? (add screenshot)
Last PDF page overflows as it wasn't taken into account when determining size and scale of PDFs

Issue originated from:
https://github.com/VadimDez/ng2-pdf-viewer/issues/988#issuecomment-1481536895
One Workaround (funnelled through the angular library) is to track canvas width on each page rendered event and adjust zoom factor if page breaks allocated width, but this leads to a visual jump every time the zoom factor gets updated.
Alternatively you can obscure the view, scrollTop() every page, recalculate the zoom, apply, scroll up again and lift the curtain (but this takes a ridiculous amount of time)
What would help here (if pdf.js can't account for 'surprises' later down the pages) would be an array of page dimensions on load completion. We already know how many pages there aught to be. Maybe there is a way to find out their sizes relative to each other in advance?

Duplicate of #5628


Redis Client Connection Errors

I've tried a few Redis clients and they are reporting connection errors, so I believe it's how I'm configuring them.
From the Vercel Storage tab / quickstart info (CLI/cURL/fetch/etc) auth info, what do I utilize in a database client like Redis Insight to get a UI / real time monitor app working?

Hey there @jb-dev1, I just gave a try to Redis Insight and this is how I configured it (it worked) 👇

Hope this helps 👍

You're a champion, thank you. I had forgotten about the TLS requirement, and the Vercel KV response messages were not meaningful (to me anyway).


Error Updating Chroot

Hello there,
Thanks to Denny L. for guiding me.
Please paste the output of the following command here: sudo edit-chroot -all

Command modified since using 32 GB SanDisk USB Drive:

crosh> shell
chronos@localhost / $ sudo edit-chroot -all -c "/media/removable/USB Drive/chroots/"
name: xenial
encrypted: no
Entering /media/removable/USB Drive/chroots/xenial...
crouton: version 1-20180329190032~master:d287904a
release: xenial
architecture: amd64
xmethod: xiwi
targets: xiwi,xorg,xfce-desktop,cli-extra,keyboard,chrome
host: version 10754.0.0 (Official Build) canary-channel wolf
kernel: Linux localhost 3.8.11 #1 SMP Tue Jun 5 04:16:38 PDT 2018 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Not unmounting /media/removable/USB Drive/chroots/xenial as another instance is using it.
chronos@localhost / $

I have Crouton open while executing command above.

Please describe your issue:
Trying to update chroot, spits out following error.

Chrome OS is on the Canary Channel, Version: 69.0.3449.0 (Official Build) canary (64-bit)
If known, describe the steps to reproduce the issue:

USB Drive w/ Crouton
Attempt updating chroot
Error

Let me know if I can do anything else for you, if more info. is required.
-Camdyn-

Closing, please track on issue: https://github.com/dnschneid/crouton/issues/3811


Shouldn't doc advise for named volume instead of hosted volume ?

Hi,
I noticed your advice about persisting data for portainer. But There is a simpler and I believe cleaner way to do so (but less "secure" I guess). It is to use named volumes. So instead of docker run -d -p 9000:9000 -v /path/on/host/data:/data portainer/portainer I personally use docker run -d -p 9000:9000 -v portainer_data:/data portainer/portainer.
I don't know if you're not advising for it on purpose or just because you missed that feature (I believe it is fairly new but I didn't find the docker version that introduced that feature).
Cheers :)

It's not really a matter of security, it's just that using a bind mount on the host is simply clearer as the user can check where the data is (it's also helpful for us when doing some support). Though, there is multiple ways to persist data using Docker, you might pick the one you prefer :-)
I'll close this issue for now as I don't think there is a real advantage of using a named volume, and if anybody want to persist data using named volumes & different drivers, they can choose to go for it !
Feel free to comment or re-open if you think that I'm missing something.


Extend Tracing for zk EVMs

Description
Currently the tracing does not provide all the necessary information necessary to create zk-proofs for the transaction executing. Specifically the following two cases are currently not covered by tracing:

when doing a CREATE(2) / CALL-type instruction we would like to defer an action to the point in time right after the child context re-enters the current context before the execution of the next opcode in the current context
for many opcodes that can change the state σ and/or the accrued state A we would need to hook into the point in time right after acting upon a REVERT instruction or an exceptional halt (so that the rollback has already happened at that point in time, but the next opcode, if any, hasn't started)

Open questions:

Which data should be provided by the tracer. Is the MessageFrame enough?
How can we best test the 2. case, as the revert can happen within nested calls?

Which data should be provided by the tracer. Is the MessageFrame enough?

We will need

accrued state (account and storage warmths, maybe selfDestructSet at that point) post rollback i.e. if a REVERT or exception causes some warmth to go back to being cold we need this updated warmth
the machine state (gas, stack , ...) which is in the MessageFrame
the current world state (nonces, balances, codes, storage etc ....)
execution context data (likely available through the MessageFrame or the Hub)

Note. The world state may be available through messageFrame.getWorldUpdater() though I'm not really clear on whether the updater only contains the relevant (currently valid) state deltas associated with the current state of the execution or if it contains the whole (currently valid) updated state.
WRT the hooks
We've implemented a rollbackDefer mechanism that allows us to schedule actions now for later execution (and modifying the relevant trace section). What we haven't done is to connect it to a Besu hook so far. Ideally I would like to request some time from someone on the Besu team to go over the existing hooks. In particular any hook that activates at context "re-entry" such as traceContextReEntry to see if that hook already fulfils the requirements (all changes that may be rolled back were already rolled back.) And similarly for hooks triggering just before exiting a context (traceContextExit).
I'll add a quote from issue #816

The TraceRollback hook should be triggered on rollbacks at the precise point in time when Besu finishes the rollback (in particular of the state (balances, nonces, storage ...) and accrued state variables such as address/storage warmth ...) and before any potential new opcode is acted upon in the parent context. For the root context, Besu hook should be triggered also at transaction end in case it exits with status code 0, similarly: when all rollbacks were acted upon, though before gas/value reimbursement to the sender/recipient and coinbase reward. These "transaction finalization" actions are a separate thing and this hook should appear before them.

@daniellehrner where did we leave this off? I believe we were waiting for the HUB refactor

Not needed anymore


Unable to cross-compile cairo-rs from macOS to Windows

I was unsuccessful at cross-compiling from macOS to Windows when using cairo-rs. I attempted to follow what I could from this document: https://github.com/gtk-rs/gtk-rs.github.io/blob/1dbd8c3ded7e83910426bb4af10d312c85a957be/docs-src/tutorial/cross.md.
I ran the following command: PKG_CONFIG_ALLOW_CROSS=1 cargo build --target=x86_64-pc-windows-gnu --release
Error log is in this github gist: https://gist.github.com/flowjo-isaach/165d7f9cb808f79c1092c0291d58bc78
Any help would be greatly appreciated!
.cargo/config:
[target.x86_64-pc-windows-gnu]
linker = "x86_64-w64-mingw32-gcc"
ar = "x86_64-w64-mingw32-gcc-ar"

main.rs:
use cairo_rs::{Context, Format, ImageSurface};

fn main() {
    let surface = ImageSurface::create(Format::ARgb32, 256, 256).expect("Can't create surface");
    let cr = Context::new(&surface).unwrap();
    // Examples are in 1.0 x 1.0 coordinate space
    cr.scale(1.0, 1.0);

    // Drawing code goes here
    cr.set_line_width(1.);
    cr.set_source_rgb(0.0, 0.0, 0.0);
    cr.rectangle(64.5, 64.5, 128., 128.);
    cr.stroke();

    let mut file = File::create("file.png").expect("Couldn't create 'file.png'");
    match surface.write_to_png(&mut file) {
        Ok(_) => println!("file.png created"),
        Err(_) => println!("Error create file.png"),
    }
}

Can you provide the full cargo build -vvvvvv output from a clean build?

@sdroege : sure, here's the clean build:
command: cargo clean && cargo build --target=x86_64-pc-windows-gnu --release
output: https://gist.github.com/flowjo-isaach/607e6379b90434f9be0c3ac1c943bc56

You forgot -vvvvv :)

my bad 😓
command: cargo clean && cargo build -vvvvv --target=x86_64-pc-windows-gnu --release
output: https://gist.github.com/flowjo-isaach/f2302c424c8af12e702e432e82c7ab09

This is the relevant code here: https://github.com/rust-lang/pkg-config-rs/blob/master/src/lib.rs#L340-L352
You need to configure your environment correctly for cross-compilation with pkg-config.

hi, can you provide a simple configuration method? after some attempts, i build with --target=i686-pc-windows-gnu by mingw-w64 on linux, I couldn't configure it successfully.


esp-tls fails to allocate even though sufficient memory is available (IDFGH-11729)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v5.1.1
Espressif SoC revision.
ESP32-D0WD-V3 (revision v3.1)
Operating System used.
Windows
How did you build your project?
Other (please specify in More Information)
If you are using Windows, please specify command line type.
PowerShell
Development Kit.

Power Supply used.
USB
What is the expected behavior?
When enough memory is available, a memory allocation should not fail.
What is the actual behavior?
Setting up TLS with esp-tls causes a memory allocation to fail. According to the statistics (see the application logs below, printed from the heap_caps_register_failed_alloc_callback hook), the allocation should not fail, since only 4 bytes are being allocated, while 1803 bytes are available for the given capabilities, of which the largest block provides 128 bytes.
Steps to reproduce.
Occurs after establishing a secure connection to a server using esp-tls and not quite enough memory is available.
Debug Logs.
E (56929) esp_firmware::init: memory allocation failed
E (56934) esp_firmware::init:  - requested_size: 4 bytes
E (56939) esp_firmware::init:  - free_caps:      1803 bytes
E (56939) esp_firmware::init:  - free_internal:  38199 bytes
E (56944) esp_firmware::init:  - free_spiram:    4118348 bytes
E (56954) esp_firmware::init:  - max_free_block: 128 bytes

Backtrace:

heap_caps_alloc_failed_hook()
heap_caps_alloc_failed()
heap_caps_calloc()
esp_mbedtls_mem_calloc()
mbedtls_calloc()
mbedtls_mpi_grow()
ecp_mod_p521()
mbedtls_mpi_mul_mod()
ecp_add_mixed()
ecp_mul_restartable_internal$isra$0()
mbedtls_ecp_mul_restartable()
mbedtls_ecdh_compute_shared()
mbedtls_ecdh_calc_secret()
mbedtls_ssl_handshake_client_step()
mbedtls_ssl_handshake_step()
mbedtls_ssl_handshake()
esp_mbedtls_handshake()
esp_tls_low_level_conn()
esp_tls_conn_new_async()
... [application code]

More Information.

External RAM is enabled, but esp-tls is set to use internal memory only.
Setting esp-tls to use external RAM fixes the issue, since enough memory is available there.

Hello @thetek42,
Sorry for the delay.
In this case the requested size in this case is the size provided for the calloc
E (56934) esp_firmware::init:  - requested_size: 4 bytes

But it does not provide the number of chunks to be allocated ( n provided in the calloc function).
so due to this maybe the memory request could be for more size than 4 bytes.
Is it possible for you to add debug prints in mbedtls calloc function?
That way we would know the exact amount required for the memory allocation which is failing.

Hi @thetek42  The issue in the memory allocation information has been fixed with https://github.com/espressif/esp-idf/commit/497c840172fe2a340eb28cfa5331eadee4834641
I am closing this issue, please feel free to re-open if required


card holder to support filters

Describe the feature you'd like to have added
now we have card holders to support the different cars that determine the extraction/insertion type (fluid, item, energy, redstone e.c.t.) how about the ability to put in the different kinds of filters into the card holder, to help keep the player inventory organised as well as having all the node configuration you may need in the holder
Additional information
No response

For now I'm going to say no -- mostly because I don't want to add the card holder UI to the Card UIs. I can allow you to put the filters in the holder, but then they wouldn't be easily accessible to cards. So I'm just gonna hold off for now - since they stack it shouldn't be that hard to keep track of it.


Outdated bootstrap .less files

While the css files were updated to bootstrap version 3.3.7 the less files are still at version 3.3.5. Users who build AdminLTE will be using outdated bootstrap files.

At this point, AdminLTE only uses the variables and mixins LESS files for reference. The Bootsrap CSS files are provided separately. Therefore, the LESS files do not affect the produced CSS files.
Thanks!


_posts/0000-01-02-lostinlux.md

adding a launch comment

added note to slide 5

closing exercise


Don't run ELS on parser

There is already elasticsearch deployed as a part of parser job. No
need to run another elasticsearch on the same VM. Moreover, ELS is
configured to consume 46% of the VM memory, which in case of 2 ELSes
means that the VM runs out of memory, starts swapping heavily and
becomes unresponsive.

@mtekel: the change you are trying to undo came from this commit: https://github.com/logsearch/logsearch-boshrelease/commit/8d50e5e5933d84841eb033c76da60dc4041ffb17
We got rid of the nested ElasticSearch and changed the release so we use the same ElasticSearch job as used in the other jobs.
Could you explain what you have done that resulted in 2 ElasticSearch processes running on your parsers?

Deployed v200. I haven't checked if it was fixed since (which is was). When are you planning to make a new release?

@mtekel: deployed v200 or upgraded to v200? If it's a fresh deployment and you actually have 2 ES processes running on the parsers then it's a massive bug in the release and we will fix it ASAP once we figured out how it happened. If that's the case, would you send me your deployment manifest?

@mtekel: ah, I see it is fixed now, good to hear. We will probably cut a new release very soon.


Added Multiprocess to run_node

Co-authored-by: Samuel Sinayoko<EMAIL_ADDRESS>Description
Added multiprocess to run_node.
Related to  https://github.com/quantumblacklabs/kedro/issues/242
Development notes
#242
Checklist

[x] Read the contributing guidelines
[x] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
[x] Updated the documentation to reflect the code changes
[ ] Added a description of this change and added my name to the list of supporting contributions in the RELEASE.md file
[x] Added tests to cover my changes

Notice

[ ] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.

I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.

I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.

Nice meeting you tonight @marcelotrevisani !

Nice meeting you tonight @marcelotrevisani ! Will try and check in over the next few days to help get the PR through.

It was nice to meet you too!
I just added to your user permissions to my fork so you will be able to push changes if you want to. I just saw that the linters are complaining.
Please accept to have permissions to push on my fork
https://github.com/marcelotrevisani/kedro/invitations

@marcelotrevisani Thank you for the contribution. Could you check the legal notice box please?

Addressed in https://github.com/quantumblacklabs/kedro/commit/a6ed5e8612e2ca34b4975965b63c54acc5f06558


Setting jPlaylist.playlist from parent component

Hi guys,
Thanks for the amazing library!
I'm new in React/Redux, so I have a question about initializing playlists in runtime. How can I to do this?
JPlayer component:
import React from 'react';
import { Gui, SeekBar, BufferBar,
  Poster, Audio, Title, Mute, Play, PlayBar,
  VolumeBar, Duration, CurrentTime, Download, BrowserUnsupported,
} from 'react-jplayer';
import JPlaylist, { connect, Playlist, Shuffle, Next, Previous, Repeat,
  MediaLink, Title as PlaylistTitle } from 'react-jplaylist';

const options = {
  id: 'JPlayer',
  verticalVolume: true,
};

const jPlaylistOptions = {
  hidePlaylist: false,
  playlist: INIT_FROM_PARENT,
};

const JPlayer = () => {
  return (<JPlaylist className='jp-sleek'>
    code here...
  </JPlaylist>);
};

export default connect(JPlayer, options, jPlaylistOptions);

JPlaylist component:
import React, { Component } from 'react';
import { Provider } from 'react-redux';
import { createStore, combineReducers } from 'redux';
import { initialState as jPlayerInitialState, reducer as jPlayers } from 'react-jplayer';
import { initialState as jPlaylistInitialState, reducer as jPlaylists } from 'react-jplaylist';
import JPlayer from './JPlayer';

const store = createStore(combineReducers({
  jPlayers,
  jPlaylists,
}), {
  jPlayers: jPlayerInitialState(JPlayer),
  jPlaylists: jPlaylistInitialState(JPlayer),
}, window.__REDUX_DEVTOOLS_EXTENSION__ && window.__REDUX_DEVTOOLS_EXTENSION__());

class JPlaylist extends Component {
  constructor(props) {
    super(props);
  }

  render() {
    const playlist = this.props.tracks.map(track => {
      return {
        title: track.title,
        artist: track.performer,
        sources: {
          mp3: track.url,
        },
      }
    });
    return (

      <Provider store={store}>
        <JPlayer playlist={playlist}/>
      </Provider>

    );
  }
}

export default JPlaylist;

I need to pass this.props.tracks from parent component (any page in website) to INIT_FROM_PARENT in JPlayer
Thanks for your help!

I do not like your naming of components. It's really confusing that you are passing playlist from your JPlaylist to JPlayer components. I'd consider renaming them.

@MartinDawson thanks for your answer!
seems yes, but I see an error You must pass a component to the function returned by connect.
Here is my changes:
JPlaylistWrapper component:
import React from 'react';
import { Provider } from 'react-redux';
import { createStore, combineReducers } from 'redux';
import { initialState as jPlayerInitialState, reducer as jPlayers } from 'react-jplayer';
import { initialState as jPlaylistInitialState, reducer as jPlaylists } from 'react-jplaylist';
import JPlayer from './JPlayer';
import JPlaylist from './JPlaylist';

const store = createStore(combineReducers({
  jPlayers,
  jPlaylists,
}), {
  jPlayers: jPlayerInitialState(JPlayer),
  jPlaylists: jPlaylistInitialState(JPlayer),
}, window.__REDUX_DEVTOOLS_EXTENSION__ && window.__REDUX_DEVTOOLS_EXTENSION__());

const JPlaylistWrapper = (props) => (<Provider store={store}>

    <JPlayer/>
    <JPlaylist tracks={props.tracks}/>

</Provider>
);

export default JPlaylistWrapper;

JPlaylist:
import React, { Component } from 'react';
import { connect } from 'react-redux';
import { actions } from 'react-jplaylist';

class JPlaylist extends Component {
  constructor(props) {
    super(props);
  }

  componentDidMount() {
     const playlist = this.props.tracks;
  	// By dispatching the react-jPlaylist action we can change
  	// the jPlaylist's playlist
  	this.props.dispatch(actions.setPlaylist(idOfMyJPlaylist, playlist));
  }
}

export default connect(JPlaylist);

JPlayer:
import React from 'react';
import { Gui, SeekBar, BufferBar,
  Poster, Audio, Title, Mute, Play, PlayBar,
  VolumeBar, Duration, CurrentTime, Download, BrowserUnsupported,
} from 'react-jplayer';
import JPlaylist, { connect, Playlist, Shuffle, Next, Previous, Repeat,
  MediaLink, Title as PlaylistTitle } from 'react-jplaylist';

const JPlayer = () => (<JPlaylist className='jp-sleek'>
  JPlayer code...
</JPlaylist>);

const options = {
  id: 'JPlayer',
  verticalVolume: true,
};

const jPlaylistOptions = {
  hidePlaylist: false,
  playlist: [],
};

export default connect(JPlayer, options, jPlaylistOptions);

More details what I need:
I have BookPage component, I need to attach JPlayer component to this page and send playlist array as prop to JPlaylist component. I don't use Redux in my app at all.

I found a problem with 206 partial requests, Service Worker does not work correctly with 206 requests in my app

@appigram Great. Glad you found a fix. As you found out, react-jPlayer doesn't have anything to do with partial requests. The only option that specifies loading is the preload option that is automatically set to only download the 'metadata' of the media file.
Closing this issue.


Minor fixups

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

3 of 3 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.007%) to 94.343%

Totals

Change from base Build<PHONE_NUMBER>:
0.007%

Covered Lines:
1601

Relevant Lines:
1697

💛  - Coveralls


Portal doesn't work with current Tink grpc version

Expected Behaviour

Portal should run and the GUI should show data from a running configured recent Tinkerbell install
Current Behaviour

Portal compiles and runs but doesn't work and returns errors in the GUI or doesnt do anything at all. Generic gui shows but nothing works.

I expected  to see data from the local running Tink instance
Possible Solution

If Portal is still supported the GRPC code should be updated to the current Tink grpc code.

Steps to Reproduce (for bugs)

Just run Portal against a current Tinkerbell implementation and see that it doesn't work...
Context

I'm trying to get a GUI on my running Tinkerbell installation

Your Environment

Standard Ubuntu Tinkerbell Provisioner VM with the most recent Tinkerbell version from the standard public registry

Operating System and version (e.g. Linux, Windows, MacOS):
Linux ubuntu

How are you running Tinkerbell? Using Vagrant & VirtualBox, Vagrant & Libvirt, on Packet using Terraform, or give details:
Vagrant Libvirt

Link to your project or a code example to reproduce issue:

@stolsma Thank you for reporting this issue.
The portal is certainly not in sync with the recent tink changes. This is primarily because the team is majorly focused on making tinkerbell more stable as a product. You can always check the projects to know what's keeping the team occupied.
We will try and bring the portal to a stable state as soon as we can.
Thanks. :)

@gauravgahlot Thanks for the quick reply!
Understandable! Didn't know about the projects page!
If I have some time next week I will try to update the GRPC and supporting code and make a pull request...

That would be awesome. 🚀 Looking forward to it.

@gauravgahlot, working on getting working code but I found another bug too, that one is related to the very basic implementation of caching in redis. When creating workflows the new workflows are not shown on the web page because data from the new flow is not cached. The same happens for all data in the web page (hardware, template, workflow) when the CLI is used for adding/changing/removing Tink data!.
This brought me to the question why the replies from Tink are cached? It doesn't give much profit as you still need to query Redis over the network instead of Tink. If the load on Tink and subsequently the database in Postgress is the problem here, then implementing some sort of response caching in Tink would be the best place to solve that problem imo.
It would definitely clean up the code if caching could be removed ;-)

@gauravgahlot, working on getting working code but I found another bug too, that one is related to the very basic implementation of caching in redis. When creating workflows the new workflows are not shown on the web page because data from the new flow is not cached. The same happens for all data in the web page (hardware, template, workflow) when the CLI is used for adding/changing/removing Tink data!.
This brought me to the question why the replies from Tink are cached? It doesn't give much profit as you still need to query Redis over the network instead of Tink. If the load on Tink and subsequently the database in Postgress is the problem here, then implementing some sort of response caching in Tink would be the best place to solve that problem imo.
It would definitely clean up the code if caching could be removed ;-)

@stolsma When I started the project, I felt like we need a cache for a better/faster response. The response from Tink API does not always have the details we want to present on the UI. And so it happens that to present certain details on the UI you make multiple calls to tink/db, which is something I wanted to avoid.
I will be happy to discuss how we can improve the overall design yet keep the portal responsive.
Please use this link to join Equinix Metal Community, if not already. You can find me as GG. 👍🏼

@stolsma When I started the project, I felt like we need a cache for a better/faster response. The response from Tink API does not always have the details we want to present on the UI. And so it happens that to present certain details on the UI you make multiple calls to tink/db, which is something I wanted to avoid.
I will be happy to discuss how we can improve the overall design yet keep the portal responsive.
Please use this link to join Equinix Metal Community, if not already. You can find me as GG. 👍🏼

Thanks for merging! 👍

Thanks for merging! 👍


shell: Add a configuration for not printing shell errors

The shell currently prints out : command not found and
Please specify a subcommand. when invalid commands are
sent.
This can cause issues in certain situations, such as if a U-Boot
console in interactive mode is on the other end of the shell's
UART, where the U-Boot console will stop booting the device
if it receives any characters, and it will print out strings that
the shell then sees as commands. This causes the shell to send
out the messages above, preventing the device running
U-Boot on the other end from booting up.
I added the configurations
CONFIG_SHELL_MSG_SPECIFY_SUBCOMMAND and
CONFIG_SHELL_MSG_CMD_NOT_FOUND to allow disabling
these messages.
Signed-off-by: Brian Moran<EMAIL_ADDRESS>

Depending on the context, might turning off shell echo address this issue?
I'm wary that the approach you've suggested might be overly drastic. If you could share more details about the specific scenarios where shell message output is a concern, perhaps we could identify a more refined solution. If possible, I would like to explore less invasive alternatives

@jakub-uC Hello, setting SHELL_ECHO_STATUS=n and CONFIG_SHELL_PROMPT_UART="" partially address the issue by no longer printing the echo and prompt, but the shell will still print out the SHELL_ERROR messages I mentioned above when it receives messages from U-Boot, such as Loading or >. If preferred, I could move the configuration checks to shell.c lines 531, 695, and 742 seeing as printing SHELL_MSG_CMD_NOT_FOUND and SHELL_MSG_SPECIFY_SUBCOMMAND are the messages that cause the issue. I could also create a separate configuration for each of them if you think it'd be beneficial.

@Brianmm94
In my opinion, your new suggestion is very good and aligns more closely with our current implementation, for example, the shell's CONFIG_SHELL_HELP printing functionality. I think it will be more coherent and potentially less disruptive.
Thank you for the alternative!

Updated! I used the names CONFIG_SHELL_MSG_CMD_NOT_FOUND and CONFIG_SHELL_MSG_SPECIFY_SUBCOMMAND to match the existing SHELL_MSG_CMD_NOT_FOUND and SHELL_MSG_SPECIFY_SUBCOMMAND #defines.

Hi @Brianmm94, I had another thought that I wanted to share with you. Would you mind checking if the CONFIG_SHELL_AUTOSTART=n config meets your needs for your use case? You can call shell_start after booting up. Just a suggestion, and I'd love to hear your thoughts on this!

Hello @jakub-uC, that's actually what we started doing at first, but that solution only works if the U-Boot device on the other end just boots up once when the product is first powered on. If the U-Boot device power cycles for any reason after that, then the issue still occurs. For example, in production we power off the U-Boot device to reduce power draw when it's not needed and we have APIs for power cycling it.

Just out of curiosity, why is the Zephyr console and the U-boot console connected together in the first place?


bugfix:   修复安装通道场景下发安装包到Proxy 接入点适配 (closed #1834)

PR
CheckList（PR Assigners）
确认已完成以下操作：

[ ] 已将分支与当前合入分支同步
[ ] 已在关联 Issue 中补充「功能自测」部分内容
[ ] 已更新关联 Issue 的状态 backlog -> todo -> for test -> tested（需通过测试同学测试） -> done
[ ] 将 PR 添加上 pr/reviewable

CheckList（PR Reviewers）
确认已完成以下操作：

[ ] 检查关联 Issue 「功能自测」部分内容
[ ] 必要时打回 pr/reviewable，要求重新修改
[ ] 及时 Approve

Codecov Report

Merging #1836 (faf352e) into v2.4.2-dev (8d92be1) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@              Coverage Diff               @@
##           v2.4.2-dev    #1836      +/-   ##
==============================================
- Coverage       74.14%   74.14%   -0.01%
==============================================
  Files             397      397
  Lines           27098    27097       -1
==============================================
- Hits            20093    20091       -2
- Misses           7005     7006       +1

Files Changed
Coverage Δ

...s/backend/components/collections/agent_new/base.py
90.84% <100.00%> (-0.06%)
:arrow_down:

... and 1 file with indirect coverage changes


add tc for vulnerability transform

Coverage increased (+10.1%) to 64.464% when pulling 8f41be471d7caa30eca0b55a85917d61186104d7 on reasonerjt:tc-vuln-data into e3e722b1f8e315d2b8dec063362b88d4a6f9312f on vmware:master.

Coverage increased (+0.2%) to 54.526% when pulling 5b6c53a1bfedd296b88ef4ab301d23c29dfaa353 on reasonerjt:tc-vuln-data into e3e722b1f8e315d2b8dec063362b88d4a6f9312f on vmware:master.


Error when parse tokens, and file .exe somehow detect virus

Check validity tokens? [Y/N]: y

Check nitro and payments on tokens? [Y/N]: y

Parse tokens...
Traceback (most recent call last):
File "d:\Project Random by Ikhari\Python\TokenChecker.py", line 284, in
main()
File "d:\Project Random by Ikhari\Python\TokenChecker.py", line 135, in main
for line in [x.strip() for x in open(f"{tokenFileName}", errors="ignore").readlines() if x.strip()]:
UnboundLocalError: local variable 'tokenFileName' referenced before assignment

fix pls

fix pls

New update


change strong to b

Hey @whaus7 , strong is also a valid tag for bold text. Any specific reason for preferring b.

Hey jPuri! Stupid bbedit for php supposedly doesn't like  im not a
backend guy so I didn't feel like messing around with the bbedit side
On Fri, Nov 10, 2017 at 12:37 PM, Jyoti Puri<EMAIL_ADDRESS>wrote:

Hey @whaus7 https://github.com/whaus7 , strong is also a valid tag for
bold text. Any specific reason for preferring b.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jpuri/draftjs-to-html/pull/27#issuecomment-343552289,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACjcNeYyv8JzZrEy3hO8C9xSnSK85m5zks5s1JfjgaJpZM4QXDlk
.

Ah I see, I would prefer to make this configurable to select tag for a style. I would be open to a PR for this. Or I will make this change myself as I get time.
Thanks for bringing up this requirement.

Awesome! I was going to do a PR for that and then delete these packages I
published. I was just forking and publishing for 2 reason. 1. Ive never
published a package before on NPM and wanted to learn. And 2. Just needed a
quick fix. Wasn't sure how responsive you would be. When I get some time
this week ill try to make a PR for the config settings. But please dont be
too harsh! I'm just starting to get into the open source scene. Been a
consuming mole rat for the longest time
On Sun, Nov 12, 2017 at 12:19 PM, Jyoti Puri<EMAIL_ADDRESS>wrote:

Ah I see, I would prefer to make this configurable to select tag for a
style. I would be open to a PR for this. Or I will make this change myself
as I get time.
Thanks for bringing up this requirement.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jpuri/draftjs-to-html/pull/27#issuecomment-343756196,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACjcNWGm0McMXwlhSlwyYfpqf1Y2EqlTks5s1zalgaJpZM4QXDlk
.


Increment version for applicationinsights releases

Increment package version after release of Azure.ResourceManager.ApplicationInsights

API change check
API changes are not detected in this pull request.


bazarr: 1.0.2 -> 1.0.3

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/morpheus65535/bazarr/releases.
meta.description for bazarr is: "Subtitle manager for Sonarr and Radarr"
meta.homepage for bazarr is: "https://www.bazarr.media/"
meta.changelog for bazarr is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
Warning: a test defined in passthru.tests did not pass
Warning: no invocation of /nix/store/vpbbvq4m96gf1077jylq9q9ldvignjsw-bazarr-1.0.3/bin/bazarr had a zero exit code or showed the expected version
0 of 1 passed binary check by having a zero exit code.
0 of 1 passed binary check by having the new version present in output.
found 1.0.3 with grep in /nix/store/vpbbvq4m96gf1077jylq9q9ldvignjsw-bazarr-1.0.3
found 1.0.3 in filename of file in /nix/store/vpbbvq4m96gf1077jylq9q9ldvignjsw-bazarr-1.0.3

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
bazarr

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/vpbbvq4m96gf1077jylq9q9ldvignjsw-bazarr-1.0.3 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A bazarr https://github.com/r-ryantm/nixpkgs/archive/9d3a2508162e770ee90c5fbf17d743ce50986831.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/vpbbvq4m96gf1077jylq9q9ldvignjsw-bazarr-1.0.3
ls -la /nix/store/vpbbvq4m96gf1077jylq9q9ldvignjsw-bazarr-1.0.3/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

bazarr

Maintainer pings
cc @d-xo for testing.

@ofborg test bazarr


ArgumentNullException thrown on parse failure instead of helpful message

Hi there,
I've just added some extra unit tests to my parsing code (following on from https://github.com/b3b00/csly/issues/254), and I'm seeing an ArgumentNullException thrown from inside the parser in one particular test case.
I'm using csly 2.8.0.
I've produced a simplified version of my app (with no extra dependencies this time!) which shows a failing test:
https://github.com/RoystonS/CslyNullIssue
It's the same parser as in my real app, but instead of generating an AST I've made this one simply generate a string during parsing to make reproduction easier.
The test output for the failing test is:
Message:
System.ArgumentNullException : Value cannot be null. (Parameter 'collection')

Stack Trace:
List`1.InsertRange(Int32 index, IEnumerable`1 collection)
List`1.AddRange(IEnumerable`1 collection)
SyntaxParseResult`1.AddExpectings(IEnumerable`1 expected)
EBNFRecursiveDescentSyntaxParser`2.ParseExpressionRule(IList`1 tokens, Rule`1 rule, Int32 position, String nonTerminalName)
EBNFRecursiveDescentSyntaxParser`2.Parse(IList`1 tokens, Rule`1 rule, Int32 position, String nonTerminalName)
RecursiveDescentSyntaxParser`2.ParseNonTerminal(IList`1 tokens, String nonTerminalName, Int32 currentPosition)
EBNFRecursiveDescentSyntaxParser`2.Parse(IList`1 tokens, Rule`1 rule, Int32 position, String nonTerminalName)
RecursiveDescentSyntaxParser`2.ParseNonTerminal(IList`1 tokens, String nonTerminalName, Int32 currentPosition)
RecursiveDescentSyntaxParser`2.ParseNonTerminal(IList`1 tokens, NonTerminalClause`1 nonTermClause, Int32 currentPosition)
EBNFRecursiveDescentSyntaxParser`2.ParseExpressionRule(IList`1 tokens, Rule`1 rule, Int32 position, String nonTerminalName)
<34 more frames...>
RecursiveDescentSyntaxParser`2.ParseNonTerminal(IList`1 tokens, NonTerminalClause`1 nonTermClause, Int32 currentPosition)
EBNFRecursiveDescentSyntaxParser`2.ParseExpressionRule(IList`1 tokens, Rule`1 rule, Int32 position, String nonTerminalName)
EBNFRecursiveDescentSyntaxParser`2.Parse(IList`1 tokens, Rule`1 rule, Int32 position, String nonTerminalName)
RecursiveDescentSyntaxParser`2.ParseNonTerminal(IList`1 tokens, String nonTerminalName, Int32 currentPosition)
EBNFRecursiveDescentSyntaxParser`2.Parse(IList`1 tokens, Rule`1 rule, Int32 position, String nonTerminalName)
RecursiveDescentSyntaxParser`2.Parse(IList`1 tokens, String startingNonTerminal)
Parser`2.ParseWithContext(IList`1 tokens, Object parsingContext, String startingNonTerminal)
Parser`2.Parse(String source, String startingNonTerminal)
ExpressionParser.Parse[T](String expression) line 94
UnitTest1.BooleanParser(String expression, String expected) line 52

This particular test is a fairly heavily nested expression
1 < 2 && 1 <= 2 && 1 == 2 && 1 >= 2 && 1 > 2 && 1 != 2 && 1 <> 2
This test case is checking grouping behaviour and also doing exhaustive testing of all my operators. The expected parse result should be
(((((((1 < 2) AND (1 <= 2)) AND (1 == 2)) AND (1 >= 2)) AND (1 > 2)) AND (1 != 2)) AND (1 != 2))
but the parse is throwing ArgumentNullException.

looking at the callstack I suspect that issue #254 fix is the root of this bug. I will look a it tomorrow. thanks for reporting

I've just done a little more debugging myself, and it looks like this particular expression is actually invalid.  (This parser grammar is missing the actual parsing logic for the <> operator, so should be failing.)
I've just wound back to csly <IP_ADDRESS> and I get different behaviour. I get a nice clear error message instead:
System.Exception : unexpected "COMPARISON [>] @line 1, column 80" (COMPARISON). Expecting MINUS, LVAR, SIMVAR, LPAREN, OFF, ON, HEX_NUMBER, DECIMAL_NUMBER, .

So the problem is a pretty minor one: the error message has simply gotten lost in the 2.8.0 update, not that something has fundamentally gone wrong with the parser.

hello @RoystonS ,
I 've found a fix. You can test it on branch bugfix/#259
As soon as you're ok , i will build a new release

Yep, looks good!

I am not totally satisified with the error message that states :
unexpected "COMPARISON [<] @line 1, column 67" (COMPARISON). Expecting LOGICAL_OR, .

the "<" token is legit at position 67. I was hoping something like
unexpected "COMPARISON [>] @line 1, column 68" (COMPARISON). Expecting ...... .

I guess the expected tokens should be operands :

LVAR
SIMVAR
LPAREN
OFF
ON
HEX_NUMBER
DECIMAL_NUMBER
maybe i'm missing some

I've pushed one more fix that help deliver better error message :
unexpected "COMPARISON [>] @line 1, column 72" (COMPARISON). Expecting MINUS, OFF, ON, HEX_NUMBER, DECIMAL_NUMBER, LVAR, SIMVAR, LPAREN, .

You can test it on the same branch as previously

I 've release nuget <IP_ADDRESS>


Tech debt: add from __future__ import annotations in the Parser feature

Why is this needed?
Reference: https://github.com/aws-powertools/powertools-lambda-python/issues/4607
Add from future import annotations to all files, including examples
Which area does this relate to?
Static typing
Suggestion
No response
Acknowledgment

[X] This request meets Powertools for AWS Lambda (Python) Tenets
[ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

Closed as completed.


npm test error

<EMAIL_ADDRESS>test G:\Pluralsight-JsDevEnv
mocha --reporter progress buildScripts/testSetup.js "src/**/*.test.js"

Warning: Could not find any test files matching pattern: src/**/*.test.js
0 passing (2ms)
Why does this give 0 passing?

This is just saying it can't find any test files. Create at least one test and this will go away.


Message 10: interpret and update Haskell annotation code

Should we really create a copy of the whole annotate.hs file when a single line changes? I usually replace the existing file and rely on Git to keep track of changes.

I could go either way on this. Given the way the documentation is creating a narrative of our discoveries along the way I have a slight preference to avoid leaking interpretations about subsequent messages in earlier writeups.

You have a valid point, but:

If we go this way, we should update all the previous messages with diffs to be consistent. Sounds like a lot of work. Not sure if you're inclined to do this :-)
It bloats the downloadable PDF version a lot, and I'd like to have a nice PDF version for my... PhD thesis, you know.

That's also a good point. I'll push a commit soon that makes the change in annotate.hs instead.


WebApiToSwaggerGenerator creates parameters with error "ONE_OF_MISSING"

I've tried generating a swagger file for a webapi endpoint. When I take the json output to http://editor.swagger.io, I get the error ONE_OF_MISSING for several (all?) parameters.
Here's my generator code:
var swaggerSettings = new WebApiToSwaggerGeneratorSettings
           {
               DefaultPropertyNameHandling = NJsonSchema.PropertyNameHandling.CamelCase,
               DefaultUrlTemplate = "api/v0.1/{controller}/{id}",
               NullHandling = NJsonSchema.NullHandling.JsonSchema,
               GenerateKnownTypes = true,
               TypeNameGenerator = new DefaultTypeNameGenerator()

           };
           var swaggerGenerator = new WebApiToSwaggerGenerator(swaggerSettings);
           var document = swaggerGenerator.GenerateForController<CountriesController>();
           string json = document.ToJson();

and here's the resulting json:

{
  "swagger": "2.0",
  "info": {
    "title": "Web API Swagger specification",
    "version": "1.0.0"
  },
  "schemes": [],
  "consumes": [
    "application/json"
  ],
  "produces": [
    "application/json"
  ],
  "paths": {
    "/api/v0.1/Countries": {
      "get": {
        "tags": [
          "Countries"
        ],
        "operationId": "Countries_GetAll",
        "parameters": [
          {
            "type": "integer",
            "title": "GetAllCountriesModel_ZoneId",
            "name": "zoneId",
            "in": "query",
            "x-nullable": true
          },
          {
            "type": "string",
            "title": "String",
            "name": "search",
            "in": "query",
            "x-nullable": true
          },
          {
            "type": "integer",
            "name": "maximumPageSize",
            "in": "query",
            "x-nullable": false
          },
          {
            "type": "integer",
            "title": "PagingModel_Page",
            "name": "page",
            "in": "query",
            "x-nullable": false
          },
          {
            "type": "integer",
            "title": "PagingModel_PageSize",
            "name": "pageSize",
            "in": "query",
            "x-nullable": false
          },
          {
            "type": "boolean",
            "title": "PagingModel_Descending",
            "name": "descending",
            "in": "query",
            "x-nullable": false
          }
        ],
        "responses": {
          "200": {
            "description": "",
            "schema": {
              "type": [
                "array",
                "null"
              ],
              "items": {
                "$ref": "#/definitions/CountryModel"
              }
            },
            "x-nullable": true
          }
        }
      }
    },
    "/api/v0.1/Countries/{id}": {
      "get": {
        "tags": [
          "Countries"
        ],
        "operationId": "Countries_Get",
        "parameters": [
          {
            "type": "string",
            "name": "id",
            "in": "path",
            "required": true,
            "x-nullable": false
          }
        ],
        "responses": {
          "200": {
            "description": "",
            "schema": {
              "type": "file"
            },
            "x-nullable": true
          }
        }
      }
    }
  },
  "definitions": {
    "CountryModel": {
      "type": "object",
      "title": "CountryModel",
      "additionalProperties": false,
      "properties": {
        "countryCode": {
          "type": [
            "null",
            "string"
          ]
        },
        "countryName": {
          "type": [
            "null",
            "string"
          ]
        }
      }
    }
  },
  "responses": {},
  "securityDefinitions": {}
}

NullHandling = NJsonSchema.NullHandling.JsonSchema,

Is not allowed for a Swagger specification.

The spec has "title" properties in the parameters:
"title": "GetAllCountriesModel_ZoneId",

Where is this coming from?
If you remove them and use NullHandling.Swagger then it should work...

Sorry, the title is set by NSwag, I'll fix that...

Strange, I cannot reproduce this, are you using NSwag v8.0?

I am referencing nswag 8 (current latest on nuget). I have removed the line
 NullHandling = NJsonSchema.NullHandling.JsonSchema,
...but still get the same error. It's great to know you're actively supporting this project.
One thing about the controller is it uses the [FromUri] attribute to bind parameter objects from the query string.

NullHandling fixed some of the validation errors.
This commit should fix the "title" validation errors: https://github.com/NJsonSchema/NJsonSchema/commit/09c795a624fddd3ecad6e72bf58ce41142dab04a

I think this fixes the "title" problem: https://github.com/NJsonSchema/NJsonSchema/commit/5fdbe73fe92178d9b487137c4356bc1e7596f1bd

Can you try with this NSwagStudio version? (I cannot reproduce this)
NSwagStudio.zip

Please try again with v8.1

Still getting the same validation error with 8.1. I will supply the json and then the swagger editor validation.

swagger: '2.0'
info:
  title: Web API Swagger specification
  version: 1.0.0
schemes: []
consumes:
  - application/json
produces:
  - application/json
paths:
  /api/v0.1/Countries:
    get:
      tags:
        - Countries
      operationId: Countries_GetAll
      parameters:
        - type: integer
          title: GetAllCountriesModel_ZoneId
          name: zoneId
          in: query
          x-nullable: true
        - type: string
          name: search
          in: query
          x-nullable: true
        - type: integer
          name: maximumPageSize
          in: query
          x-nullable: false
        - type: integer
          title: PagingModel_Page
          name: page
          in: query
          x-nullable: false
        - type: integer
          title: PagingModel_PageSize
          name: pageSize
          in: query
          x-nullable: false
        - type: boolean
          title: PagingModel_Descending
          name: descending
          in: query
          x-nullable: false
      responses:
        '200':
          description: ''
          schema:
            type: array
            items:
              $ref: '#/definitions/CountryModel'
          x-nullable: true
  '/api/v0.1/Countries/{id}':
    get:
      tags:
        - Countries
      operationId: Countries_Get
      parameters:
        - type: string
          name: id
          in: path
          required: true
          x-nullable: false
      responses:
        '200':
          description: ''
          schema:
            type: file
          x-nullable: true
definitions:
  CountryModel:
    type: object
    additionalProperties: false
    properties:
      countryCode:
        type: string
      countryName:
        type: string
responses: {}
securityDefinitions: {}

Swagger Error

Not a valid parameter definition
Jump to line 39
Details
Object
code: "ONE_OF_MISSING"
params: Array [0]
message: "Not a valid parameter definition"
path: Array [5]
0: "paths"
1: "/api/v0.1/Countries"
2: "get"
3: "parameters"
4: "5"
schemaId: "http://swagger.io/v2/schema.json#"
inner: Array [2]
0: Object
code: "ONE_OF_MISSING"
params: Array [0]
message: "Data does not match any schemas from 'oneOf'"
path: Array [5]
0: "paths"
1: "/api/v0.1/Countries"
2: "get"
3: "parameters"
4: "5"
inner: Array [2]
0: Object
code: "OBJECT_MISSING_REQUIRED_PROPERTY"
params: Array [1]
0: "schema"
message: "Missing required property: schema"
path: Array [0]
1: Object
code: "ONE_OF_MISSING"
params: Array [0]
message: "Data does not match any schemas from 'oneOf'"
path: Array [0]
inner: Array [4]
0: Object
code: "OBJECT_ADDITIONAL_PROPERTIES"
params: Array [1]
0: Array [1]
0: "title"
message: "Additional properties not allowed: title"
path: Array [0]
1: Object
code: "OBJECT_ADDITIONAL_PROPERTIES"
params: Array [1]
0: Array [1]
0: "title"
message: "Additional properties not allowed: title"
path: Array [0]
2: Object
code: "OBJECT_ADDITIONAL_PROPERTIES"
params: Array [1]
0: Array [1]
0: "title"
message: "Additional properties not allowed: title"
path: Array [0]
3: Object
code: "OBJECT_MISSING_REQUIRED_PROPERTY"
params: Array [1]
0: "required"
message: "Missing required property: required"
path: Array [0]
1: Object
code: "OBJECT_MISSING_REQUIRED_PROPERTY"
params: Array [1]
0: "$ref"
message: "Missing required property: $ref"
path: Array [5]
0: "paths"
1: "/api/v0.1/Countries"
2: "get"
3: "parameters"
4: "5"
level: 900
type: "Swagger Error"
description: "Not a valid parameter definition"
lineNumber: 39

Sorry, I cannot reproduce the problem.. Can you provide a sample project or at least the Web API controller code?

I have added a project to recreate:
https://github.com/jameschch/webapisample

You are using FromUri with a complex object... This should be easily fixed...

Great I guessed it might be connected to fromui

Will the fix for this issue be posted to nuget? I can still recreate with 8.1.6192

The fix is not released yet, use the CI build artifacts: https://ci.appveyor.com/project/rsuter/nswag-25x6o/build/artifacts


Build Docker image using Google Cloud Build

Adding some notes on building Docker images using Google Cloud Build, and uploading to Artifact Registry.

Closing for now, might revisit at some other stage with something more readable ;-)
And thanks for reviewing Leo!


Simplified Feign

with this change we no longer treat retries as a continuation of a previous span. That way the Feign code simplifies a lot. RequestInterceptor starts a span and the TraceFeignClient will always close it no matter what's happening.
fixes #202
cc @adriancole

looks good!

Current coverage is 74.78% (diff: 88.57%)

Merging #372 into master will decrease coverage by 1.01%

@@             master       #372   diff @@
==========================================
  Files           136        130     -6
  Lines          2867       2768    -99
  Methods           0          0
  Messages          0          0
  Branches        395        383    -12
==========================================
- Hits           2173       2070   -103
- Misses          525        531     +6
+ Partials        169        167     -2

Powered by Codecov. Last update 9c09222...1f9a169


[BUG] vm-dhcp-agent failed to communicate with kube-apiserver

Describe the bug

This is the issue I discovered while testing the vm-dhcp-controller's limits. When creating an IPPool with a subnet contains the internal Kubernetes pod/service network (10.5[23].0.0/16), the agent will fail to communicate with kube-apiserver because the network is shadowed. For example, the following IPPool has CIDR <IP_ADDRESS>/8
apiVersion: network.harvesterhci.io/v1alpha1
kind: IPPool
metadata:
  namespace: default
  name: super-net
spec:
  ipv4Config:
    serverIP: <IP_ADDRESS>
    cidr: <IP_ADDRESS>/8
    pool:
      start: <IP_ADDRESS>
      end: <IP_ADDRESS>
    leaseTime: 300
  networkName: default/super-net

After creating the IPPool object, check the agent's log:
$ kubectl -n harvester-system logs default-super-net-agent
Defaulted container "agent" out of: agent, ip-setter (init)
time="2024-02-16T11:30:56Z" level=info msg="Starting VM DHCP Agent: default-super-net-agent"
time="2024-02-16T11:30:56Z" level=info msg="monitor ippool default/super-net"
time="2024-02-16T11:30:56Z" level=info msg="(ippool.EventListener) starting IPPool event listener"
time="2024-02-16T11:30:56Z" level=info msg="(ippool.Run) starting IPPool controller"
time="2024-02-16T11:30:56Z" level=info msg="Starting HTTP server"
time="2024-02-16T11:30:56Z" level=info msg="Listening on port: 8080"
time="2024-02-16T11:30:56Z" level=info msg="(dhcp.Run) starting DHCP service on nic eth1"
W0216 11:30:59.535227       1 reflector.go:533] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
E0216 11:30:59.535326       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: Failed to watch *v1alpha1.IPPool: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
W0216 11:31:03.439592       1 reflector.go:533] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
E0216 11:31:03.439703       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: Failed to watch *v1alpha1.IPPool: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
W0216 11:31:09.519542       1 reflector.go:533] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
E0216 11:31:09.519759       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: Failed to watch *v1alpha1.IPPool: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
W0216 11:31:17.263172       1 reflector.go:533] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
E0216 11:31:17.263236       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: Failed to watch *v1alpha1.IPPool: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
W0216 11:31:27.471175       1 reflector.go:533] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
E0216 11:31:27.471234       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: Failed to watch *v1alpha1.IPPool: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
W0216 11:31:53.071193       1 reflector.go:533] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
E0216 11:31:53.071221       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.9/tools/cache/reflector.go:231: Failed to watch *v1alpha1.IPPool: failed to list *v1alpha1.IPPool: Get "https://<IP_ADDRESS>:443/apis/network.harvesterhci.io/v1alpha1/namespaces/default/ippools?limit=500&resourceVersion=0": dial tcp <IP_ADDRESS>:443: connect: no route to host
time="2024-02-16T11:31:59Z" level=warning msg="(dhcp.dhcpHandler) NO LEASE FOUND: hwaddr=64:4b:f0:37:d8:c3"
time="2024-02-16T11:32:00Z" level=warning msg="(dhcp.dhcpHandler) NO LEASE FOUND: hwaddr=64:4b:f0:37:d8:c3"
time="2024-02-16T11:32:03Z" level=warning msg="(dhcp.dhcpHandler) NO LEASE FOUND: hwaddr=64:4b:f0:37:d8:c3"

Losing the ability to connect to kube-apiserver means the agent will be unable to serve DHCP clients because it cannot sync with the IPPool object.
To Reproduce
For a more specific case, just create an ippools.network object like the following:
apiVersion: network.harvesterhci.io/v1alpha1
kind: IPPool
metadata:
  namespace: default
  name: overlap-net
spec:
  ipv4Config:
    serverIP: <IP_ADDRESS>
    cidr: <IP_ADDRESS>/16
    pool:
      start: <IP_ADDRESS>
      end: <IP_ADDRESS>
    leaseTime: 300
  networkName: default/overlap-net

Expected behavior

The agent should be able to communicate with kube-apiserver without problems.
Support bundle

Environment

Harvester ISO version: v1.2.1 (with vm-dhcp-controller v0.1.0)
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): KVM

Additional context
There are two remedies that I can think of:

Denied such IPPool objects (CIDR contains the pod/service network) from creating
Add a static and strict routing rule <IP_ADDRESS>/32 by the init container

Suppose an extreme scenario...
The user creates an IPPool object with CIDR <IP_ADDRESS>/16, precisely the same as the Kubernetes service network subnet. The vm-dhcp-webhook and controller must ensure the following:

Knowing what the subnet is used for the service network. For example, a Kubernetes cluster installed using kubeadm will have <IP_ADDRESS>/12 as its default service network subnet; in Harvester's context, it is --service-cluster-ip-range=<IP_ADDRESS>/16.
The embedded DHCP server's IP address must not be the same as the one of the default/kubernetes Service, e.g., <IP_ADDRESS>
A strict static route, <IP_ADDRESS>/32, must be configured in the agent Pod for traffic to and from the Kubernetes API server
Finally, the IP address <IP_ADDRESS> must not be allocated to any DHCP clients (marked as reserved)

It could be done. Theoretically, the Kubernetes cluster network would not be compromised in such cases since their underlying networks are different, albeit they have the same L3 subnet info configured.
Any insight into this?
cc @w13915984028 @mingshuoqiu
Thank you.

The first step will be exclude those subnets in both webhook and document.
To battle with k8s internal subnet will only get endless issues and troubleshootings.

There seems to be no good way to grab such information (service CIDR) using native Kubernetes API calls. I'm considering making it a configuration via a chart value, e.g. .webhook.serviceCIDR, but not detecting it in runtime.

For Harvester worker nodes, no --service-cidr flag is set in the nodes' rke2.io/node-args annotation. If the user enables the harvester-vm-dhcp-controller add-on on a 2-node cluster (or clusters of more than three nodes), the webhook fails to validate the request during IPPool object creation.

For vm-dhcp-controller to be vendor-agnostic, it's better to have a general solution. Its usage would be limited depending on one annotation that only applies to a specific Kubernetes distribution. Since there's still no good way to retrieve such information via native Kubernetes API, I suggest we expose a new command line option for users to configure the cluster-wide service CIDR for the webhook to consume.

Test Plan

Install and enable the harvester-vm-dhcp-controller add-onapiVersion: harvesterhci.io/v1beta1
kind: Addon
metadata:
  name: harvester-vm-dhcp-controller
  namespace: harvester-system
  labels:
    addon.harvesterhci.io/experimental: "true"
spec:
  enabled: true
  repo: https://charts.harvesterhci.io
  version: 0.3.1
  chart: harvester-vm-dhcp-controller
  valuesContent: ""

Create a VM Network (NAD) called test-net
Create the following IPPool object with kubectlapiVersion: network.harvesterhci.io/v1alpha1
kind: IPPool
metadata:
  namespace: default
  name: test-net
spec:
  ipv4Config:
    cidr: <IP_ADDRESS>/24
  networkName: default/test-net

The above kubectl create request should be rejected by the webhook.

This looks great on  Version: v1.3-f8469881-head w/ v0.3.1 chart version for the addon :smile: :+1: - thanks for this fix @starbops  - I'll go ahead and close this out :smile:


Create weekly_broken_link_finder.yml

This PR adds a caller workflow that will run notebook-ci-actions workflow broken_link_checker.yml  to identify broken links in https://spacetelescope.github.io/mast_notebooks/ weekly every Sunday at 0400 UTC (2000 ET).

What! That's so cool; happy that we're adding this


Rofi prompt change with rofi.theme

$ rofi -v
Version: 1.5.0-dirty (tags/1.5.0)

$ wal -v
wal 1.3.2

Enabling the wal theme via the new ~/.config/rofi/config file removed the colon and space in the prompt.
Here is rofi without the theme enabled:

And here it is with the theme enabled:

I don't see the problem but if you want to resolve the problem, you could overwrite the rofi config with this template and place in your file .config/wal/templates/. It's really easy, it's a bit like css.

Fixed.


Support Kelvin as temperature input

In addition to a number from 0-100 (the macOS config format) we can use a more standardized expression of color temperature, like so:
nightlight temp 4000K

Lower and upper bounds should be these (parity with Linux/GSettings night light):
nightlight temp 3500K
nightlight temp 6000K

I made an attempt. I didn't touch your error messages since I wasn't sure where else they were being used. However, I do bounds checking for kelvin temps outside of your specified range. LMK what you think. Thanks!


Add support for automated builds

Use https://github.com/kubernetes/kubernetes/blob/master/.travis.yml as a starting point. @vaikas-google: allowed travis access to this repo on 10/22. Need to find out if travis is still the recommended solution for build and test automation.

Verified in #81.


Distinguish error cases in delActiveWorkflowById()

@RackHD/corecommitters @uppalk1 @tannoa2 @zyoung51 @keedya @geoff-reid

+1

:+1:


Termination message could not be parsed as JSON: parsing message json

Expected Behavior
Pipelines pass
Actual Behavior
When we run multiple pipelines (pipelineruns of the same pipeline), some of them hang on random tasks with events.
In a pipeline with 18 tasks it happens to 1-2 tasks causing the pipeline to run until the timeout.
Warning  InternalError          3m (x19 over 22m)  TaskRun  2 errors occurred:
           * parsing message json: invalid character 'i' in literal false (expecting 'l')
           * parsing message json: invalid character 'i' in literal false (expecting 'l')

Some pipeline hang
Steps to Reproduce the Problem

Run multiple pipelines at the same time (min 4 pipelines with 18 parallel tasks)

Additional Info
Logs from controller:
logger: "tekton-pipelines-controller"

message: "termination message could not be parsed as JSON: parsing message json: invalid character 'i' in literal false (expecting 'l')"

stacktrace: "github.com/tektoncd/pipeline/pkg/pod.setTaskRunStatusBasedOnStepStatus
	github.com/tektoncd/pipeline/pkg/pod/status.go:156
github.com/tektoncd/pipeline/pkg/pod.MakeTaskRunStatus
	github.com/tektoncd/pipeline/pkg/pod/status.go:135
github.com/tektoncd/pipeline/pkg/reconciler/taskrun.(*Reconciler).reconcile
	github.com/tektoncd/pipeline/pkg/reconciler/taskrun/taskrun.go:537
github.com/tektoncd/pipeline/pkg/reconciler/taskrun.(*Reconciler).ReconcileKind
	github.com/tektoncd/pipeline/pkg/reconciler/taskrun/taskrun.go:184
github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun.(*reconcilerImpl).Reconcile
	github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/taskrun/reconciler.go:235
knative.dev/pkg/controller.(*Impl).processNextWorkItem
	knative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/controller/controller.go:542
knative.dev/pkg/controller.(*Impl).RunContext.func3
	knative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/controller/controller.go:491"

Kubernetes version:

Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.1", GitCommit:"3ddd0f45aa91e2f30c70734b175631bec5b5825a", GitTreeState:"clean", BuildDate:"2022-05-24T12:26:19Z", GoVersion:"go1.18.2", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.12-gke.2300", GitCommit:"e55564cf3a1384026a54920174977659c8c56a50", GitTreeState:"clean", BuildDate:"2022-08-16T09:24:51Z", GoVersion:"go1.16.15b7", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.24) and server (1.22) exceeds the supported minor version skew of +/-1

Tekton Pipeline version:

v0.40.2

When we run a single pipeline it works correctly, the issue occurs when we run multiple pipelines at the same time.

Update: when we change names tasks and pipeline issue doesn't occurs:
Changes:
tasks:

xxxxxxx-yyy-test-high-cpu  ->  yyy-high-cpu
xxxxxxx-yyy-test  ->  yyy
pipelines:
xxxxxxx-yyy-test-> yyy


Broken navbar on non-mobile narrow device

This can be fixed in this issue as it's for the non-mobile devices but on 600px width.

On 600px width non-mobile, I see this

Address select dialogue
but I guess that's going to be resolved in #4405

Originally posted by @yangwao in https://github.com/kodadot/nft-gallery/issues/4483#issuecomment-1354745469

👋🏻

this is problem as well, resizing window + using navbar seems to reproduce


CHEBI:82012, please CHECK

Hi Adnan,
Please check fluometuron as it is needed for a Rhea reaction.
Cheers,
Kristian

Now checked


fix(select): remove selection if option is removed

Currenty when an option is selected and the option is somehow programmatically removed the select updates its options.
Right now it does not remove the selection if the selection has been removed from the options.

Fixes #2524

@kara Alright, only kept the tests for now. Let's see if #2499 makes this PR green (once it's merged)


fix: add more correct header format validation

Fix #22

We need to revise the display about the invalid header

We need to revise the display about the invalid header