Error when writing to parquet for columns of struct type

Describe the bug
I have a glue table including a column of type Array<struct<A:string, B:int>>. If I read the data in a data frame, and then try to write it again as parquet, all values of A and B gets null.
To Reproduce
source_prefix = "source/dt=2021-01-01/" # here there is a parquet with a column of type Array<struct<A:string, B:int>>
destination_prefix = "destination/dt=2021-01-01/"
df = wr.s3.read_parquet(source_prefix, boto3_session=boto3_session, dataset=True, validate_schema=True)
dtype = wr.s3.read_parquet_metadata(source_prefix, boto3_session=boto3_session)[0]
wr.s3.to_parquet(df,
path=destination_prefix,
mode="overwrite_partitions",
dataset=True,
compression="snappy",
boto3_session=boto3_session,
max_rows_by_file=None,
dtype=dtype
)
--> In df I see the correct values of the column of type Array (with type object in pandas). However, all the values are set to null after writing to parquet destination.
P.S. Don't attach files. Please, prefer add code snippets directly in the message body.

Thanks for the quick reply.
To reproduce the error:
Below are two parquet files as created by AWS firehose. Place them at an s3 location such  as s3://bucket_name/test/dest/dt=2021-01-07/. Then run the following:
session = boto3.Session(profile_name=your_profile, region_name="eu-north-1")
df = wr.s3.read_parquet("s3://bucket_name/test/dest/dt=2021-01-07/", boto3_session=session, dataset=True, validate_schema=True)
dtype = wr.s3.read_parquet_metadata("s3://bucket_name/test/dest/dt=2021-01-07/", boto3_session=session)[0]
wr.s3.to_parquet(df, path="s3://bucket_name/test/dest/dt=2021-01-08/", mode="overwrite_partitions",
                 dataset=True, compression="snappy", boto3_session=session, dtype=dtype)
df2 = wr.s3.read_parquet("s3://luminita-820918/test/dest/dt=2021-01-08/", boto3_session=session, dataset=True, validate_schema=True)

Now if you check the column onboardcountdetails in df and df2. While in df it looks like this:
[{'CarRef': 'A', 'OnboardCount': 10}, ...
In df2 the fields are empty
[{'carref': None, 'onboardcount': None}, ...
So the only thing I am doing is merging two parquet files in one, then read again the merged parquet.
I am using awswrangler 2.3.0 and python 3.8.2
Let me know if you need additional information.
Thanks
Archive.zip

Alright, got around testing this and you are right @luminita it's a bug on our end, thank you for raising it.
In short, it's caused by this operation on the dtype supplied. The recursion call in the function coupled with the lower method is causing the return type to not match the original type.
I will raise a PR to fix this in the next release, in the meantime you can fix your issue by simply not passing the dtype:
wr.s3.to_parquet(df, path="s3://bucket_name/test/dest/dt=2021-01-08/", mode="overwrite_partitions",
                 dataset=True, compression="snappy", boto3_session=session)

not ideal, but will be corrected in the next version

Great that this got fixed! When will it be released? I really need to use dtype to get the data types right

It will be available in the next release 2.7.0. We cannot commit to a date but our releases are usually monthly.
In the meantime, you can also test it directly by installing wrangler based on the current main branch:
pip uninstall awswrangler -y
pip install git+https://github.com/awslabs/aws-data-wrangler.git@main

Covered in 2.7.0 release

Hi @jaidisido
I came across a similar issue with a struct that include a decimal(6,3) field:
{
"Type": "struct<3h:decimal(6,3)>",
"Name": "rain"
},
The _split_struct function is going to split it using the comma, but it shouldn't.
It should instead use the ":" to determine how many fields are in there then if there is more than one, split using the ":" and ",".
How do you want to proceed on this? Shall I open a new issue?


query! macro doesn't work with composite types

Bug Description
Using the query! macro to insert a row does not work with a composite type defined in the schema and as a Rust struct, but it works using the query function (which obviously does not perform compile-time checking). I've found similar issues, but I don't think any that directly explain why I can't do this.
Minimal Reproduction
In my schema file:
CREATE TYPE metadata AS (
    name text,
    age integer
);

CREATE TABLE person (
    id serial PRIMARY KEY,
    metadata metadata,
    favorite_food text
);

and in main.rs:
use sqlx::{postgres::PgPool, query};

#[derive(Debug, sqlx::Type)]
#[sqlx(type_name = "metadata")]
struct Metadata {
    name: String,
    age: i32,
}

struct Person {
    metadata: Metadata,
    favorite_food: String,
}

#[tokio::main]
async fn main() {
    let metadata = Metadata {
        name: "ahmed".to_string(),
        age: 100,
    };

    let person = Person {
        metadata: metadata,
        favorite_food: "pb&j".to_string(),
    };

    let pool = PgPool::connect("postgres://localhost/test").await.unwrap();

    // This works fine
    query("INSERT INTO person (metadata, favorite_food) VALUES ($1, $2)")
        .bind(&metadata)
        .bind(&person.favorite_food)
        .execute(&pool)
        .await
        .unwrap(); // This works

    // This does not work
    query!(
        "INSERT INTO person (metadata, favorite_food) VALUES ($1, $2)",
        metadata,
        person.favorite_food
    )
    .execute(&pool)
    .await
    .unwrap();
}

The latter, using the query! macro, throws the following error:
error: unsupported type metadata for param #1
  --> src/main.rs:36:5
   |
36 | /     query!(
37 | |         "INSERT INTO person (metadata, favorite_food) VALUES ($1, $2)",
38 | |         metadata,
39 | |         person.favorite_food
40 | |     )
   | |_____^
   |
   = note: this error originates in the macro `$crate::sqlx_macros::expand_query` which comes from the expansion of the macro `query` (in Nightly builds, run with -Z macro-backtrace for more info)

error: could not compile `scamplers-sql` (bin "scamplers-sql") due to 1 previous error

Am I just using sqlx fundamentally wrong, or is this an actual bug? I have a really complex data model, and I can save myself a lot of code by factoring out common metadata fields into these composite types.
Thanks!
Info

SQLx version: 0.7.4
SQLx features enabled: ["migrate", "macros", "json", "postgres", "tls-rustls", "runtime-tokio", "time"]
Database server and version: Postgres 16.3
Operating system: macOS
rustc --version: 1.79.0

From what I can see, this looks like an issue that wouldn't be a straight forward fix since the current parameter type checking is done against a static list of known types which cannot include your custom compile time derived type.
Where your error originates
https://github.com/launchbadge/sqlx/blob/0eb2ee93650eaf10d9cadc107a88bfabfdf377a7/sqlx-macros-core/src/query/args.rs#L58-L73
This uses a database's TypeChecking trait to verify the parameter types. For each database that trait is generated using a macro_rules! block.
https://github.com/launchbadge/sqlx/blob/0eb2ee93650eaf10d9cadc107a88bfabfdf377a7/sqlx-core/src/type_checking.rs#L130-L188
The macro essentially builds a giant if-else if-else block checking all database specified types for a match on TypeInfo. For postgres (and any other database) the list of accepted types is known at compile time from within the crate so your type is never going to be verified.
https://github.com/launchbadge/sqlx/blob/0eb2ee93650eaf10d9cadc107a88bfabfdf377a7/sqlx-postgres/src/type_checking.rs#L10-L213
My guess is that the only solution would involve a special provision using the postgres feature to allow for composite types to bypass the checking.

@KairuDeibisu You're problem is 2 fold:

The query macros can not understand composite types because otherwise you query would be fine to parse out.
The query macros do not currently use FromRow derived traits implementations so your query would not work.

Furthermore, you query would not even work if the query macros used the FromRow implementation because you are returning a row with the composite type in it, not a flattened row. You would need to modify your query to something like:
INSERT INTO user_mstr as um (username, email, password, audit_info)
VALUES ($1, $2, $3, ROW($4, $5, $6, $7))
RETURNING um.username, um.email, um.password, (um.audit_info).*

But again, even if you modified your query to the above and provided FromRow implementations for both types and used the #sqlx(flatten) attribute on the User.audit_info field, the query macros do not work with those implementations of FromRow, rather the query macros desugar to code that resembles the FromRow derived implementation. This desugaring does not respect the flatten attribute and therefore will attempt to construct a User struct instance with the expanded composite type fields included as well which will fail.
You can try to update your code to include my modified SQL query and check the macro expansion of the query_as! to see it attempt to build a row with the audit_info attributes included in the User type.

I was just being dumb. The proper way to do what I was trying to do is to just deconstruct the Metadata struct:
query!(
    "INSERT INTO person (metadata.name, metadata.age, favorite_food) VALUES ($1, $2)",
    metadata.name,
    metadata.age,
    person.favorite_food
)
.execute(&pool)
.await
.unwrap();

just like any other insertion you might do (until sqlx has support for inserting structs in their entirety)


Update all vendors and fix conflicts

What does this PR do?
It updates the repository vendors, especially (but not only) controller-runtime ones, the latest versions and adapts the code base accordingly.

[ ] Adds new functionality
[ ] Alters existing functionality
[ ] Fixes a bug
[ ] Improves documentation or testing

Code Quality Checklist

[x] The documentation is up to date.
[x] My code is sufficiently commented and passes continuous integration checks.
[x] I have signed my commit (see Contributing Docs).

Testing

[ ] I leveraged continuous integration testing

[ ] by depending on existing unit tests or end-to-end tests.
[ ] by adding new unit tests or end-to-end tests.

[ ] I manually tested the following steps:

x
[ ] locally.
[ ] as a canary deployment to a cluster.

Would you want this PR rebased?


UpdateSpec rather than choicemap for update specifications

I think it can sometimes make sense for custom generative functions to know how to perform special types of updates which are not simply of the form of of changing the value of one of the choices in the choicemap.
For instance, in some code I'm writing, we have a type of "split" update which splits a generated object into two generated objects with the same properties as the original.  One hacky thing we can do now is have an update call along the lines of:
update(tr, args, argdiffs, choicemap((:inner_gen_fn_addr => :split, object_id))

But I think that really the call signature for the update function should be something along the lines of
Gen.update(tr::Trace, args, argdiffs, update_spec::UpdateSpecification)

where we have a built-in ChoicemapUpdateSpecification <: UpdateSpecification that choicemaps can get cast to by default so the old form of the interface works.
Then specifying a split update as in the example above could look like
update(tr, args, argdiffs, updatespec((:inner_gen_fn_addr => SplitUpdate(object_id)))

or, if we wanted, we could even have some automatic way to convert Pairs into update specs, so we could have the interface simply look like
update(tr, args, argdiffs, :inner_gen_fn_addr => SplitUpdate(object_id))

(Really, I think that the update method interface should take a fifth optional kwargs argument, as I describe in https://github.com/probcomp/Gen/issues/182, but this is a separate issue.)

Changing the discard return value
Elaborating on the above idea, if we implement this, we will need some way for generative functions to propose these custom UpdateSpecifications for use in MCMC.  I believe that thinking about how this could be implemented suggests that just as the current constraints choicemap should be changed to a more general UpdateSpecification, the discard choicemap returned by the update function should be changed to a reverse_spec UpdateSpecification.
In other words, the current update interface is
new_tr::Trace, weight::Float64, retdiff::Diff, discard::ChoiceMap = Gen.update(tr, args, argdiffs, constraints::ChoiceMap)

but I think we should change it to
new_tr::Trace, weight::Float64, retdiff::Diff, reverse_spec::UpdateSpecification = Gen.update(tr, args, argdiffs, spec::UpdateSpecification)

This is because, as far as I am aware, the only place we use the discard is to tell us how to make reverse moves in MCMC.  Therefore if we make it possible to update traces using more general UpdateSpecification objects, the discard should be moved to an object of this type as well.

The propose framework seems to be lacking...
Say that we were to modify the update interface as I'm suggesting above.  Then when doing MCMC with custom proposals, we may want our proposals to be able to propose UpdateSpecification objects.  In the above text blurb, I used the example where we want to have an update call along the lines of
update(tr, args, argdiffs, :inner_gen_fn_addr => SplitUpdate(object_id))

Say we want this to be proposed.  Then we could write a generative function along the lines of
@gen function propose_split_update(tr)
  @trace(distribution_returning_split_update_spec(tr), :inner_gen_fn_addr)
end

Where distribution_returning_split_update_spec is a Gen.Distribution which will figure out the object_id and return a SplitUpdate.  This is unnatural, though; this means we have to implement a custom distribution which samples an object_id and then returns a SplitUpdate.  It seems like determining the object_id should be handled by a GenerativeFunction, and then wrapping this in a SplitUpdate should somehow be automatic.
One thing we could do is have something along the lines of:
@gen function find_obj_id(tr)
  # whatever logic we need to sample the object id, for instance maybe just sampling
  # from a distribution, but possibly something more complicated
  object_id = @trace(sample_id(tr[:object_ids]), :object_id)
  return object_id
end
@gen function propose_split_udpate(tr)
  id = find_obj_id(tr) # DON'T TRACE THIS so this will not put anything in the constraints
  @trace(identity(SplitUpdate(id)), :inner_gen_fn_addr)
end

Here, we sample the object_id using a gen function without using an address, and then wrap this in an UpdateSpec and put it in the trace by routing it through an identity distribution.  The fact that we have to pass it through an identity distribution feels kind of silly, though.  Also, since we can't trace the call to the gen function where we sample the ID, Gen will not be able to track the probability of selecting the given ID, which I believe it needs to to correctly MH acceptance ratios.  So I don't think this is a good solution.
My sense is a better solution is to change the semantics of what propose does for the dynamic/static DSL, so that if a generative function returns an UpdateSpecification, this is returned by propose rather than a choicemap populated with the @traced values.  Thus our proposal could look like this:
@gen function generate_update_spec(tr)
  object_id = @trace(sample_object_id(tr[:ids]), :id)
  return SplitUpdate(object_id)
end
@gen function proposal(tr)
  @trace(generate_update_spec(tr), :inner_gen_fn_addr)
end

Here we have a generative function generate_update_spec which samples the needed object id and constructs an UpdateSpec.  We then have a proposal function which hierarchically calls propose on generate_update_spec (which returns UpdateSpec(object_id) rather than a choicemap since it detects that the function has a return of a subtype of UpdateSpecification), and nests this in the address hierarchy behind :inner_gen_fn_addr as needed.
Another way we could write this in the framework I'm proposing is
@gen function proposal(tr)
  object_id = @trace(sample_object_id(tr[:ids]), :id)
  return updatespec(:inner_gen_fn_addr => SplitUpdate(object_id))
end

Here, the updatespec function constructs a hierarchically nested UpdateSpecification that is directly returned by the proposal distribution, causing it to be the return value of propose (according to the new semantics I am suggesting).

TLDR
In short my suggestions are:

The constraints::ChoiceMap parameter for Gen.update should be changed to an update_spec::UpdateSpecification
The discard::ChoiceMap return value for Gen.update should be changed to a reverse_update_spec::UpdateSpecification.
The behavior of propose for the static and dynamic DSLs should be changed so that when the generative function returns an object of type <: UpdateSpecification, propose returns this UpdateSpecification rather than a choicemap of the decisions made during execution of the generative function.

  In this example, say I am running "split/merge" MH updates on a trace.  Say I have a custom generative function generate_objects which understands how to "split" and "merge" objects using SplitUpdate and MergeUpdate UpdateSpecifications.  If I pass in a SplitUpdate(obj_id) which results in obj_id splitting to objects with obj_1 and obj_2 as IDs, then the discard from this update call will be a MergeUpdate(obj_1, obj_2), the move which reverses the split by merging the split objects into one object.
If I want to run MH, I can create a proposal distribution which will decide which objects to split and merge:
@gen function splitmerge_proposal(tr)
  if @trace(bernoulli(0.5), :do_split)
    @trace(sample_one(tr[:object_set], :target)
  else
    @trace(sample_one(tr[:object_set]), :target1)
    @trace(sample_one(tr[:object_set]), :target2)
  end
end

Then to use mh, we can use the involution variant to translate from the outputs of propose(splitmerge_proposal) to SplitUpdate and MergeUpdates.  This can look something like
function involution(trace, fwd_choices::ChoiceMap, fwd_ret, _)
  args = get_args(trace)
  argdiffs = map(_ -> NoChange(), args)
  if fwd_choices[:do_split]
    new_tr, weight, retdiff, reverse_update_spec = update(trace, args, argdiffs, SplitUpdate(fwd_choices[:target]))
  bwd_choices = choicemap((:do_split, false), (:target1, reverse_update_spec.target1, reverse_update_spec.target2))
  else
    new_tr, weight, retdiff, reverse_update_spec = update(trace, args, argdiffs, MergeUpdate(fwd_choices[:target1], fwd_choices[:target2))
    bwd_choices = choicemap((:do_split, true), (:target, reverse_update_spec.target))
  end
  return new_tr, bwd_choices, weight

Then to run MH, we can call
tr, acc = Gen.mh(tr, splitmerge_proposal, (), involution)

The involution function handles translating from the decisions made by the splitmerge_proposal to the UpdateSpecifications that should be passed into the update function, and handles translating from the reverse UpdateSpecifications output by update to what values in the proposal distribution could have produced them.
(If we wanted, we could actually implement a whole new version of mh where instead of passing in an involution, we pass in 2 functions: 1 to map from proposal outputs to UpdateSpecs, and one to map from UpdateSpecs to proposal outputs which should imply those UpdateSpecs.)


Split release artifacts

From<EMAIL_ADDRESS>on December 13, 2009 10:40:23
E.g. powermock-easymock-full.jar must be split into two, one powermock-
easymock-junit-full.jar and one powermock-easymock-testng-full.jar. The same
goes for mockito.
Original issue: http://code.google.com/p/powermock/issues/detail?id=216

From<EMAIL_ADDRESS>on July 22, 2010 02:26:18
Status: Fixed


Convert V3 DDO to V4 DDO

Fixes #1
Changes proposed in this PR:

Creating a new Typescript package
Including dev tools: ESLint, Prettier, Husky, dependabot, etc.
Setting codeowners
Manually adding V4 Types to avoid importing ocean.js twice
Importing V3 DDO from Aquarius
Converting V3 DDO to V4 format
Adding initial unit tests
Updating readme

Looks good to me!


Blog setup: fetch owner by role

There is a little inconsistency in how we fetch the owner user on blog setup.
The inconsistency is between local password authentication and remote authentication.
https://github.com/TryGhost/Ghost/blob/master/core/server/api/authentication.js#L81
^ We fetch the owner via the role.
https://github.com/TryGhost/Ghost/blob/master/core/server/auth/auth-strategies.js#L135
^ We fetch the owner via the slug.
I think fetching the owner by role is less error prone.
Because as soon as there is one update to the owner user, it might happen that the owner slug changes.

Closed by #8784


Connectivity checks even from an ICE-lite agent

I believe that the ICE agent is performing connectivity checks even though I've set the lite flag to true in the AgentConfig used to create it.  If I understand it correctly, ice-lite agents don't generate connectivity checks: https://datatracker.ietf.org/doc/html/rfc8445#section-2.5
Having a lite agent not generate connectivity checks is useful when using unsignalled WebRTC connections, because then one agent doesn't need to know the ICE password (or potentially even the username) of the peer that is connecting to it.
I tried to make a minimal example: https://github.com/evan-brass/webrtc-rs-ice-bug/blob/master/src/main.rs
Which gave me a network trace like this:

The second Binding request is from Peer1 (the lite-client) to Peer2 (the full client).
I'm still not-very familiar with the webrtc-ice library's code, but I believe that perhaps a check is needed within the ControlledSelector implementation for AgentInternal?: https://github.com/webrtc-rs/ice/blob/8a41f856b73fcceb1c12bac08eff2d877579db82/src/agent/agent_selector.rs#L539  I've also seen a browser send a USE_CANDIDATE attribute on the first binding request which resulted in a the ice-lite agent sending a binding request without sending a binding response: https://github.com/webrtc-rs/ice/blob/8a41f856b73fcceb1c12bac08eff2d877579db82/src/agent/agent_selector.rs#L535
I'm still new to ICE and the protocol level of WebRTC so if I'm misunderstanding the protocol, just let me know.
Thank you,

Can you investigate what happens in Pion's ICE implementation since this is a port of that project.

I don't know Go very well, but I'll give it a try this weekend.

There should be a similar example in Pion that you can just run, so Go knowledge shouldn't be too important

Hmm, I guess Pion's ICE implementation behaves the same way when I port my example to Go+Pion.

I must not be setting the lite parameter correctly, then?  Do I need to set any other agent_config parameters to make it use lite selection?
Or maybe I have the wrong idea about what constitutes a connectivity check?  Is the binding request not actually a connectivity check?
https://github.com/pion/ice/blob/2faac6229960d898132ecf8cc75c18166704470d/agent_config.go#L105-L106

There is however a lite selector implementation in Pion that I don't see in WebRTC-rs: https://github.com/pion/ice/blob/4669f6667393590d3e67fcc27379679d9f8f42d3/selection.go#L277-L292
I guess the code for that was merged into the controlling / controlled selector when it was ported to Rust.

I would need to re-read the RFC to recall what's supposed to happen here. If you think the selector might help with this could you send a PR? Alternatively if you believe we are violating the RFC can you send a PR with the changes and outline the relevant parts of the RFC that we aren't adhering too?

We have migrated this crate to the monorepo(webrtc-rs/webrtc) please re-open this issue over there.


Constraint Mux1H to provide deactivated output when given a zero-hot selector.

https://github.com/chipsalliance/chisel3/blob/1c5d1b5317a0c9fe7ef9d15138065a817380a1e4/src/main/scala/chisel3/util/Mux.scala#L24
The documentation for Mux1H says "If zero or multiple selectors are set the behavior is undefined", however, the methodology for implementing 1-hot muxes automatically makes the output all zeros when none of the inputs are selected by the selector bits (aka 0-hot selection). I've tested this with Vec[Bool] and Vec[UInt] and the current code is operating as I described. I propose the documentation support this case officially.

If Mux1H is used with single-element vectors, this isn’t true: the lone data is returned. “Unspecified” captures both behaviors.
But I suppose there’s no harm in documenting both the 1-input and (n>1)-input cases explicitly, rather than leaving it unspecified, since it seems extremely unlikely we’d be tempted to change to an alternate unspecified behavior at this point.

Unfortunately, one of the four apply methods has different behavior than the others for the 1-input case: https://github.com/chipsalliance/chisel3/blob/1c5d1b5317a0c9fe7ef9d15138065a817380a1e4/src/main/scala/chisel3/util/Mux.scala#L30
This one arguably shouldn't exist: it's actually less code to inline the expression (sel & in).orR in this case anyway.  So, my proposal is to deprecate this one, and after it's gone, we can document the unspecified behavior.

If Mux1H is used with single-element vectors, this isn’t true: the lone data is returned.
Is there a reason this is preferred? Seems to me that a 0-hot input is currently considered invalid. The single element vectors appears to be handled by a special case in oneHotMux (correct me if I'm wrong). Could that case be made to return zeros then the functionality for 0-hot select would be specified and accurate for all cases?

This one arguably shouldn't exist: it's actually less code to inline the expression.
I don't agree with this. Using the name Mux1H describes intent and makes the code readable. If one wrote (sel & in).orR instead, another reader would find it hard to understand what is happening. Not everyone knows that 1-hot muxes are and/or trees, but everyone should know what a Mux is.

Is there a reason this is preferred?

Yes, it reduces area and latency.

I don't agree with this. Using the name Mux1H describes intent and makes the code readable. If one wrote (sel & in).orR instead, another reader would find it hard to understand what is happening. Not everyone knows that 1-hot muxes are and/or trees, but everyone should know what a Mux is.

I think that’s a defensible POV, but if we retain this method and its outlier semantics, we’re probably going to leave the documentation as-is and close this ticket.

That is probably the best solution. For my purposes, I have unit tests that verify the "undefined" behavior for my project and I'll rely on it. If the implementation underneath changes in a future causing the 0-hot case behavior to change, my tests should catch that.


Add initial (very limited) support for line reduce scatter

Ticket
#11178
Problem description
We don't support line reduce scatter
What's changed
This commit adds the initial support for reduce scatter. However, only a
few cases are functional. Future work will improve correctness across
more cases.
Line Reduce Scatter Algorithm
The algorithm for line reduce scatter will send the minimal ammount of
data over each line and out of each chip. All diagrams are for an
example 4-chip line reduce scatter.
 For example, the first output is generated by
reducing the top chunk of each input tensor.
The reduction is performed by having each chip forward its input to its
neighbour. For chips that are not at the end of the line, they reduce
with their input and forward.
      |-|    |-|    |-|    |-|
      |#|<---| |<---| |<---| |
      |-|    |-|    |-|    |-|
      | |--->|#|<---| |<---| |
      |-|    |-|    |-|    |-|
      | |--->| |--->|#|<---| |
      |-|    |-|    |-|    |-|
      | |--->| |--->| |--->|#|
      |-|    |-|    |-|    |-|

Chip   0      1      2      3

However, note that each arrow from the diagram heading out of of a chip
in a given direction shares ethernet resources for all other arrows
heading in the same direction from that chip. This means there is
inherently serialization here. For that reason, we schedule the chunks
in some way.
The general scheduling strategy is to send the chunks that are furthest
from the final reduce output first and step through chunks that are
incrementally closer to the final output.
Each direction from a chip can be processed independently.
The diagram below is annotated with the "timesteps" when each chunk is
sent. Each timestep is marked relative to the chunk source.
      |-| t=0|-| t=0|-| t=0|-|
      |#|<---| |<---| |<---| |
      |-|t=2 |-| t=1|-| t=1|-|
      | |--->|#|<---| |<---| |
      |-|t=1 |-|t=1 |-| t=2|-|
      | |--->| |--->|#|<---| |
      |-|t=0 |-|t=0 |-|t=0 |-|
      | |--->| |--->| |--->|#|
      |-|    |-|    |-|    |-|

Chip   0      1      2      3

Finally, not that the final output requires a reduction from both
directions. Given that the two directions of the line are executing
completely indepdently, we require some sort of merge operation. At the
time of this commit, the merge strategy is to designate a master and
slave reducer direction. We arbitrarily choose the 'right' or
'clockwise' direction as the master.
The master direction will write its output to the output tensor but note
that this will only be a partial output. The slave direction will read
from the output tensor to merge with the data from producer chip. It
will read from the output tensor based on a credit passing from master
(implemented via semaphores)
     -------Input Tensor
     |
     |
     |
     |
     |
     |      |---------|----------
     |----> |         |         |
            | Reader  | Sender  |---
  From EDM  | (master)| (master)|  |
 ---------> |         |         |  |
            |---------|---------|  |
                                   |
                                   |
        |------  Output Tensor <---|
        |                 ^
        |                 |---------
        |                          |
        |   |---------|----------  |
        --> |         |         |  |
            | Reader  | Sender  |--|
  From EDM  | (master)| (master)|
 ---------> |         |         |
            |---------|---------|

As a part of the line reduce scatter implementation, new CCL
componenst were added: ccl send and ccl command generators/readers.
The ccl_send (kernel) was used to implement the starting ends of the lines
(i.e. the first senders). Although the ccl_send provides more generic
send capabilities than line reduce scatter currently requires, it was
chosen because it is a basic building block also for future CCL
send/recv "operations" and higher level CCL programming models.
CCL Send (Kernel)
The ccl_send kernel acts like an interpreter of CCL commands.  The command specifies some information about the tensor
(shape, slice/view shape, view offset, etc.).
CCL send is capable of executing multiple commands back to back. In the
context of line reduce scatter, the ccl_send implements the separate
sends of the fractured chunks on the left and right ends of the line. To
do this for a line reduce scatter, we invoke n commands where n=#chips
in the line. Future commands will let an invoker specify this basic
pattern as a single command.
Looking at the third diagram that outlines the timesteps for each chunk,
for the left/right tensors, each timestep directly maps to a separate
ccl command.
CCL Command Generators/Readers
To facilitate command generation, initial components have been added to
let the host serialize commands for the the ccl_send kernel.
Correspondingly, command unpacking logic is also specified for each
command. This is used to help simplify command generation for the host.
Note that ccl_send as a standalone kernel and operation is experimental
and has several limitations:

Slice reads currently constrained to page aligned slices
Host command generation doesn't support proper 4D shape support
(although the kernel side will internally represent shapes as 4D)
Only one command is currently supported (send tensor slice to EDM)

Checklist

[x] Post commit CI: https://github.com/tenstorrent/tt-metal/actions/runs/11043482824
[x] t3000 frequent: https://github.com/tenstorrent/tt-metal/actions/runs/11043294192
[x] t3000 model perf: https://github.com/tenstorrent/tt-metal/actions/runs/11043404805

Not completely green but neither is main. Seeing the same failures as on main and failures are unrelated to changes here.

[x] t3000 unit test: https://github.com/tenstorrent/tt-metal/actions/runs/11043407305
[x] TG frequent: https://github.com/tenstorrent/tt-metal/actions/runs/11043423582
[x] Blackhole Post commit (if applicable)
[x] New/Existing tests provide coverage for changes

@SeanNijjar and i synced offline, there's a part 2 coming.


CMake Tools Language Services - Hover (Quick Info)

Brief Issue Summary
Add support for hover - quick info in CMakeLists.txt files.
CMake Tools Diagnostics
No response
Debug Log
No response
Additional Information
No response

Overrarching topic issues: #534 #93


A click on "Reload Live Bookmark" should only close the context menu, not the whole bookmark menu.

When right-clicking on a feed, hovering "Livemarks" and then clicking on the "Reload Live Bookmark" button, the feed gets reloaded, but the whole menu is closing.
In the past, when Firefox could still handle feeds by itself, the "reload all" button (that reloaded all feeds, not just one) only closed the context menu, not the whole bookmark menu. Then you could directly see the new items. But with Livemarks you have to re-click through the whole menu again.
The copy or cut buttons, for example, in the same context menu do not close the whole bookmark menu, but only the context menu, so I assume this must be possible for extensions as well.

Any news here?

Nothing we can do here.

Is this a limitiation of the extension API? Because Firefox itself has non-menu-closing context menus, like described in the issue.

Yes there doesn't seem to be a way to specify that the bookmarks menu shouldn't be closed when clicking something on the context menu. We use this API: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/menus/create.


feat: pages

:tada: This PR is included in version 0.86.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Trophies for Vita and Ps3 works

Yeah, it is sub thing that is actually works!

Included in https://github.com/ZeruLight/Erupe/commit/796927ac61ee33d5a773515f6391b151e7560c22.


feat: change version number

Thanks @Light2288  please create release here as well https://github.com/ministero-salute/it-dgc-verificaC19-android/releases


Add basic github workflow for CI

This addresses https://github.com/CesiumGS/3d-tiles-tools/issues/19 . But there recently have been reports about TypeScript compilation failing in the validator at https://github.com/CesiumGS/3d-tiles-validator/issues/272 , and since this part of the code was moved here (from the validator), the build will likely fail here as well. I could not (yet) reproduce the build failure locally, but will try to investigate it ASAP.

The comment at https://github.com/CesiumGS/3d-tiles-validator/issues/272#issuecomment-1518924102 describes the (likely) reason for the upcoming build failure, and a workaround. That workaround will/should probably be done in a dedicated PR, after this one is merged, to be sure that the workaround works... around.


Provide annotation support to  override auto generated operationId from OpenAPI spec

Description:
In relation to this issue https://github.com/ballerina-platform/openapi-tools/issues/943 , we are suggested to use annotation in case the user needs to override the generated operationId with their value.
Describe your solution(s)
Provide resource level annotation to keep the resource metadata that needs operation need to be included.
import ballerina/openapi;
import ballerina/http;

service /v1 on new http:Listener(9090) {
    @openapi:ResourceInfo {
          operationId: "user provide operationId"
    }
    resource function post pet (@http:Payload json payload) {
    }
}

When providing the operation Id annotation, we need to implement it to return validation error in case two resource functions have the same operationId value user given.
@hevayo , @shafreenAnfar , @chalitha1989 , @NipunaRanasinghe Looking forward to hearing your suggestions on any changes or improvements that can be done 🙂
Related Issues (optional):
https://github.com/ballerina-platform/openapi-tools/issues/943 , https://github.com/wso2-enterprise/choreo/issues/12436

LGTM

@lnash94 What is the status of this? Did we start implementation?

@lnash94 What is the status of this? Did we start implementation?

Currently, we are going to redesign the client generation and still, it is in the designing phase, therefore we won't able to proceed with the annotation implementation. Because of that to reduce the user pain point we suppose to normalize the generated operationID via this issue https://github.com/ballerina-platform/openapi-tools/issues/943 , we have already provide the fix for this, will be available with ballerina 2201.1.0 version 🙂


Why has Bootstrap 4 changed to using rem?

I know a lot of people like to use rem these days, but I'd like to know what is the reason behind changing to using it over px?
Not only is pixel units far easier to work with, research shows that they should be used over rem units.
https://mindtheshift.wordpress.com/2015/04/02/r-i-p-rem-viva-css-reference-pixel/
http://stackoverflow.com/questions/11799236/should-i-use-px-or-rem-value-units-in-my-css
Seems a little odd to go backwards in V4 when V3 was using pixels.

research...or just opinion pieces? the one aspect that rem still handles that px doesn't are user settings and user agents which don't use 16px font size baseline. https://nicolas-hoizey.com/2016/03/people-don-t-change-the-default-16px-font-size-in-their-browser.html

Look I don't pretend to be an expert on these things, but I wouldn't say those links I posted are opinion pieces; the first one most certainly based the article on a lot of research that he did and made some strong points and it most certainly isn't the only article stating to use px over rem.
Either way you look at it IMO there isn't a massive difference between the two in accessibility for the user but px is certainly far easier to work with for the developer.

For me personally, rems provide a lot of good while working.

It's easier for me to think of margin and padding values as rem values because it allows me for simpler understanding of them and keep them consistent. Example - 1 unit of margin all around or twice the margin around.

With rems, it's easier to get text behave responsive because it's dependent only on one declaration at the root element. While customizing bootstrap for my own project, i had to scale text in every component down from 16px base to 14px base and it was a hell lot easier to do with rem.

In general practice, I have realized that it allows me to have a more robust grid and sizing of elements with less number of unique values.

If DPI would be constant across different devices across time, px would be OK to use. But when you check new 4K screens, most of them are packing twice more pixels in the same space, and basing design on pixels means everything will be too damn small.
Of course, there are exceptions, like ~40 inch monitors having 4K res which keep DPI, and increase both size and resolution, but they are minority.
Use of any kind of relative units is in long term better solution than use of unit which size is highly dependent on technology. Yes, web sites need to change from time to time, myself and my colleagues need to be paid, it's not a good idea to make some web site too good so owner won't need to invest some money for web devs services once in a while, but counting on pixels simply feels wrong.
I understand why someone feels px is the right unit (for some limited cases it IS the RIGHT UNIT), but once again, number of such cases is LIMITED. I'll give an example, to put this to rest:
Lets say you are making design for an in-house app in some rather big company which for some reason is using quite standardized equipment, like some big bank, which before used Oracle Forms or similar "toolkit" with fixed grid size, and they want exactly the same layout and fonts and ...., which is not a problem for them as they have specification for equipment they buy in large quantities, and there is no odd equipment (monitors) with DPI higher or lower than usual, they ALL have the same monitors of same size which are working on same resolution, and if a new app for this would not be made in Java but in JavaScript or Python and you would have an opportunity to make UI with CSS/JS framework of your choosing, using relative units in your design would be quite probably poor choice, as you would get expected result sooner and easier to be according to specification if you would be using px.
But as I wrote at the beginning, number of such use cases where usage of px is not just better, but preffered way is severely LIMITED.

Thanks for your post @stemd but I believe that first article covered the DPI aspect.

TLDR disclaimer (forgot it last time :))
You are right @cyphix333 , first article mentioned:

reference pixel is the visual angle of one pixel on a device with a pixel density of 90dpi and a distance from the reader of an arm’s length

Only thing is that standards or proposed standards in IT are not written in stone, sometimes software vendors are promoting new standards, and sometimes they are just implementing them.
If unit used in CSS is px, which stands for reference pixel, that IS good enough, I agree, if all vendors are implementing that.
As I hear, not every OS and app is scaling to 4K the same. Idea is - apps need to scale according to technology changes, and how they will do it (which units will scale) is irrelevant.
It is surely misleading if browsers scale pixels and OS level GUI toolkits like Qt, GTK, WxWindows doesn't, but as newer versions of both Qt and GTK both implement CSS styling of its components (Qt5 and GTK3 do, it's quite nice, actually), I suppose if all browsers will support "reference pixel", OS toolkits will probably do likewise, eventually. Time will tell.
Disclaimer: OS toolkit discussion is a bit out of scope here as Bootstrap is CSS framework not GUI toolkit, I'm aware, but from our (users) viewpoint, we just want that software we use scale well, and when that does not happen, we are forced to dig through the registry or settings and tweak. That should not happen, and only way is if some relative unit is clearly both defined as relative to screen size and scaling with DPI change. Hopefully if we once switch from 4K to 8K or something, then everything will scale without end users ever noticing.

...when you check new 4K screens, most of them are packing twice more pixels in the same space, and basing design on pixels means everything will be too damn small.

@stemd
Uhhh..... have you ever heard of device pixel ratios?

@Jakobud Wrong address, ask authors of toolkits why their software is not scaling.

closing this, as the discussion is getting circular. point is: rem takes the browser/user's base font size (usually 16px, but not always) as a starting point. if a user has set their browser to explicitly use a different default font size, or if it's a device/browser which sets a different font size, this unit will allow for it.
px in browsers is "CSS pixel", not hardware pixel, so 4K/8K are a red herring in most cases, since the browser (provided the OS is sane) will have whatever the browser/device manufacturer/user has deemed to be an appropriate resolution/dpr and the CSS pixel will correspond to whatever the device pixel ratio demands. yes, in most modern browsers zooming by default also enlarges the size of the CSS pixel/modifies the browser's interpretation of the dpr, so scalability is nowadays also achievable quite comfortably with px units. however, the missing part is that px units don't take into account browser/user default font size. as such, rem still win out.

A while back I proposed that Bootstrap use a SASS mixin that converts px units to rem's, as a middle-ground solution. That way people have the comfort of using the familiar px values but all the resulting css will be using rem. Zurb did this with Foundation a while back and it was very nice. But I think the main Boostrap guys did not care for this approach for some reason. I was never sure what the problem was.

Just want to add one really really cool thing about REM.
You can create websites that scales the design perfectly to the screen or window size. Not sure what to call it, maybe scalable design since liquid and elastic is already taken.
Say we have the following CSS:
html {
  font-size:62.5%; /* not really needed but make calculations from design to screen 1 to 0.1 */
}
@media screen (min-width:640px) and (max-width:1000px) {
  /* here we say scale everything between 640px screen width to 1000px screen width
     to look the same (e.g. scale to the exact same proportions) */
  font-size: 1vw;
}

There is a natural lower and upper bound for when design is too small or too big, but this technique makes everything in between look exactly like the designer envision
Check it out here (between 640px and 100px): https://www.firefund.net
I have used this technique for years (since 2013) and have only en-counted one bug in Chrome and Safari, when you scale below font-size below 9px. So only use this for tablet to desktop design.
Another scale everything demo (which glaringly show the bug in blink/webkit): http://dotnetcarpenter.bitbucket.org/bugz/rem-bug/

FWIW scaling using just vw/vh results in pages that simply can't be scaled/zoomed on desktop browsers (since zooming changes the browser's "virtual" pixel density, it changes the viewport size - any pure vh/vw based font sizes render exactly at the same size - since the window's proportions etc haven't actually changed, and those measurements are meant to maintain the relationship to the viewport).

One last gotcha. Don't ever set width, height or border to 0.01rem; instead of 1px. Browsers will eventually round it down to 0px (things will disappear). You can use min-height: 1px; and min-width: 1px; in conjunction with  0.01rem; to remedy though.

@patrickhlauke Not if you use it within media queries. Then you can scale as usual. Try it out on https://www.firefund.net

Yes even when you use it in media queries - for those sizes where the viewport width falls between those min/max values (that zone where your sizing is purely in vw/vh) the text remains unzoomable. once you zoom up/down enough to escape that zone, then yes normal sizing works again.

Yes you're right. Guess a another gotcha is that you should always use media queries with a small range. I only use it for translating desktop design to tablet sizes, and the technique is successful at that. Never had any problems zooming on touch screens, not sure if that is because they out-range the media query so fast or they just don't change the meaning of vw/vh.
Another option is to use a small script. I did that the first 2 years.

A while back I proposed that Bootstrap use a SASS mixin that converts px units to rem's, as a middle-ground solution. That way people have the comfort of using the familiar px values but all the resulting css will be using rem. Zurb did this with Foundation a while back and it was very nice. But I think the main Boostrap guys did not care for this approach for some reason. I was never sure what the problem was.

Yeah, seems a bit silly not to provide one.

Never had any problems zooming on touch screens, not sure if that is because they out-range the media query so fast or they just don't change the meaning of vw/vh

The latter. On mobile/tablet devices, zooming does not change the actual width/height of the layout viewport (which is why you get horizontal/vertical scrolling) nor the "virtual" dpi, while on desktop browsers zooming both are modified.

Guess you're not really sold on the technique @patrickhlauke?

everyone is missing one important thing, that padding and margins are also set in rems in bootstrap4 and bootstrap5. rem for font sizes is good, but, for padding and margin it is bad. This is explained in detailed here<EMAIL_ADDRESS>If a user wants to increase the font size as well as the gaping/gutters, they can just ctrl + and ctrl - vice versa. Some users just want to increase the font size without affecting the layout(gaps/gutters). This can be achieved by on websites with px based paddings and margins by changing default font size in the browser setting.  bootstrap takes away this from the users by using rem in padding and margin.

I agree, when we increase font size - screen size does not increase, we are still limited by tiny phone screen. And if padding is linked to font size - we can get counter productive results like shown in this example. There are two areas here, bottom one is using rem for padding, which takes away space that could be taken by text.


AST Resolver Improvements

Idea here is to get towards forward type declaration / resolution as close as possible, including if there's anything that I catch that needs to be adjusted (but nothing big).

Entry source unit discovery improvements.
Improvements in discovery source unit exported symbols.
Helpers to get from source unit Contract() and BaseContracts().
LibraryNode and InterfaceNode are just plain Library and Interface.
LinearizedContracts and ContractDependencies improvements.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

87 of 172 (50.58%)  changed or added relevant lines in 8 files are covered.
2870 unchanged lines in 57 files lost coverage.
Overall coverage decreased (-36.4%) to 39.519%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

ast/inheritance.go
0
1
0.0%

ast/imports.go
16
20
80.0%

ast/base_contract.go
0
6
0.0%

ast/source_unit.go
3
9
33.33%

ast/library.go
8
20
40.0%

ast/reference.go
39
56
69.64%

ast/interface.go
0
39
0.0%

Files with Coverage Reduction
New Missed Lines
%

ast/license.go
1
93.75%

ast/function_call.go
2
58.38%

ast/if.go
3
85.07%

ast/return.go
3
84.13%

ast/variable.go
3
77.63%

ast/using.go
4
65.63%

ast/contract.go
5
74.1%

ast/assignment.go
6
66.88%

ast/root_node.go
6
50.82%

ast/expression_statement.go
9
30.56%

Totals

Change from base Build<PHONE_NUMBER>:
-36.4%

Covered Lines:
3139

Relevant Lines:
7943

💛  - Coveralls


Hello World

Name: Nick Nicholas
GitHub: @your_username
Twitter: @nichbarry
Linkedin: link
Website: yourblog.com

Completed Courses
Name of the Section

Course
Files

Course Name
link

Please consider using this issue.


The use of JSON_FORCE_OBJECT flag when encoding props in blade template

Hey 👋  Awesome project and thanks for open-sourcing it!
I was just wondering, is there a particular reason for using JSON_FORCE_OBJECT flag on json_encode() in the blade template?
<div id="app" data-component="{{ $component }}" data-props="{{ json_encode($props, JSON_FORCE_OBJECT) }}"></

When fully loading f.e. contacts page, Vue shows prop validation errors as the links inside contacts prop is object because of it, and Pagination component expects it to be an array.
Thanks again. Cheers!

Good catch! I've fixed this:<PHONE_NUMBER>468ffee03c59159288aa7ae279180d
I was using JSON_FORCE_OBJECT for situations where there are no props, json_encode() results in an empty array ([]), but Inertia expects an object ({}). The problem is JSON_FORCE_OBJECT doesn't only apply to the top level, but all nested data, which is not what we want. Simply casting the $props array to an object solves this.


Addresses the vertical misalignment between the branch name and the "…

…Default" badge in the list group item
Closes #7890
Change Description
Addresses the vertical misalignment between the branch name and the "Default" badge in the list group item.
Background
This issue can cause a customer to mistakenly delete the wrong branch.
Bug Fix
If this PR is a bug fix, please let us know about:

Problem - The delete icon and the name of the branch are not aligned in the UI. The name of the branch is aligned to the top of the cell, while the icon is aligned to the bottom of the cell. This can cause someone to delete the wrong branch by mistake.
Root cause - There is no alignment for the object
Solution - use flexbox to achieve proper vertical centering of both elements

Logs, outputs, screenshots of changes if applicable (CLI / GUI changes)
Contact Details
How can we get in touch with you if we need more info? (ex<EMAIL_ADDRESS>

Fix for the fix

@iddoavn Is this ok?

Fix for the fix

@iddoavn Is this ok?

Amazing. Thank you!


Docs build failure (API docs gen)

Noticed in #96
Warning, treated as error:
/home/runner/work/MELODIES-MONET/MELODIES-MONET/docs/api/melodies_monet.rst:26:autosummary: stub file not found 'melodies_monet.melodies_monet.driver'. Check your autosummary_generate setting.

This can happen when a submodule imports the top-level module (https://github.com/sphinx-doc/sphinx/issues/8963), but we don't have that afaict. And it was working in the last develop CI run, with the same Sphinx version as in this failure (4.5.0). So I am pretty confused.
I was able to reproduce locally only after recreating the conda environment.
The conda environment before this failure is not identical to when it starting failing, but I haven't determined yet what could have caused this.
Differences in versions

name
version_old
version_new

asttokens
2.0.5
2.0.8

executing
0.9.1
0.10.0

giflib
5.2.1
-

libsqlite
-
3.39.2

libwebp
1.2.3
-

libwebp-base
1.2.3
1.2.4

nbconvert
6.5.2
6.5.3

nbconvert-core
6.5.2
6.5.3

nbconvert-pandoc
6.5.2
6.5.3

openjpeg
2.4.0
2.5.0

partd
1.2.0
1.3.0

pytz
2022.1
2022.2.1

pyzmq
23.2.0
23.2.1

setuptools
63.4.2
65.0.0

stack_data
0.3.0
0.4.0

xz
5.2.5
5.2.6

This is fixed in develop (#98).

These changes have been merged to main on April 14, 2023, so I'm closing this issue.


http://scratch.mit.edu/projects/116574928/

Doesn't want to play sound =/
http://scratch.mit.edu/projects/116574928/
http://phosphorus.github.io/#116574928
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

The reason for this is that some browsers (like chome and firefox) cannot decode the media files that scratch store sound in. This has been mentioned many times before, so please check other issues to see if yours is something not mentioned before, before you post.


update ss58 type to u16

Since we've run out of ss58 the prefix size was increased to u16 but Substrate didn't account for this yet. This PR should address this. I have a couple of questions regarding this:

Do I need to increase spec_version of Substrate (Polkadot,Westend...)? The type changed from u8 to u16 it's stored as a constant, is that something that is actually stored in the genesis or is it provided by the node / runtime itself? If it's stored in genesis do we need to do a migration?
Do we need companion to Polkadot?
Do we change all the types in tests? (u8 fits into u16 but it might be nicer to have it synced?)

CC @jacogr

Since we've run out of ss58 the prefix size was increased to u16 but Substrate didn't account for this yet. This PR should address this. I have a couple of questions regarding this:
1. Do I need to increase spec_version of Substrate (Polkadot,Westend...)? The type changed from u8 to u16 it's stored as a constant, is that something that is actually stored in the genesis or is it provided by the node / runtime itself? If it's stored in genesis do we need to do a migration?

Nothing needed.

2. Do we need companion to Polkadot?

Probably not.

3. Do we change all the types in tests? (u8 fits into u16 but it might be nicer to have it synced?)

Get is implemented in a way that it can do the conversion, so I think it is fine to keep the code as is.

I would expect the Polkadot runtimes to not compile on the type change so I think we do need a companion.

I would expect the Polkadot runtimes to not compile on the type change so I think we do need a companion.

Why?

I would expect the Polkadot runtimes to not compile on the type change so I think we do need a companion.

Can I do the companion to speed this up if needed?

I would expect the Polkadot runtimes to not compile on the type change so I think we do need a companion.

Why?

Because of the type mismatch between the current value in the runtime and what would now be expected from frame_system.
But, reading this:

Get is implemented in a way that it can do the conversion, so I think it is fine to keep the code as is.

Does that mean I can do the following?
parameter_types! {
    pub const MyU32Param: u32 = 50;
}

impl some_pallet::Config for Runtime {
    type MyU64Param = MyU32Param;
}

impl<I: From<$type>> $crate::traits::Get<I> for $name {
			fn get() -> I {
				I::from($value)
			}
		}

Ah, I can! Cool.
Never mind, then.
Sorry for the noise.

bot merge


Add Non-Markovian Reward Modelling (2022)

Adds the NeurIPS 2022 paper "Non-Markovian Reward Modelling from Trajectory Labels via Interpretable Multiple Instance Learning" (Early et al.)
This paper uses the terminology "Reward Modelling" instead of RLHF but I believe it is still relevant to this excellent list of papers.
Disclaimer: I am the lead author on this work.

Thanks for your contribution to awesome-RLHF repo! BTW, please modify the name of PR like our commit format, e.g. feature(author anme): add non-markovian reward modelling paper info, then we will merge this PR.

Thanks for your contribution to awesome-RLHF repo! BTW, please modify the name of PR like our commit format, e.g. feature(author anme): add non-markovian reward modelling paper info, then we will merge this PR.

Thanks for getting back to me. I've updated the commit message and PR name.


Change LaserChess link in Featured Components

Hey, we recently redeployed the Laser Chess widget to another account. Can you please change the link at the Featured Components? Here is a new link: https://near.org/ostolex.near/widget/LaserChess3D

Hey sorry to bother, but this issue marked as closed, but it doesn't seems to be true


Multiple Simultaneous Output/LED Hardware

Feature request
It'd be nice to be able to add multiple LED Hardware outputs & have them update simultaneously.

What problem does this feature solve?
Being able to use Hue Entertainment for Hue Plays alongside a traditional LED SPI or PWM. This would be useful to have LED strips driven behind the TV & ambient lighting up other parts of the room using Hue Entertainment.
What does the proposed API look like?
n/a
How should this be implemented in your opinion?
Update the LED Hardware section to allow multiple outputs (potentially restricted by the type, ie you can't run two SPI's at the same time)
Are you willing to work on this yourself?
No, sorry I don't know too much C++

you ever noticed the different instances you could use with different hardware?

Oh, I didn't notice that option 🤦  Maybe nesting the LED hardware into the same page/subpage of an instance would make it a bit more obvious. It doesn't seem very intuitive.
I'll just quietly close this....


Added example VCF from spec (v4.3)

Might add further (small) annotated vcfs on that folder later... hope the naming/directory structure makes sense.

Thanks Roman!


on-board autom8 members

What
allow autom8 members access to login to concourse with github
How to review
Verify that it's ok for these people to login to concourse
Who can review
Not @chrisfarms (nor probably any of the users listed)

Can y'all go in https://github.com/alphagov/paas-bootstrap/blob/master/manifests/concourse-manifest/github_auth/dev_ci_additional_users.yml


fix(tooltipIcon): show text on hover

Closes https://github.com/IBM/carbon-components-react/issues/1078
TooltipIcon uses aria-label to show the text. We were placing the text inside the title attribute.
Changelog
Changed

Use aria-label instead of title

@asudoh do you want to approve


fix(deploy) Increase container security

Change default values to run containers with read-only file system and non-root user for both kong and postgres deployments.

What this PR does / why we need it:
Which issue this PR fixes

Fixes https://github.com/Kong/charts/issues/753

Special notes for your reviewer:
Checklist

[x] PR is based off the current tip of the main branch.
[x] Changes are documented under the "Unreleased" header in CHANGELOG.md
[x] New or modified sections of values.yaml are documented in the README.md
[x] Commits follow the Kong commit message guidelines

Do you know of anything that's functionally different in terms of enforcement or compliance validation if like settings are set at the Pod versus container level?
The rough justification for having defaults only in the container section is more or less what you've mentioned: we apply those context constraints to any container the chart itself manages, and any other container in the Pod is an unknown sidecar (with Kuma having a special status as a well-known sidecar that we try to accommodate by default). Pod-level settings might thus break those unknown sidecars and AFAIK there's no difference when applying the same setting at both levels. If there's something that's Pod-level only, we should consider that in isolation.
The Postgres settings should be fine, though we've normally mostly ignored it since it's kind of a legacy wart we discourage use of and plan to ultimately remove.

As you already wrote, the pods securityContext applies to all containers in the pod while the securityContext of a container only applies to the single container. There are some fields that can be only configured in the securityContext of the pod (fsGroup, fsGroupChangePolicy, supplementalGroups and sysctls) and some fields that are exclusive to the containers securityContext (allowPrivilegeEscalation, capabilities, privileged, procMount and readOnlyRootFilesystem). Fields runAsGroup, runAsNonRoot,  runAsUser, seLinuxOptions, seccompProfile and windowsOptions are supported by both.
In terms of compliance, IMO it would be better to set most of the stuff in the pods securityContext. But that breaks injected containers. Not sure how much thought was put into kube-linters rules which we're trying to make happy in this PR.

What issue were you seeing with Kuma, on which version, and with what install configuration?

I was running tests locally with the versions specified in the test files. As said the injected Kuma initContainers were not happy with the runAsNonRoot setting as these were set explicitly to run with uid 0

Would the Pods come online if you manually added a runAsNonRoot: false to the kuma initContainers? I think that should work in namespaces that permit it.
Trying to do my own tests, but have been frustrated by some other issue interfering with my ability to run the transparent proxy mode locally.

Would the Pods come online if you manually added a runAsNonRoot: false to the kuma initContainers? I think that should work in namespaces that permit it.

As long as the pods securityContext does not already specify runAsNonRoot: true, it should work. But I did not test this.

@rainest is this still something worth merging?


Serve images as static.

Instead of serving product's image as blob, give link to the image in static folder.

please specify the path, e.g. /static/assets/images or however you want to structure it


Auto parse .yaml and .yml

Looks like you are autoparsing files with a .yml extension as YAML, however YAML themselves seem to like .yaml See: http://yaml.org/faq.html
This is a pretty easy fix to make, I think even I can handle it as a non-python programmer :)
https://github.com/ManiacalLabs/BiblioPixel/blob/1255ecd3c9bc91dcafecff493e152934e0e24c87/bibliopixel/util/json.py#L19
Better would maybe be to allow a flag to force a type of file encoding

We are actually moving to parsing everything with the Yaml parser, so all of this will be moot in a very short time!  (There's a pull request in flight as we speak...)
Stay tuned.  :-D


BliXemSA's Profile

Adding profile for Niel Gerber aka BliXemSA

Hi @BliXemSA
You need to fork this repo under your account and then make your changes there. For more help, have a look at https://help.github.com/articles/fork-a-repo/
Once you're done, you can then open a Pull Request from your version to this main repository


PR: Add ability to modify other plugin options

Description of Changes

[x] Wrote at least one-line docstrings (for any new functions)

Other plugins can now change other plugin options. This is needed to group some settings that are better displayed on a different configuration page of the config dialog. Should be used with care and sparingly!
Issue(s) Resolved

Fixes #
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: @goanpeca

Please report it in a separate issue, it's probably a bug.


FINERACT-2081: Update password policy

Description
Update password validation to ensure consistency across application layers.
Ignore if these details are present on the associated Apache Fineract JIRA ticket.
Checklist
Please make sure these boxes are checked before submitting your pull request - thanks!

[x] Write the commit message as per https://github.com/apache/fineract/#pull-requests

[x] Acknowledge that we will not review PRs that are not passing the build ("green") - it is your responsibility to get a proposed PR to pass the build, not primarily the project's maintainers.

[x] Create/update unit or integration tests for verifying the changes made.

[x] Follow coding conventions at https://cwiki.apache.org/confluence/display/FINERACT/Coding+Conventions.

[x] Add required Swagger annotation and update API documentation at fineract-provider/src/main/resources/static/legacy-docs/apiLive.htm with details of any API changes

[x] Submission is not a "code dump".  (Large changes can be made "in repository" via a branch.  Ask on the developer mailing list for guidance, if required.)

FYI our guidelines for code reviews are at https://cwiki.apache.org/confluence/display/FINERACT/Code+Review+Guide.

@adamsaghy I'd like to check this myself. Please do not merge yet.

Hi @galovics, I've updated the PR. Please review.


Added semi-hidden debug modal

A PR for #460
@pathmapper to not add clutter to the toolbar, how about a hidden modal. Just press ! for the debug modal (? to see the shortcuts)
Then you'll get a modal pop up with some debug options.

I figured because we don't display that these options are enabled in the UI anywhere at the moment, a novice user could accidentally enable these options. For this reason they will reset on browser refresh (we can change that later down the road)
I've also removed the url param method of enabling these feature as a part of this PR
Does that meet your requirements?
Demo: https://1280-84182601-gh.circle-artifacts.com/0/artifacts/build/index.html#0.34/0/0
Fixes #460 (...if @pathmapper likes it 😄 )

Great @orangemug !
Some thoughts:

These debug options are only working for the GL JS renderer.

If you turn a debug option on while using OL as renderer nothing happens. If you then switch the renderer to GL JS, this debug option is active but it is not checked in the debug modal anymore.

For 1. we could change the tiltle of the debug modal in something like:
Debug (only Mapbox GL JS renderer).
If 2. is not an easy fix, we could fix it later and merge this PR now.

@pathmapper give https://1305-84182601-gh.circle-artifacts.com/0/artifacts/build/index.html a try let me know what you think.

Perfect @orangemug ! :tada:

Yay!


Bump flatten-maven-plugin to 1.3.0

This fixes an issue first reported by @hasys, where he could not run mvn gwt:run on packages/serverless-workflow-diagram-editor/sw-editor/sw-editor-kogito-app.
We found out this issue was a bug (https://github.com/mojohaus/flatten-maven-plugin/issues/100) on flatten-maven-plugin, which was clashing with shade-maven-plugin, leaving the installed POM with a reference to the CI friendly version ${revision}.
We contacted the maintainers of flatten-maven-plugin through this issue https://github.com/mojohaus/flatten-maven-plugin/issues/292 and they kindly released a new version containing the fix (https://github.com/mojohaus/flatten-maven-plugin/pull/281) right away. (Thanks a lot, @slawekjaranowski 🎉)

Thanks a lot @slawekjaranowski

Works as a charm, thank you @tiagobento!


Not able to hide chart title in line & pie chart

Hi,
I am not able to hide chart title with the following code in "line" and "pie" chart.
from pptx import Presentation
from pptx.chart.data import ChartData
from pptx.enum.chart import XL_CHART_TYPE
from pptx.util import Inches

prs = Presentation()
blank_slide_layout = prs.slide_layouts[6]
slide = prs.slides.add_slide(blank_slide_layout)

chart_data = ChartData()
chart_data.categories =  ['May 01', 'May 02', 'May 03', 'May 04', 'May 05']
chart_data.add_series('Total fans', (100, 120, 130, 140, 150))
chart =slide.shapes.add_chart(XL_CHART_TYPE.LINE_MARKERS,Inches(0.7), Inches(1.42), Inches(9), Inches(2.5), chart_data).chart
#hide chart title
chart.has_title = False
prs.save("line_chart.pptx")

I'm seeing this too chart.has_title = False doesn't seem to work.

I think this is an unexpected behavior of PowerPoint. In a line chart, I think you'll find it goes away if you add a second series, i.e. it only occurs on single-series charts.
I would start by trying adding a blank chart title and see what you get.
I think if you examine the chart XML you'll find there's no title element present in these cases, rather it's something PowerPoint renders by default for single-value charts, perhaps a bug or at least a mis-feature.

Cool thanks @scanny - seems similar to #293 Currently getting round it by adding a nicely formatted custom title. But you're right it's picking the title of the first series by default.

I think if you examine the chart XML you'll find there's no title element present in these cases, rather it's something PowerPoint renders by default for single-value charts, perhaps a bug or at least a mis-feature.

That's really interesting - thanks. Good to know it's not just a bug with Python PPTX :)

We don't do bugs here :)

Should be fixed in release v0.6.13 later today.


Fix compute typos upstream

We currently have to ignore some typos in the compute discovery json. It would be better to try and fix these at the source.

Punt once.

Too hard to chase upstream. Closing.


Add quantized-mesh endpoint

Find dem tile data normally, then pass to pymartini, then pass to quantized-mesh-tile, then return

Working (at least with aws terrain tiles) as of https://github.com/kylebarron/dem-tiler/commit/a8d3cc28f859f9c316c8874c3be064614683f484


I want to propose my custom module to be listed on the Wiki Module: Custom: Simple

https://github.com/monteship/yourmeetings

Click 'Edit' and type away...


Users v0.1

Coverage decreased (-2.9%) to 72.98% when pulling cc32b41359c044556c76007dbb1772e3564acfc6 on users-v0.1 into 95c5796ee2d4777da5a3fc60ae751cf97d0e34da on master.

Coverage decreased (-9.6%) to 66.337% when pulling c73435f6632eb0b4323820c4fd8dccda4a99ff8a on users-v0.1 into 95c5796ee2d4777da5a3fc60ae751cf97d0e34da on master.

Coverage decreased (-8.9%) to 66.997% when pulling 9b8e8c5a4c88316e16ec3642bada2c388b5356ee on users-v0.1 into 95c5796ee2d4777da5a3fc60ae751cf97d0e34da on master.

Coverage decreased (-2.4%) to 73.485% when pulling bae65ace64a89907244262dad0e5501295194d50 on users-v0.1 into 95c5796ee2d4777da5a3fc60ae751cf97d0e34da on master.


PANIC when running ModelAPIs.lean

When running ModelAPIs.lean, a warning PANIC: Invalid UTF-8 string is throwed, with no meaningful cause/position information. We can try to find its cause and fix it.

Fixed in PR #114.


Add optional config parameters

Right now most of the config options are mandatory, but as we start to support more service configurations there are many options that are not relevant for every install.  We should add functionality to the config parser to have optional parameters.

Fixed by #301


Add pe-terminal package to enable go-based relay-term

This PR adds go-based relay-term package called pe-terminal which will be enabled by default, disabling the old nodejs one instead of removing it, so that we can bring up the old one incase some issue is encountered with the new one.
Note: CI builds might fail as the repo is currently private, we're working on making it public.

See a similar PR for distro-pelion-edge:
https://github.com/PelionIoT/distro-pelion-edge/pull/191

Looks fine to me.  CI is failing for a strange reason, but the root cause is probably as you said, that the repo is still private.

Pulling pe-terminal
Cloning into '/build/parts/pe-terminal/src'...
fatal: could not read Username for 'https://github.com': No such device or address
...
Failed to pull source, command 'git clone --recursive https://github.com/PelionIoT/pe-terminal.git /build/parts/pe-terminal/src' exited with code 128.
[Pipeline] }
ERROR: script returned exit code 2

@costanic I've added the source-tag and the build seems to be passing now.
This PR is ready to be reviewed.

ok, seems fine.  One request, can you please add more details in one of the commit messages as to why this package was added?  We need to understand the rationale so that we can have a follow-up conversation about what additional actions we need to take and when.  I don't think it matters which commit message you add it to because this will probably all be squashed during merge.

ok, seems fine. One request, can you please add more details in one of the commit messages as to why this package was added? We need to understand the rationale so that we can have a follow-up conversation about what additional actions we need to take and when. I don't think it matters which commit message you add it to because this will probably all be squashed during merge.

Done, added the message in last commit.

Since pe-terminal has replaced relay-term in meta-pelion-edge, I think it makes sense to remove relay-term here.


Change SerializerEncoding usage and REST proxy updates

Description

Reduces severity of logging performed by SerializerEncoding to verbose/debug.
Update handling in AsyncRestProxy and SyncRestProxy to prefer using BinaryData as the payload instead of using the older APIs on HttpRequest or converting the body to Flux<ByteBuffer>.
Add support for InputStream in SyncRestProxy.
Null or empty check the body before attempting to deserialize.

All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.


Combined labeling for non-sequential label IDs

I am trying to extract DTI metrics from dorsal, intermediate, and ventral white matter columns but I want to extract separate outputs for right versus left. I tried editing the info_label.txt file an added the following lines under the already existing combined labels as is described in issue #1807.
56, left dorsal WM, 0,2
57, right dorsal WM, 1,3
58, left intermediate WM, 4,6,8,10,12
59, right intermediate WM, 5,7,9,11,13
60, left ventral WM, 14,16,18,20,22,24,26,28
61, right ventral WM, 15,17,19,21,23,25,27,29

None of these new labels seem to work for sct_extract_metric. Did I do this correctly or is there another way?

Hi @Aaronearlerichardson which info_label.txt file did you edit? you need to edit the one under your metrics working directory (the one generated by sct_warp_template). If you do so, it should work, e.g.:
sct_extract_metric -l 56 -i mtr.nii.gz

--
Spinal Cord Toolbox (jca/1920-process_seg-csa-perslice/36a2d957e6de90735c11478e3d6ed12ba1e4e76f)
Running /Users/julien/code/sct/scripts/sct_extract_metric.py -l 56 -i mtr.nii.gz

Checked parameters:
  data ...................... mtr.nii.gz
  path to label ............. label/atlas
  label ..................... 56
  method .................... wath
  slices of interest ........
  vertebral levels ..........
  vertebral labeling file.... ./label/template/PAM50_levels.nii.gz
  advanced parameters ....... ['10', '10']

Load metric image...
WARNING: labels #10 contains only null voxels. Mean and std are set to 0.
WARNING: labels #11 contains only null voxels. Mean and std are set to 0.
WARNING: labels #26 contains only null voxels. Mean and std are set to 0.
WARNING: labels #27 contains only null voxels. Mean and std are set to 0.
WARNING: labels #28 contains only null voxels. Mean and std are set to 0.
WARNING: labels #29 contains only null voxels. Mean and std are set to 0.

Saving results in: metric_label.txt ...
	Done.

Results:
ID, label name [total fractional volume of the label in number of voxels]:    metric value +/- metric STDEV within label
56, left dorsal WM [183.74]:    54.6115741778 +/- 4.73656276218
Total processing time: 0 min 1 s

According to set_warp_template I am using the default file path which is: /Users/spineimaging/sct_3.2.2/data/PAM50
and the info file I edited is
/Users/spineimaging/sct_3.2.2/data/PAM50/atlas/info_label.txt
This file is attached at the bottom.
This is what I got when I tried to run set_extract_metric
Spinal Cord Toolbox (version: 3.2.2)
Running /Users/spineimaging/sct_3.2.2/scripts/sct_extract_metric.py -i mtr.nii.gz -perslice 1 -method map -l 56 -o /Users/spineimaging/Desktop/Pipeline_results_temp/001C_2/MTR_in_RDWM.xls

Checked parameters:
  data ...................... mtr.nii.gz
  path to label ............. label/atlas
  label ..................... 56
  method .................... map
  slices of interest ........
  vertebral levels ..........
  vertebral labeling file.... ./label/template/PAM50_levels.nii.gz
  advanced parameters ....... ['10', '10']

ERROR: At least one of the selected labels ([56]) is not available according to the label list from the text file in the atlas folder. Exit program.

Traceback (most recent call last):
  File "/Users/spineimaging/sct_3.2.2/scripts/sct_extract_metric.py", line 1195, in <module>
    fname_mask_weight, fname_vertebral_labeling=fname_vertebral_labeling, perslice=perslice, perlevel=perlevel)
  File "/Users/spineimaging/sct_3.2.2/scripts/sct_extract_metric.py", line 325, in main
    labels_id_user = check_labels(indiv_labels_ids + combined_labels_ids, parse_num_list(labels_user))
  File "/Users/spineimaging/sct_3.2.2/scripts/sct_extract_metric.py", line 853, in check_labels
    sct.printv('\nERROR: At least one of the selected labels (' + str(list_ids_of_labels_of_interest) + ') is not available according to the label list from the text file in the atlas folder. Exit program.\n\n', type='error')
  File "/Users/spineimaging/sct_3.2.2/scripts/sct_utils.py", line 1089, in printv
    raise RuntimeError("printv(..., type=\"error\")")
RuntimeError: printv(..., type="error")
Sentry is attempting to send 1 pending error messages
Waiting up to 10 seconds

info_label.txt

@Aaronearlerichardson, sct_warp_template default is indeed /Users/spineimaging/sct_3.2.2/data/PAM50/atlas/info_label.txt, while sct_extract_metric default is ./label/atlas/info_label.txt. Therefore, the file that you edited was not considered by sct_extract_metric. You have two options:
Option 1:

Edit /Users/spineimaging/sct_3.2.2/data/PAM50/atlas/info_label.txt
Run sct_warp_template
Run sct_extract_metric -l 56 -i METRIC

Option 2:

Run sct_warp_template
Edit ./atlas/info_label.txt
Run sct_extract_metric -l 56 -i METRIC

For more information, please see p91 of the SCT Course Paris'18.

It worked, thank you!


Help needed?

I noticed that there has not been much activity here recently. Ok, carto is a stable project, but I'm not sure how high on Mapbox' agenda carto still is (mapbox-gl has a lot more activity). Note that this is not intended to accuse somebody of not doing enough work. But I'd like to ask if the community could/should help in maintaining carto as there are still some things carto could benefit of, especially since Mapnik 3 has some new features.
cc @pnorman @yohanboniface @gravitystorm @math1985 @matkoniecz

cc @olt - as we're looking at both carto the program and CartoCSS the language

CartoCSS is fundamentally a language: there are many, many stylesheets written in CartoCSS in the wild, some in TileMill, some in Mapbox Studio, in CartoDB, or saved as files on people's computers, or on a server.

Changes to CartoCSS must be backwards-compatible
Additions to CartoCSS need to either pass through older versions, or be accompanied with a 'everyone upgrade' campaign, or have a migration tool, or be clearly part of a "CartoCSS 2.0" that's obviously incompatible

As with any other tool, community help is always appreciated; I still review & merge PRs, often, as long as they pass tests, maintain backwards-compatibility, and are roughly within the spirit of the project.
From my perspective, this being an open source project with an open process implies that community help is appreciated and accepted. Apparently this being under a 'company' organization implies that it's entirely the product of a regime or entirely paid work; in this case neither of those are true.
From the perspective of the Mapbox system, we continue to use CartoCSS to render server-served maps, and we still distribute TileMill & Mapbox Studio Classic. Future new-feature development on my paid time is entirely geared toward Mapbox Studio & Mapbox GL JS, because I strongly believe that these are the way forward and strongly believe that CartoCSS has some deep flaws that are corrected in those technologies. I say that being both one of the principal authors of CartoCSS as well as those systems.

That's just a longwinded "yes, please contribute!". A more clear "here's what would be useful" would be:
These are the hard tasks with lots of discussion that would have the biggest impact for CartoCSS itself. They are hard, but important, and not features, so they aren't what people have been PR'ing for, but it's what I wish they would.

z-index fix https://github.com/mapbox/carto/issues/247
nested conditions breakage https://github.com/mapbox/carto/issues/232
field comparison https://github.com/mapbox/carto/issues/121
nested regex performance https://github.com/mapbox/carto/issues/213

Modernization tasks that are unticketed (things we would have done if we had written carto in 2016 not 2009)

mapnik 3 support
use browserify to bundle carto, correctly
add eslint to the testing process
swap underscore for lodash
modernize dependencies

I'm a bit concerned that there's a bottle neck in reviewing and/or merging PRs - for example, in openstreetmap-carto we were hit by #370 and the PR to fix it #393 has been waiting for over 7 months.
@nebulon42 originally asked here "But I'd like to ask if the community could/should help in maintaining carto" (emphasis added) - any thoughts for or against adding some non-mapbox contributors as committers / maintainers?

@gravitystorm you're right: let's start opening things up. Just gave you push access to CartoCSS!

@tmcw In that case I would like to request push access too. I was thinking/working on improvements for some time now. In the meanwhile I created https://github.com/alacarto/alacarto, but I would deprecate that again in favour of this repository.

@nebulon42 done!

Thanks!

Great development, thanks!

@tmcw What type of review model do you employ on this repo? I will add PRs for new things I'd like to patch in. How much time before merge? Should it be mandatory that PRs will be reviewed by others? Maybe you could list some short guidelines, you can also mail them to me privately (see adr in my profile). Thanks!

Anyone here that needs feedback from me please /cc @springmeyer. I'm swimming in notifications and at the moment am only able to track mentions. But I'm happy to see more contributors and keens to see Carto.js/CartoCSS stay stable and maintainable as bug fixes are needed.

I'd like to follow up on this after I have been trying to take care for Carto for some while. Thanks for trusting me with push access BTW. We got a 0.16 release out, which is good. There have been some regressions, for which I assume full responsibility even if my code was lacking reviews.
However, progress is difficult because I don't have the rights to release new versions on npm. I can understand that Carto has implications on other projects so some sort of control over the npm package is necessary. On the other hand, there is a fix available for some time for the (hopefully) last colour regression, which has never made it into a new bugfix release. For me the motivation of improving the code with no ability to release is quite low. Another problem is the dependency of mapnik-reference. Here e.g. changes for Browserify compatibility are waiting to be merged. Without them Browserify compatibility for Carto is impossible.
The question remains open if functionality like that is still necessary. Since "The end of CartoCSS" appeared on the Mapbox blog I consider CartoCSS officially abandoned by Mapbox. I will soon be moving towards vector tiles and in-browser rendering too. Is there still the need to invest time into CartoCSS? Does the OSM rendering ecosystem need improvements to CartoCSS?

Since CartoCSS is used by a much wider group than just Mapbox, it's only dead when the last person leaves and switches off the lights. I would encourage you to ignore the blog post and keep working on it, it's still used by thousands of people.
As for the releases, is it possible to give @nebulon42 npm release permissions for carto please?

@nebulon42 I've given you permission to publish Carto. mapnik-reference is within the Mapnik organization, and it's not something that I'd feel comfortable granting myself; that's up to the Mapnik team.
Re: releasing, feel free to do so but if there's a high chance of regression, please bump to 1.0.0 so that code using a semver ^ or ~ doesn't auto-upgrade anyone.

Agree with @gravitystorm - as I'm happy to grant npm release access to mapnik-reference as well: @nebulon42 I've done that now and the repo shows up at https://www.npmjs.com/~nebulon42. I've also added @tmcw's comment to the contributing to make sure we have a working doc on how to release that we can improve on in the future: https://github.com/mapbox/carto/blob/master/CONTRIBUTING.md#releasing

Thanks @tmcw and @springmeyer. I promise I'll be careful.


Out of the box support for Wasm plugin with static rule config

See https://github.com/intel/modsecurity-wasm-filter/issues/35
This PR updates the methods in envoy-wasm-modsecurity.cc based on the code in envoy-wasm-modsecurity-dynamic.cc
Also provides a prep-Dockerfile-static.sh that generates a Dockerfile-static file based on Dockerfile.

@dspeg Can you please fix the DCO?

@dspeg Can you please fix the DCO?

@bryanwux Fixed the DCO. Sorry for the delay.

Thanks!


Add an action checkconfig?

Would be great to be able to run an action like:
. ./collector-http.sh -a checkcfg -c test.xml
of -a test or -a dry-run or -a check or -a configtest
That would just (syntactically) validate the configuration file before running an erroneous crawler. In the same manner as you run:
apache2ctl configtest
before crashing your production website :)

What do you mean by "crashing your production website"?   Your configuration may be "valid" but still cause issues to your site if, for instance, you configure it to crawl too aggressively.
Often you will see errors generated upon startup if you have major configuration issues.  Do you have a problematic example that a separate validation step would have prevented?  Maybe such issue should be something to be fixed in the code.
Because people can create their own class implementations and have their own configs for it, it is not possible to validate if all options are what they should be.  Also, sometimes there might be items that are lazy loaded and/or not always possible to test until it runs for real.  Still, we can at a minimum ensure it is parsed properly and does not throw any obvious configuration exceptions as a separate command-line action.  This can be done and I am making it a feature request.

Of course syntactical verification does not guarantee perfect working of the code (I've just verified that with setting a wrong CSS selector), but it helps spotting problems. That's indeed the case with apache2ctl's tool also.
I know that we can get the syntax errors when launching with "-a start", but in case the file is valid, the crawler is actually started, which in the case I was evoking (just checking a minimal config file validity) would not be wanted.
I do not have a specific case, but as a beginner, I like to work incrementally: e.g. write a bit of code, verify it is (almost-)OK, then iterate with another piece of code (e.g. write the Crawling config part, "verify", then the write the Importing part, "verify", then write the "Committing" part), instead of writing a full & complex config file as a one-shot process, then test.
Yes, just ensuring a minimum proper parsing would be OK for me, as a beginner. Even running exactly like it is today with "-a start" (trying to load classes, etc), but stopping right before the first step (Crawling), would be OK for me. It would even help debug/detect CLASSPATH problems.

The latest snapshot version now has this feature.
You can use -a checkcfg and it will load your config without doing anything with it (throwing errors when something is obviously wrong).
Please confirm.

Hi,
Just a quick note to tell you that this new option just saved my [...] today!
Indeed, I did some refactoring/cleaning in my code and moved my committer class into a new directory/package. Therefore, the FQN of the class I had put in the <committer> tag was now wrong. The new option /checkcfg/ gave me a hint about the problem, telling me it cannot find the class, whereas the /starting/ option launched a code that was doing nothing...
Saved me a lot of time (and hairs!) :)
Unfortunately, I also left an unclosed XML comment that was not detected. If you leave an opening comment "<!--" only, there are chances that even a /start/ will raise an error, as soon as it will encounter a "--" (eg. opening another comment). However, leaving only a closing "-->" might not raise anything, neither at /checkcfg/ nor at /start/. Fortunately, my editor's coloring saved my [...] here...

Glad to hear that new feature worked for you.
I'll close, and we can rely on https://github.com/Norconex/importer/issues/27 for possible updates to config validation.

Hi Pascal!
Is there a way to check the configuration (HttpCollectorConfig) by code ?

Yes, have a look at the code for testValidation method in HttpCollectorConfigTest.
You'll see you can get a count of the errors.  If you want to do something with the errors (e.g. store them somewhere), you can add your own ErrorHandler to the log4j appender.

Ok, saw it, thanks.


crash

https://pastebin.com/E4zrR7Lc

Your launcher is providing an incorrect version of Guava. Incorrect means "different from what vanilla MC specifies as a dependency" (which every other launcher obeys) here. Since FerriteCore depends on some internals of Guava this will not work. I consider this a bug in your launcher rather than one in FerriteCore.


Add a CI pipeline

On any PR, add a CI pipeline to:

docker build
score-compose
compose-up
score-helm
helm install

FYI I'll take it.


[RFC] Whether can we reduce bundle size by extract common code?

How do you envision this feature/change to look/work like?
Hi，I have a question about plasmo build. I want to know if there is any way to solve it now.
When there are two .tsx in content and a .tsx in tabs，plasmo will generate three js files after build like content.xxx.js、sidecard.xxx.js、full-chat.xxx.js.
Because the three tsx will be built separately，so if they import many common components、utils、mobx store，it will cause every js bundle contain repeated content and the package zip will be large.
So whether can we extract the common code to a content.js which can be include by other js or html?
For Example, we can combine all js code in one file named content.js，the full-chat.html in tabs or option.html in options can include content.js directly，so there will be only content.js in package zip.
What is the purpose of this change/feature? Why?
avoid repeated code in different js bundle to reduce package zip size
(OPTIONAL) Example implementations
For Example, we can combine all code in one file named content.js，the full-chat.html in tabs can include content.js directly，so there will be only content.js in package zip.
(OPTIONAL) Contribution

[ ] I would like to contribute to this RFC via a PR

Verify canary release

[X] I verified that the issue exists in plasmo canary release

Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I checked the current issues for duplicate problems.

I've got the same problem.

metoo.


Errors for Yeelight component after upgrade HA to 0.45.1

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version):
0.45.1
Python release (python3 --version):
Python 3.4.2
Component/platform:
light/yeelight
Description of problem:
I have two Xiaomi yeelight bulbs. After upgrade HA to 0.45.1 (from 0.44.1) when the power is cut off from the bulbs in the log errors appear
2017-05-28 15:37:41 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last): File "/home/homeassistant/.homeassistant/deps/yeelight/main.py", line 271, in send_command self._socket.send((json.dumps(command) + "\r\n").encode("utf8"))
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/asyncio/tasks.py", line 233, in _step result = coro.throw(exc) File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/core.py", line 1015, in _event_to_service_call yield from service_handler.func(service_call)
File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/components/light/__init__.py", line 265, in async_handle_light_service yield from light.async_turn_on(**params)
File "/usr/lib/python3.4/asyncio/futures.py", line 388, in __iter__ yield self # This tells Task to wait for completion.
File "/usr/lib/python3.4/asyncio/tasks.py", line 286, in _wakeup value = future.result() File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result raise self._exception
File "/usr/lib/python3.4/concurrent/futures/thread.py", line 54, in run result = self.fn(*self.args, **self.kwargs)
File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/components/light/yeelight.py", line 302, in turn_on self._bulb.turn_on(duration=duration)
File "<decorator-gen-7>", line 2, in turn_on File "/home/homeassistant/.homeassistant/deps/yeelight/main.py", line 50, in _command result = self.send_command(method, params).get("result", [])
File "/home/homeassistant/.homeassistant/deps/yeelight/main.py", line 275, in send_command self.__socket.close() AttributeError: 'NoneType' object has no attribute 'close'

or
2017-05-28 21:55:03 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/home/homeassistant/.homeassistant/deps/yeelight/main.py", line 288, in send_command data = self._socket.recv(16 * 1024)
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/asyncio/tasks.py", line 233, in _step result = coro.throw(exc)
File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/core.py", line 1015, in _event_to_service_call yield from service_handler.func(service_call)
File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/components/light/__init__.py", line 265, in async_handle_light_service yield from light.async_turn_on(**params)
File "/usr/lib/python3.4/asyncio/futures.py", line 388, in __iter__ yield self # This tells Task to wait for completion.
File "/usr/lib/python3.4/asyncio/tasks.py", line 286, in _wakeup value = future.result()
File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result raise self._exception
File "/usr/lib/python3.4/concurrent/futures/thread.py", line 54, in run result = self.fn(*self.args, **self.kwargs)
File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/components/light/yeelight.py", line 302, in turn_on self._bulb.turn_on(duration=duration)
File "<decorator-gen-7>", line 2, in turn_on
File "/home/homeassistant/.homeassistant/deps/yeelight/main.py", line 50, in _command result = self.send_command(method, params).get("result", [])
File "/home/homeassistant/.homeassistant/deps/yeelight/main.py", line 291, in send_command self.__socket.close() AttributeError: 'NoneType' object has no attribute 'close'

Expected:
No errors when bulbs have no power.
Problem-relevant configuration.yaml entries and steps to reproduce:
light:
  platform: yeelight
  devices:
    <IP_ADDRESS>:
      name: yeelight color bulb salon 1
      transition: 500
      save_on_change: false
    <IP_ADDRESS>:
      name: yeelight color bulb salon 2
      transition: 500
      save_on_change: false

Is the problem still here with the 0.46 update ?

With HA 0.46 and 0.47 I don't have this issue.


ci: CHANGE_TARGET is not present release branches

Testing

🤷♂
Trade-offs

Dependencies

Screenshots

:tada: This PR is included in version 2.40.11 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket: