pyspark_cassandra.types.Row doesn't play nicely with dataframes

It may have made more sense to subclass the pyspark.sql.Row rather than try to copy it's functionality, as new features that are in spark 1.3, specifically dataframes, don't work out of the box.

I'm not a big fan of the way this spark code works - we'll need _infer_schema to work properly for data frames support.
https://github.com/apache/spark/blob/ab9128fb7ec7ca479dc91e7cc7c775e8d071eafa/python/pyspark/sql/_types.py#L573

Wasn't that happy myself either. I'll look into it.

I went for the easy way out for now: the issue was that the pyspark_cassandra Row wasn't iterable, now it is. If time permits I'll change to the pyspark.sql.types.Row implementation.

I don't think this is working correctly.  Rows that are being turned into data frames look like this:
Row(_1=u'bacon', _2=u'Jon'), missing the correct field names.

Compare, for instance, when I manually fix the schema:
users2 = user.map(lambda x: Row(favorite_food=x.favorite_food, name=x.name))
df2 = users2.toDF()

In [31]: df2
Out[31]: DataFrame[favorite_food: string, name: string]

In [32]: df2.printSchema()
root
 |-- favorite_food: string (nullable = true)
 |-- name: string (nullable = true)

A temporary workaround for existing code:
In [54]: result = user.map(lambda x: Row(**x.asDict()))
In [55]: result.toDF()
Out[55]: DataFrame[favorite_food: string, name: string]

Looking at the spark code at https://github.com/apache/spark/blob/ab9128fb7ec7ca479dc91e7cc7c775e8d071eafa/python/pyspark/sql/_types.py#L573, we see this:
    elif isinstance(row, (tuple, list)):
        if hasattr(row, "__fields__"):  # Row
            items = zip(row.__fields__, tuple(row))
        elif hasattr(row, "_fields"):  # namedtuple
            items = zip(row._fields, tuple(row))
        else:
            names = ['_%d' % i for i in range(1, len(row) + 1)]
            items = zip(names, row)

Adding this to the Struct class seems to work:
	@property
	def _fields(self):
		return self.keys()

@rustyrazorblade and @mjhb I take it you guys are building Spark from the master branch? There is a spark/python/pyspark/sql/types.py file in the 1.3 branch. I tested against the 1.3.0 and 1.3.1 pre-built releases taken from [https://spark.apache.org/downloads.html].
Apparently the guys at Spark don't care to much about stability of the internal 'API'. The 1.3.1 release performs the following test:
elif hasattr(row, "__FIELDS__"):  # Row
    items = zip(row.__FIELDS__, tuple(row))

But when I looking into the 1.3 branch the __FIELDS__ was lower cased. I've added the suggestion from @rustyrazorblade to master.
I could switch to using the 'normal' pyspark.sql.types.Row but I really like the pyspark_cassandra version because it accepts adding a field. If things don't get worse ... I'll keep it at this.

I'm using spark 1.3.1 release, and building off master for all my work with this lib.
I'm not sure how useful adding a field to an existing Row, as everything by default in spark is immutable.  You'd want to create new Rows, rather than updating an existing one.  However, it's probably not going to be a problem any further in 1.3 so I don't really see any reason to change it now either.

I didn't experience the issue with the 1.3.1 pre-built release.
Well, adding a field is useful in transforming (mapping) an rdd. I'm not getting at making immutable RDD's mutable. E.g. when some derived field is used over and over again, it's convenient to just calculate it once in a map call and then cache the rdd.
Silly example, but say I have a table with circles which have a radius, and I want to do things with the circles based on their area And then operate on this calculated / derived field area.
import pyspark_cassandra
import math

def calc_area(circle):
    circle.area = math.pi * circle.radius**2
    return circle

circles = sc.cassandraTable(...).map(calc_area).cache()
area_stats = circles.map(lambda circle: circle.area).stats()
large_circles = circles.filter(lambda circle: circle.area > 10)
large_area_stats = large_circles.map(lambda circle: circle.area).stats()

There are other ways of doing this of course, but I think it's rather concise. Of course this is trading space for CPU cycles. But say you'd have an iterative algorithm which operates on such a derived field, calculating this field once - if space permits - probably is booster.
I'll see if this is worth the hassle of keeping up with Spark internals :)

FYI - I am building Spark from branch 'branch-1.3'.
DataFrames is working much better this morning - I was able to complete the relevant exercises from the SparkSQL and DataFrame Guide (https://spark.apache.org/docs/latest/sql-programming-guide.html) using a small table from Cassandra.
Thanks!

Thanks, yes I noticed that also in branch 1.3 the name of FIELDS was changed. Good to hear that the DF functionality is working!


[JS] Remove unnecessary production dependencies

Describe the enhancement requested
The Apache Arrow JS bundle includes a few unnecessary packages that increase the dependency graph. The culprits are:

command-line-usage: used by integration.ts, json-to-arrow.ts, and arrowtocsv.ts. I think these can move to dev dependencies since it looks like the intent of the scripts is integration testing and benchmarking.
@swc/helpers: Introduced in https://github.com/apache/arrow/pull/38500. @swc/helpers is never imported. I think it's used for https://github.com/apache/arrow/blob/9b931af14e5a710cba0aaa6b899e2ca696bfd785/js/tsconfig.json#L22. If ts-node is only for development, this can move to devDependencies.
@types/node: Should move to devDependencies. I don't think there's any benefit to having it listed as a production dependency.
@types/command-line-args: move to devDependencies
@types/command-line-usage: move to devDepdencies
json-bignum: used by integration.ts, json-to-arrow.ts, and arrowtocsv.ts. Can move to dev dependencies.

Here's the graph of the current dependencies.

Component(s)
JavaScript

arrow2csv is a bin provided by the library so we need to keep it as a dependency. We could probably remove that script, though but that's for another day. I removed swc in https://github.com/apache/arrow/pull/41274. The types should be in the dependencies so people get the right versions.

The types should be in the dependencies so people get the right versions.

I think we only need to export types transitively used by exported libraries, not exported binaries. Since @types/command-line-args, and @types/command-line-usage are only used for the arrow2csv command and integration tests, they can probably live in dev dependencies.
Similarly, I suspect @node/types is only used for binaries and integration tests, but that's harder to verify.

We could probably remove that script, though but that's for another day

Alternately, publish as a separate package.

Good point. I'll update the pull request.

arrow2csv is a bin script for library consumers to use, e.g. curl /my/cool/api | npx arrow2csv | less

Yeah, let's not remove it for now, that's a separate issue.
One could argue that cli tools should be separate packages and that js isn't the ideal language for cli tools (which is why I made https://github.com/domoritz/arrow-tools/tree/main/crates/csv2arrow ;-)).


jp .pot file kickoff

This PR may be a bit early, but it may show other languages how easy it is to get going!
Japanese translation is now officially going, with some way to go...

I'll take it, thanks!!

ありがとぢます！

Renamed this to .po ;)


kprom v2.0

I thought that there was room for improvement in the kprom plugin. So I created this draft PR for further discussion.
I tried to consolidate the best parts from examples/hooks_and_logging/expansive_prometheus and plugins/kprom.
@twmb will you please take a look when you have time.

@weeco Thanks for taking a look, I incorporated you feedback in 70b479e8b58416cc035e06665b8df4070d9cfc8a and rebased on upstream, Are there anything thing else, or should I promote the daft as ready for PR?

I think the main breaking change is removing namespace as a required argument to NewMetrics. We can keep this in the v1.x series if you add back in a required namespace. A few options in this have With prefixed, but exist in the old version without With. I personally go back and forth on whether options should have With, because once you have a bunch of options, the With becomes a lot of noise (kgo actually started out with With but I dropped it quickly).
I'd only add each set of timings if requested, and not opt into any set of timings by default. They quickly become a huge set of metrics / cardinality that some orgs don't want to deal with.

I dont think we should keep it in 1.x, the metrics and labels change name and dimensions. Also I dont think a ns should be required and I would like it to work as kgo.NewClient, where all config follows the opt pattern.

@twmb I made some updates to the plugin, making the Histograms optional, worked with Opts and I added documentation and examples.
I think there are too many breaking changes to metric names, labels, opts, interfaces etc and I dont see how I will be able to keep V1 compatibility and still reach my original goal of reworking the kprom plugin consolidating the best parts from both the kprom V1, the expansive_prometheus example and rethinking the contract.
Let me know where you want to go from here, I am open to take a couple of more iterations on this, but if you already now know that you dont want to introduce a V2 with breaking changes I should properly close the PR here.

This was complicated to analyze so I needed to find some time to sit down and actually diff everything in detail.
Breakage
APIs
ADDED

var DefBuckets
type FetchProduceLabel (Topic | NodeID) // comment: uppercase NodeId -- allows topic and node_id labels to be optional (required in old)
type Histogram (ReadWait | ReadTime | WriteWait | WriteTime | RequestDurationE2E | RequestThrottled
type HistogramOpts
func (*Metrics) OnBrokerE2E
func (*Metrics) OnBrokerThrottle
func Buckets([]float64) Opt
func FetchProduceLabels(...FetchProduceLabel) Opt
func Histograms(...Histogram) Opt
func HistogramsFromOpts(...HistogramOpts) Opt
func Namespace(string) Opt
func Subsystem(string) Opt

CHANGED

func NewMetrics(namespace string, opts ...Opt) => NewMetrics(opts ...Opt)

REMOVED

func (*Metrics) Handler (add doc: providing prometheus metrics for the gatherer)
func (*Metrics) OnBrokerRead
func (*Metrics) OnBrokerWrite
func Registry(RegistererGatherer) Opt
func Gatherer(prometheus.Gatherer) Opt
func GoCollectors() Opt
func HandlerOpts(promhttp.HandlerOpts) Opt

Metrics
ADDED

(label) client_id ConstLabel on everything
write_wait_seconds, histogram
write_time_seconds, histogram
read_wait_seconds, histogram
read_time_seconds, histogram
request_duration_e2e_seconds, histogram
request_throttled_seconds, histogram
produce_compressed_bytes
produce_batches_total
produce_records_total
fetch_compressed_bytes
fetch_batches_total
fetch_records_total

RENAMED

connects_total => conn_connects_total
connect_errors_total => conn_connect_errors_total
disconnects_total => conn_disconnects_total
produce_bytes_total => produce_uncompressed_bytes
fetch_bytes_total => fetch_uncompressed_bytes

Internal defaults changed

node_id dropped from fetch & produce bytes
kafka subsystem added

Proposals
It's possible to do a lot of this without breaking APIs.
Keep all old APIs

OnBrokerRead, OnBrokerWrite change to noop in favor of OnBrokerE2E replacement
No harm to keep Registry, Gatherer, GoCollectors, HandlerOpts, (*Metrics) Handler

Drop Namespace option, drop internal "kafka" subsystem default

purely option driven might be a decent goal, but if we can keep a 1.x release that's preferable, and namespaces are basically required in Prometheus so it made sense to me to make it a required argument
Default subsystem "kafka" makes less sense if you produce to Redpanda / MSK / Event Hubs -- I'd rather force a person to specify the subsystem here ;)

Add WithClientLabel() Opt

Replaces new breaking ConstLabel default

Keep old names for 5 renamed metrics

Presumably the conn_ prefix is to help for grouping but it's not critical
Keeping (produce|fetch)_bytes_total can avoid breakage and can be explained away in docs.

The reason I chose uncompressed bytes only is because per-broker write bytes shows you the compressed sizes (albeit without per-topic)

Change 6 new produce&fetch metrics, as well as FetchProduceLabel and FetchProduceLabels

new option FetchAndProduceDetail(...Detail), with internal default (UncompressedBytes, ByTopic, ByNode)
new enum type, Detail (UncompressedBytes, CompressedBytes, Batches, Records, ByTopic, ByNode)

Can even add ConsistentNaming to force rename (produce|fetch)_bytes_total to (produce|fetch)_uncompressed_bytes

I'm not entirely positive about the histogram APIs yet but they don't break the current APIs at all. What do you think about the above proposals?

Would be nice to be able to unregister all the metrics too: https://github.com/twmb/franz-go/pull/469. That may not be necessary with for us though with a WithClientLabel API addition.

I updated the PR, I have not tested the changes yet, but now the PR should conform to the proposal. If you have time please glance over it.

This is ready for for review.
@freeformz I also implemented Unregister, but I did it as part of the kgo.HookClientClosed.

No idea how I borked your master and closed this PR, I moved a few things around and fixed a few docs, reopened as https://github.com/twmb/franz-go/pull/505, this lgtm. I'm going to test locally -- I'm a bit worried about installing histograms to the factory by default but I think they don't show up because they aren't ever incremented.


Schema definition

Motivation
Fixes #59
Internal ID DEVREL-323
Checklist

[X] Code follows coding conventions held in this repo
[X] Automated tests have been added
[X] Tests are passing
[ ] Docs have been updated (if applicable)
[X] Temporary settings (e.g. variables used during development and testing) have been reverted to defaults

How to test
Try to generate schema with gatsby develop, check generated schema. Schema should be generated for types without item representation as well.

TODO:

image width and height 👍
fix CI (probably downgrade rxjs to 6.4.0 - introduced in https://github.com/Kentico/gatsby-source-kontent/commit/12e922a2d25fdf1272e2feafe962f91bd608a645)
name of elements - use snake_case instead of camelCase

check sub-properties of the elements if the case is fine
cover that by tests

adjust /docs/Upgrade.md according to the new functionality


J2 perturbation

🐞 Problem

🖥 Please paste the output of following commands

conda info -a (only if you have conda)
conda list (only if you have conda)
pip freeze

# Paste your output here:

# Paste your output here:

💡 Possible solutions

📋  Steps to solve the problem

Comment below about what you've started working on.
Add, commit, push your changes
Submit a pull request and add this in comments - Addresses #<put issue number here>
Ask for a review in comments section of pull request
Celebrate your contribution to this project 🎉

Issue is empty, closing.

exercising QUICKSTART programs. Interested in the moon and MARS mission design.


Add init container to ensure Vault token

Towards https://github.com/giantswarm/giantswarm/issues/14611
Checklist

[ ] Update changelog in CHANGELOG.md.

You also need to mount secret in app to get access to it. E.g. https://github.com/giantswarm/cert-operator/blob/master/helm/cert-operator/templates/deployment.yaml#L69-L74
I'm adding this part in #34 already.

And also whole this thing will work only after vault configuration
What needs to be configured, specifically? The auth/kubernetes/role/config-controller  role mentioned here?

Forgot to mention: @corest :point_up:


Run rflint with different rules

Based on a path, in config file, run subset of rules. Subset of rules are also defined in the config file.

Closing because of the rewrite.


Add a Codacy badge to README.markdown

I think its wroth running this tool manually and seeing what we can do with resolving whatever results it turns up before merging this.

Sure, Codacy always runs the PMD ruleset on its server on every commit, this is just for the badge if you want the badge in the readme.   You can run PMD locally with:

Download from https://github.com/pmd/pmd/releases
place in /build/pmd
sh build/pmd/bin/run.sh pmd -d force-app/main -R apex-ruleset.xml -f csv >> -build/pmd-output.csv

The apex-ruleset.xml file is new and in #931 This lets you run the same rules locally as well as part of a pre-commit hook, PR review, etc.


How to get the inference model? Thank you！

How to get the inference model? Thank you！

Thanks for your attention! We will release inference models after this paper is officially accepted. At present, it is still in the revision stage, and I guess it will take about 3~6 weeks for peer review.


feat: unbounded lookahead using adaptive predict

Implements an LL(*) lookahead algorithm and removes the existing LL(k) lookahead. The design is heavily inspired by Antlr's existing LL(*) solution titled "Adaptive LL(*)". In fact, the general structure of the Antlr lookahead code has been adapted to provide this feature.
There is a caveat that differentiates this approach to the LL(k) lookahead algorithm: As we don't simply iterate over a sorted list of alternatives, we can't decide on taking the "first" alternative in the alternation. Instead the LL(*) algorithm will always return the longest matching alternative, just like Antlr does.
While this PR is mostly feature complete, there are still some things that have yet to be addressed. Nevertheless, I wanted to create the PR at this stage to get some feedback on the design of the feature. The documentation for this feature is still missing and validation

Hello @msujew
I did not dive into the algorithm yet. below are my initial impressions.
Feature Questions

What is the the relationship (if any) between adaptivePredict and maxLookahead ?
If adaptivePredict is enabled, does it mean that all lookahead logic is done using the new logic?
If the above is true, does a hybrid approach make sense?
If an hybrid approach does not make sense could maxLookahead = "*" remove the need for the new adaptivePredict config option?
What is the performance impact of enabling the adaptive predict? (I may check this tomorrow).

Code Organization / Beginning Code Review

I think it may be better to separate the new logic and tests to separate new files.
Also some of the functions are getting quite large, e.g the lookahead function builder should probably be split
into multiple helper functions.
the use of isArray(alternatives) is multiple places is not the most readable style, we should more clearly distinguish between the different flows, and maybe consider if it is possible to avoid "overloading" the return function of the lookahead helpers (depends how distinct the LL(K) and LL("*") flows are.

What is the the relationship (if any) between adaptivePredict and maxLookahead? If adaptivePredict is enabled, does it mean that all lookahead logic is done using the new logic?

So, basically the adaptivePredict flag indicates that after k=maxLookahead has been reached when computing the lookahead function and an ambiguous alternative has been found (i.e. LL(k) is not powerful enough to parse the alternative), it instead builds an prediction automaton. See this code. It does this on demand, the usual approach will be used if LL(k) is enough to parse the alternative. The current implementation is a hybrid approach. You can force to use the prediction automaton everywhere if you set maxLookahead=0, as I've done in a few of my test cases.

What is the performance impact of enabling the adaptive predict? (I may check this tomorrow).

Good question. I haven't tested it so far. I plan to evaluate the performance more in-depth (mostly cpu/memory consumption) for my thesis, but that will take some time. I would be grateful if you were to do some basic performance testing for this feature.
In theory the performance of this should be quite good, especially after it completed its "warmup"-phase. Compared to the usual LL(k) approach, the lookahead strategy in adaptive LL(*) is build during parsing and not during the self analysis. When it parses an alternative for the first time, it will run the possiblePathsFrom function with a lookahead of 1 for each token it parses, creating a new state for that token. Afterwards the new state is cached, which will be used on subsequent runs through that alternative. This should be faster than the usual approach of iterating over every alternative for every token encountered (up to k). The Antlr paper states that this approach is O(n) for most grammars (although O(n^4) in the worst case, with n being the number of tokens needed to identify the alternative).
Thanks for the initial review/feedback, I will get to that soon.

Hybrid Approach
Generally this the optimal approach imho.
Some things to consider:

If the performance impact is not large (or even positive) we could consider only using LL(*) lookahead.
Alternatively we could also consider an hybrid approach (hidden from the user) where LL(1) (old) lookahead is used and LL(*) is used for anything else.
If the end-user can choose adaptivePredict on the grammar level, we may also want to enable choosing it on a specific
dsl production level, just like the maxlookahead can be used: https://chevrotain.io/docs/guide/resolving_grammar_errors.html#AMBIGUOUS_ALTERNATIVES

I've tried running some benchmarks but I am getting parsing errors.
For example, for the basic JSON Grammar example:

https://github.com/Chevrotain/chevrotain/blob/master/examples/grammars/json/json.js#L52

If I change it to use "maxLookahead=0" and "adaptivePredict=true" it fails the very simple unit test.

https://github.com/Chevrotain/chevrotain/blob/master/examples/grammars/json/json_spec.js

I've tried running some benchmarks but I am getting parsing errors.

Interesting, it seems like every example grammar does not work with that config. Weird that my tests couldn't catch that. Thanks for the info, I'll investigate it.

@bd82 I fixed most grammars for now. The buildSingleAlternativeLookaheadFunction implementation for my DFAs was incorrect. While most grammars simple run well now, there are still a few issues:

CSV parser: Empty alternatives are currently ignored.
GraphQL parser: I have no idea what's currently wrong there, more investigation needed.
ECMA5 parser: Due to an ambiguity the DFA never reaches an accepting state. Antlr somehow deals with this, but my implementation doesn't.

I was able to test my approach for the JSON and CSS grammars using the benchmark. However, I am not happy with the current results:

I will rethink my approach and take a closer look at what Antlr does.

I will rethink my approach and take a closer look at what Antlr does.

is your benchmark in "parser only" mode?
I will start with correctness first and only then move to performance optimizations.
Some things areeasier to optimize if code generation is involved, I am unsure if a code generation step fits Chevrotain
(probably not). But it may still be interensting to inspect what could be gained by generating the lookahead functions.
a Hybrid solution may still have value if it can make it easier to parse certain languages, particularly if the performance can be improved.

for example: the java-parser package uses quite a-lot of backtracking which significantly hurts performance.
could adapative predict simplify that grammar implementation ? could it imrpove performance?

https://github.com/jhipster/prettier-java/issues/516

is your benchmark in "parser only" mode?

Yes.

I would start with correctness first and only then move to performance optimizations.

Sure, I planned on correcting the issues first. Nevertheless, I think my current approach to the issue is too naive. I looked at Antlr's implementation of LL(*) (which is actually interpreted, there is no code generation done for the lookahead automaton), and I'm missing some crucial elements of the algorithm, which should be computed during the self analysis phase. For example, Antlr actually builds two separate Automaton. The lookahead automaton is build during parsing, while the other is build at startup and is used to compute the actual lookahead automaton.

a Hybrid solution may still have value if it can make it easier to parse certain languages, particularly if the performance can be improved.

That's true, however my main incentive for building this feature - besides writing about it in my masters thesis - is using it in Langium for every lookahead k > 1. That would heavily improve usability of the framework for more complex alternatives. Therefore I aim to keep performance impact at a minimum.

is using it in Langium for every lookahead k > 1

If we can achieve good enough, performance I'd like to evaluate using the adaptive predict like this in Chevrotain
as well. Basically get rid of the old LL(K) lookahead logic, mainly for long term maintainability concerns
(fewer lookahead modes --> simpler --> easier to support).

BTW @msujew WDYT about implementing the new lookahead logic as a separate package?

e.g: like the utils package.

I am trying to slowly transition Chevrotain to a mono-repo structure.
So it might make sense for significant new functionality to be added as a new sub-package.
Naturally this would only be possible if the new Lookahead logic has no depedencies on
concrete implemntations inside the "main" chevrotain package, otherwise we would have cyclic package dependencies...

WDYT about implementing the new lookahead logic as a separate package?

Sounds reasonable.  I've limited myself a bit by reusing existing interfaces as much as possible. Makes testing probably also a bit easier.

@bd82 I have spent the last few weekends on working on a new implementation based on the original ANTLR implementation. The results for JSON and CSS processed purely through LL(*) are quite promising, even though they are LL(1) languages:

I still have find out how to deal with empty alternatives (as that's not really supported in ANTLR afaik) and ambigious alternatives. While ANTLR has a solution to that problem, it's not easily applicable Chevrotain in its current form.
I assume I'll have something finished to show next weekend. :)

I just also added LL(1) optimization, I'm a bit confused by the CSS results, but I assume this is a consequence of not caching token categories correctly in the lookahead DFA. Everything else looks as expected:

I will try to have a look at this on the weekend

@bd82 Alright, pushed everything to the upstream branch connected to this PR. 14 tests are currently failing, but that's because they either rely on old behavior (I completely removed LL(k) lookahead for now) or override the related APIs, which aren't called anymore.

@bd82 Great news, I just added a small optimization, replacing the DFA cache (previously a Map) with an Object, which was used in a really hot loop. This increased performance quite drastically:

It is now way faster than LL(k) for more complex grammars (at least in my benchmarks).

Great news, I just added a small optimization, replacing the DFA cache (previously a Map) with an Object, which was used in a really hot loop. This increased performance quite drastically:

Oh wow very impressive, Gives me more incentive to find time on the weekend to dive into this 😄

Okay I'm finally starting to dive into this topic.
I've read the LL-star document and hopefully understood most of it...
Also ran the benchmark on:

Chrome Canary
"Parser Only" flow
versus release 9.1.0 (should be similar (but not exact) baseline to current master)

It shows slightly different results than what you have posted, but they are still very good (none blocking) and there can be differences between different engines/cpus/... so that does not matter...

Hi @msujew
I am merging the current work into new branch master-LL-star branch on this repo.
and sent you an invite for write permissions on the repo so we can more easily collaborate on this topic.


⚠ Secret Website 3 has degraded performance

In ba51a9f, Secret Website 3 ($SECRET_WEBSITE_3) experienced degraded performance:

HTTP code: 200
Response time: 10716 ms

Resolved: Secret Website 3 performance has improved in e49c712.


chore: add the readme.md file for the generator

Changes Added.

added the readme.md file for the generator.

Related issue(s)
Related to: https://github.com/asyncapi/generator/issues/1044

@asyncapi/bounty_team

/rtm

/rtm

/bounty


Extension name and version change

@MashaMSFT
SQLIaasExtension -> SqlIaasExtension
1.2 -> 2.0

@vasivara-MSFT : Thanks for your contribution! The author, @rothja, has been notified to review your proposed change.

#sign-off

@MashaMSFT: I'm sorry - only the author of this article, @rothja, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.


XMB retroactive & retroactive_marked filename's incorrect

The following files are named incorrectly and because of that the images will display blank.
An update of the two following assets folders with name changes to the files will fix the blank image issue.
RetroArch\assets\xmb\retroactive
RetroArch\assets\xmb\retroactive_marked\
NEC - PC Engine - TurboGrafx 16-content (E).png
NEC - PC Engine - TurboGrafx 16-content (J).png
NEC - PC Engine - TurboGrafx 16-content (U).png
Nintendo - Nintendo Entertainment System-content (E).png
Nintendo - Nintendo Entertainment System-content (J).png
Nintendo - Nintendo Entertainment System-content (U).png
Nintendo - Super Nintendo Entertainment System-content (E).png
Nintendo - Super Nintendo Entertainment System-content (J).png
Nintendo - Super Nintendo Entertainment System-content (U).png
Sega - Dreamcast-content (E).png
Sega - Dreamcast-content (J).png
Sega - Dreamcast-content (U).png
Sega - Mega Drive - Genesis-content (E).png
Sega - Mega Drive - Genesis-content (J).png
Sega - Mega Drive - Genesis-content (U).png
Should be:
NEC - PC Engine - TurboGrafx 16 (E)-content.png
NEC - PC Engine - TurboGrafx 16 (J)-content.png
NEC - PC Engine - TurboGrafx 16 (U)-content.png
Nintendo - Nintendo Entertainment System (E)-content.png
Nintendo - Nintendo Entertainment System (J)-content.png
Nintendo - Nintendo Entertainment System (U)-content.png
Nintendo - Super Nintendo Entertainment System (E)-content.png
Nintendo - Super Nintendo Entertainment System (J)-content.png
Nintendo - Super Nintendo Entertainment System (U)-content.png
Sega - Dreamcast (E)-content.png
Sega - Dreamcast (J)-content.png
Sega - Dreamcast (U)-content.png
Sega - Mega Drive - Genesis (E)-content.png
Sega - Mega Drive - Genesis (J)-content.png
Sega - Mega Drive - Genesis (U)-content.png

@Kivutar Outdated issue?

I'm not sure really. We don't have any playlist that is named with those (E) (J) and (U) anyway. So those images will never be displayed.

I think the maker of retroactive and retroarchive_marked made everything to let users changing icons if they wanted to (according to their regions or such). You only see (U), (E), and (J) files in ./xmb/retroactive/ and ./xmb/retroactive_marked/ anyway.
If changing icons, one would omit/rename region names anyway.
Sega - Dreamcast (E).png
Sega - Dreamcast (J).png
Sega - Dreamcast (U).png
...to...
Sega - Dreamcast.png

Sega - Dreamcast-content (E).png
Sega - Dreamcast-content (J).png
Sega - Dreamcast-content (U).png
...to...
Sega - Dreamcast-content.png

I think this issue should be closed.

I agree.

I know this issue is closed but I missed seeing it.  I apologize.  As @Kivutar indicated, I have made extra icons for the theme that indicate region or color variations and they can be switched out manually as per @lasers instructions.  I tacked on the region code before the file extension but they can be renamed down the line if there ever comes a time when a playlist could distinguish via region code.  I don't think that feature is currently on the to-do list.


How to make the character move around origin point?

currently seems the hips postion always to zero, how to make it move around origin point then it can looks more natural?

You can move Pelvis's postion, it's the root joint.

Is the character's position right?

@yanch2116 Hi, I am using the cam_trans from ROMP output as transition. I applied 100*cam_trans convert it to cm unit.
But the character can not really saw move.  Do u have a video what's your motion looks like?

I just use bpy.data.objects['Armature'].pose.bones['Pelvis'].location to change the hip's translation.
It's here.


Stream live video

Is it possible currently ? If yes - how to do it ? :)

http://acestream.org/ru/ live stream torrent
Old version and old protocol
https://github.com/tarasian666/acestream


add for...of loops

Can't see a page for for...of loops.
These are new in ES6. Firefox supported them early. Would be useful to know which browsers have this feature now.

Duplicate of #4097

why not close the duplicate?

This is implemented as far as I can tell: https://caniuse.com/?search=for of


Test: Remaining tests for LitNodeClient

Some tests are throwing errors which have been commented out.

Thank you @Adarsh-Kumar28 for writing these and @Ansonhkg for reviewing and merging!


Improve longhand into multiple shorthand merging

E.g (shorter value).
p{background:-webkit-linear-gradient();background:linear-gradient();background-repeat:repeat-x} ->
p{background:-webkit-linear-gradient()repeat-x;background:linear-gradient()repeat-x}
or (default value)
p{background:-webkit-linear-gradient();background:linear-gradient();background-repeat:repeat} ->
p{background:-webkit-linear-gradient();background:linear-gradient()}

these proposed transformations may be invalid due to style cascading, e.g.
.block-1 {
  background:url(image.png) repeat-y;
}
/* ... somewhere further down */
.block-2 {
  background:-webkit-linear-gradient();
  background:linear-gradient();
  background-repeat:repeat-x; /* <- trying to merge this into two properties above */
}

where merging in repeat-x would not override repeat-y from .block-1 declaration anymore - I recommend we leave it as is to avoid potential issues.


num_boost_round  and early_stopping_rounds in xgboost.train() API

Hi,
I am confused if we use early stopping with number of boost round larger than 1, are we still getting multiple boosted random forest and each rf has n_estimators trees?
Thanks

For xgboost.train,  I don't think there's an n_estimators parameter.  Use num_parallel_tree for boosting forest.

Each boosting rounds adds a random forest with num_parallel_tree trees.  Hope that helps.

cool, however if i use sklearn xgboost API, when i call XGBClassifier.fit(), i dont get to pass 'num_boost_round' anyways which meaning i am only trianing boosting trees within 'n_estimators'? depend on my 'early_stopping_round' set up? thanks in advance

n_estimators is an scikit-learn version of num_boost_round I think.

oh , so assume i pass n_estimators 100, i train on 50 iterations with early_stopping_round then how many random forests and how many trees i end up get ? thanks

100 n_estimators means 100 iterations, resulting in 100 stacked trees.  If you want to do random forest, with native API that would be:
xgboost.train({'num_parallel_tree': 50, 'tree_method`: ... }, X, num_boost_round=100)

The above line will result in 100 stacked random forests with 50 trees each.  The sklearn API is mostly a wrapper around above native API.

i see so with sklearn, you can only do boost trees instead of RFs. Thanks for confirm,rly helpful thanks

For non-boosted random forest (only 1 random forest), you can use XGBRFRegressor and XGBRFClassifier.

You can do boosting RF in sklearn.  Just provide num_parallel_tree as part of the kwargs.  Or use set_param class method.

sorry could you be more specify how do i train rf booster with XGBClassifier  in sklearn xgboost API? n_estimators and num_parallel_tree both larger than `?

import numpy as np
import xgboost

kRows = 1000
kCols = 100

X = np.random.randn(kRows, kCols)
y = np.random.randint(low=0, high=4, size=kRows)

dtrain = xgboost.DMatrix(X, y)

cls = xgboost.XGBClassifier(
    n_estimators=100,
    n_jobs=8,
    # following belongs to kwargs so may not work with some
    # scikit-learn methods
    num_parallel_tree=8,
    tree_method='hist'
)

cls.fit(X, y)

Be careful that due to multi-class training uses one tree for each class.  So when you set num_parallel_tree to 8 and with 4 classes, you get 32 new trees for each iteration, with 100 iterations you will have total 3200 trees in final booster ...
@hcho3 Correct me if I'm wrong.  Also we need to revisit the sklearn wrapper for updating parameters when time allows, so I'm keeping this issue open. ;-)

good, will follow this practice and interested to see the result then, rly helpful!

@trivialfis I think your explanation is reasonable.

so if i pass n_estimator (num_parallel_tree) to larger than 1 and also num_boost_round larger to one, what will happen, what will n_estimator convert to and am i also training boosted rfs?

@wyd1582 You should not pass num_boost_round since you are using the scikit-learn interface.

i mean when i was using xgboost.train() the learning API not the XGBClassifier- sorry to ask back and force cause these parameters are pretty confusing for me now

@wyd1582 You should pick which interface to use.

If using xgboost.train() : use num_boost_round
If using XGBClassifier: use n_estimator

sorry but i actually trained an object with this set up : now i am confused what i trained:

XGB_KWARGS= dict(num_boost_round = 50)
PARAM = dict(silent=1, n_estimators = 100, max_depth= 7, gamma =0.01,
min_child_weight=1, max_delta_step=1, subsample=1, colsample_bytree=1,
colsample_bylevel=1,  scale_pos_weight=1,
base_score=0.05, seed=0, missing=None, random_state=0,
obj= 'binary:logistic')

and do

xgb.train(** PARAM, **XGB_KWARGS)

am i training 50 boosted RFs each has 100 trees??

n_estimators should not be there, because you are using xgb.train(). So n_estimators has no effect.

cool if it is confirmed then i do understand the relation with those parameters now, so with this set up i am training boosted trees no Random Forests
thanks 👍

and to confirm, my num_parallel_tree by defualt is 1 , and if i don't pass that set up , i will also only boost one tree each iteration

Yes

this really solved a lot confusions thanks


Title: Enhancing Footer and Icons

Initiative (Required)
GSSoC 2024 Extd 🚀
Is your feature request related to a problem? Please describe.
Feature
Describe the solution you'd like.
Updating footer and add designs and important links
Add any other context or screenshots about the feature request here.
No response

I am the mentor of this project.
Hello @devxMani
I appreciate your pull issue , can you please describe little bit more and give screenshots!!!

Hello @devxMani ,
Mentor @SakiraAli1115 is inquiring about something. If you’re not serious about addressing it, this issue will be closed by the end of the day today. @SakiraAli1115, please go ahead and close it if he does not respond by then.

Hello @devxMani ,
Mentor @SakiraAli1115 is inquiring about something. If you’re not serious about addressing it, this issue will be closed by the end of the day today. @SakiraAli1115, please go ahead and close it if he does not respond by then.

okay!

@SakiraAli1115 Yes firstly I'll enhance the footer by adding navigation links and icons of different navigations to create a separation between them

@SakiraAli1115

like this

@devxMani The Footer we have it's perfectly fine. Thank YOU


Add support for gorilla/mux

Is your feature request related to a problem? Please describe.
go-sdk is not configurable with gorilla/mux.
Describe the solution you'd like
add a function which adds a middleware for different KEPLOY_MODE's.
Describe alternatives you've considered
go-sdk provides support for chi, gin, echo, webgo routers.

Closing this because Mux support has been added in #67 PR


Remove unnecessary TODO

Description
https://github.com/rubyforgood/diaper/issues/310 is to go through TODO items in the codebase. This PR addresses one of those TODO items by cleaning up a spec.
Checklist:

[x] I have performed a self-review of my own code,
[x] ~I have commented my code, particularly in hard-to-understand areas,~
[x] I have made corresponding changes to the documentation,
[x] ~I have added tests that prove my fix is effective or that my feature works,~
[x] New and existing unit tests pass locally with my changes ("bundle exec rake"),
[x] ~Title include "WIP" if work is in progress.~

Yessssssss!!
I didn't want to seagull into the codebase with a bunch of personal
preferences, but if you also like betterspecs, expect my future spec
changes to quietly update to follow betterspecs convention.
On Tue, Jun 12, 2018, 18:21 Aaron H<EMAIL_ADDRESS>wrote:

@armahillo commented on this pull request.
In spec/models/distribution_spec.rb
https://github.com/rubyforgood/diaper/pull/427#discussion_r194933167:

   two_days_ago = 2.day.ago

   expect(create(:distribution, issued_at: two_days_ago).distributed_at).to eq(two_days_ago.strftime('%B %-d %Y'))
   expect(create(:distribution).distributed_at).to eq(Time.zone.now.strftime('%B %-d %Y'))
 end

it "copy_line_items" do

 @donation = create(:donation)

 @donation.line_items << create(:line_item, item: @first, quantity: 5)

 @donation.line_items << create(:line_item, item: @first, quantity: 10)

 expect(@distribution.copy_line_items(@donation.id).count).to eq 2

it "#copy_line_items" do

TBH I think it should actually be
describe "#copy_line_items" and then the it example line actually
describe what the behavior is supposed to be. I realize that I'm probably
the one that wrote it as it "copy_line_items" initially. I'll make the
tweak.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rubyforgood/diaper/pull/427#discussion_r194933167,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD2ut6NnqBfB67uHLpXtBucNDN3JZjo5ks5t8GkHgaJpZM4Uh678
.


Fix blob field display

Thanks.
Please provide a small description next time you provide a pull request. It makes reviewing simpler and the pull request is likely to get in quicker :-)


Fix #856: Only cache failed scenario's for rerun

Tests are failing.

grrr.. Sorry about that. Fixed it.
Thanks @everzet

Fixes #856

would be great to add a scenario covering this to prevent regressions

Thanks @stof,
I will try get to sort this in coming week. Sorry quite busy with regular stuff...


fix: Apply isMuted to the participant list

Description Of Changes

Apply isMuted to the participant list

Operator can unmute the muted participants on the participant list

UIKIT-3712

@HoonBaek The Participant class was introduced in v4.3.0, so the minimum required Chat SDK version seems to be up-to-date


[DO NOT MERGE]: tmp: annotate exceptions

@mamiksik  @phanak-sap Please, comment in the commit which serves the only purpose to mark exceptions we need to deal with.
See the file exceptions.txt and verify I didn't miss any case.
Then we should go through the exceptions again and classify them by the rules in exceptions.txt

I think you did not miss any, however, looking at the code I think it would be good if a custom exception would be raised in method parse at line 159 if a value is None and in method property at line 1 222 when a property is not found.
Also before you pushed this commit I compiled the list of changes for last(closed) PR as requested https://github.com/mamiksik/python-pyodata/wiki/Exception-change-list, but I think there is of little use for it now.

I think it is generally OK. I would like split the "service block" to distinguish somehow "metadata invalid" use case for usage pyodata for example only as metadata validator - therefore logically out of "service" block since there can be no networking at all, only local XML file parsing.
I should be able to distinguish exceptions that came out during the phase "model building/metadata validation" from phase "runtime exception from model from valid/invalid metadata depeding on set policies".
I should have clear subset of exceptions that can be thrown from metadata validation and if none of that is thrown, I consider the metadata valid.
Other possibility is to have really method for only metadata parsing and validation, and not be hidden under client initialization (in use case when I don't really want to use the generated model to access any data)
pyodata.Client('NOT_VALID', None, metadata=local_metadata, config=custom_config)

@filak-sap @phanak-sap If there aren't any further objections should I implement the exception as described in this PR?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


fix bug where sometimes the runtime would not be present on output

What kind of change does this PR introduce?
bugfix
Did you add tests for your changes?
yes
Summary
fixed bug where sometimes the runtime would not be present on output
Does this PR introduce a breaking change?
i don't think so, more likely improves compatibility with webpack 3
Other information
Hello webpack team
I found a bug where sometimes the runtime would not be present on output
I encountered it by using:

optimization.runtimeChunk: true
webpack.optimize.MinChunkSizePlugin
Dinamic imports

It seems the runtime chunk would get integrated into one of the asyncChunks, and after integration i would get removed from the chunk graph, causing it's EntryPoint.runtimeChunk to be a chunk thats not longer on the Graph.
Why is it getting integrated? Well is small since it has 0 modules, shares the same groups as the asyncChunk, and especilly shares the same EntryPoint
Example output from the new runtime-chunk-integration statsTestCase
BaseLine

optimization.runtimeChunk: false
webpack.optimize.MinChunkSizePlugin not used

   Asset       Size  Chunks             Chunk Names
    0.js  287 bytes       0  [emitted]
    1.js  296 bytes       1  [emitted]
    2.js  290 bytes       2  [emitted]
main1.js   7.32 KiB       3  [emitted]  main1
Entrypoint main1 = main1.js
[0] ./b.js 20 bytes {0} [built]
[1] ./c.js 20 bytes {1} [built]
[2] ./d.js 20 bytes {2} [built]
[3] ./main1.js 66 bytes {3} [built]

Before

optimization.runtimeChunk: true
webpack.optimize.MinChunkSizePlugin used
Where are the 7KiB of the runtime?
"./b.js", "./c.js", "./d.js"got integrated

   Asset       Size  Chunks             Chunk Names
    0.js  728 bytes       0  [emitted]
main1.js  539 bytes       1  [emitted]  main1
Entrypoint main1 = 0.js main1.js
[0] ./main1.js 66 bytes {1} [built]
[1] ./b.js 20 bytes {0} [built]
[2] ./c.js 20 bytes {0} [built]
[3] ./d.js 20 bytes {0} [built]

After

optimization.runtimeChunk: true
webpack.optimize.MinChunkSizePlugin used
Runtime now present
"./b.js", "./c.js", "./d.js"got integrated

      Asset       Size  Chunks             Chunk Names
       0.js  719 bytes       0  [emitted]
   main1.js  542 bytes       1  [emitted]  main1
manifest.js   7.73 KiB       2  [emitted]  manifest
Entrypoint main1 = manifest.js main1.js
[0] ./b.js 20 bytes {0} [built]
[1] ./c.js 20 bytes {0} [built]
[2] ./d.js 20 bytes {0} [built]
[3] ./main1.js 66 bytes {1} [built]

Proposals for resolution:
Option 1
Everytime a chunk that chunk.hasRuntime() === true gets integrated; make sure the its Entrypoint.runtimeChunk gets set to undefined Entrypoint.setRuntimeChunk(undefined)
Pros

none

Cons

The runtime chunk would likely get integrated to another chunk; users explicitly setting optimization.runtimeChunk most likely don't want this

Option 2

Set a new property on Chunk Chunk.isRuntimeOnly
check against it on Chunk.canBeIntegrated()

Pros

Doesn't integrate the runtimeChunk, so users have a predictable output

Cons

Reduces the cohesion of the Chunk class

Option 3

Subclass Chunk, RuntimeChunk
Create method Chunk.isRuntimeOnly() --> false
Create method RuntimeChunk.isRuntimeOnly() --> true
Check against it on Chunk.canBeIntegrated()

I personally like this one best, seems to be the approach you follow with ChunkGroup and EntryPoint
Pros

Doesn't integrate the runtimeChunk, so users have a predictable output

Cons

New bug: What if a namedChunk is called "manifest", and the optimization.runtimeChunk.name: "manifest", current behavior would merge them together, witch is correct.

this solution would not merge them, not sure how to solve it.
in this case the namedChunk is not a RuntimeChunk, is just a Chunk

Reduces the cohesion of the Chunk class, arguably not as much
Too many classes?

This Pull-request
This PR follows Option 2
So i think it's best not to allow a Chunk.isRuntimeOnly to be integrated onto another chunk
One question that i can't seem to answer is:
Should a Chunk.isRuntimeOnly allow other chunk to be integrated onto it self?
I think yes, but i am not sure, this PR assumes yes
Thoughts ?

Thanks


Feat rc error pages

Add error pages based on mockups here, removing old error page: https://app.zeplin.io/project.html#pid=572ab8d06aa0cb081f94fadc&sid=5754d19884d6ad0c7579ddd5
The not found page handles any not founds.  Will need to redirect to the error page if something does go wrong.
In app/config/routes
  <Route path="*" component={NotFound} />
   <Route path="error" component={SomethingWentWrong} />

JSX Files Added

app/components/Errors/SomethingWentWrong.jsx
app/components/Errors/NotFound.jsx

CSS Files added:

app/stylesheets/pages/somethingWentWrong.css
app/stylesheets/pages/notfound.css

Looks great! Amazing job! :) :+1:


TrinamicException: Invalid Value but still moving

When calling some functions the motor reacts as accepted, but the code trows an error.
For example:
motor.rotate_right(1600)

but if I call it again several times I sometimes get:
---------------------------------------------------------------------------
TrinamicException                         Traceback (most recent call last)
<ipython-input-38-1ab72c70bd37> in <module>()
----> 1 motor.rotate_right(1700)

~/anaconda3/lib/python3.6/site-packages/TMCL-1.2-py3.6.egg/TMCL/motor.py in rotate_right(self, velocity)
     69
     70         def rotate_right(self, velocity):
---> 71                 reply = self.send(Command.ROR, 0, self.motor_id, velocity)
     72                 return reply.status
     73

~/anaconda3/lib/python3.6/site-packages/TMCL-1.2-py3.6.egg/TMCL/motor.py in send(self, cmd, type, motorbank, value)
     51
     52         def send( self, cmd, type, motorbank, value ):
---> 53                 return self.bus.send(self.module_id, cmd, type, motorbank, value)
     54
     55         def stop( self ):

~/anaconda3/lib/python3.6/site-packages/TMCL-1.2-py3.6.egg/TMCL/bus.py in send(self, address, command, type, motorbank, value)
     54                         rep = self.serial.read(REPLY_LENGTH)
     55                         reply = Reply(struct.unpack(REPLY_STRUCTURE, rep))
---> 56                         return self._handle_reply(reply)
     57
     58         def get_module( self, address=1 ):

~/anaconda3/lib/python3.6/site-packages/TMCL-1.2-py3.6.egg/TMCL/bus.py in _handle_reply(self, reply)
     92         def _handle_reply (self, reply):
     93                 if reply.status < Reply.Status.SUCCESS:
---> 94                         raise TrinamicException(reply)
     95                 return reply
     96

TrinamicException: Invalid Value

This seems to be a timing issue.
Thanks for your help.

TrinamicException instances are raised when the controller module responds to a command, but with a non-success status code. This implies that the python code is functioning properly, but that the module is unwilling to execute the command (however I can't hypothesise as to why at this time).
Response status codes are mapped in accordance with the TMCL docs (http://www.mctechnology.nl/pdf/TMCL_reference_2015.pdf#page=7) .
Possible workaround
I assume the purpose of multiple high-speed calls to the rotate_right command are to change the speed of the motor while it is rotating. The rotate_right method maps to TMCL command ROR which is described in the docs as follows:

Internal function: First, velocity mode is selected. Then, the velocity value is transferred to axis
parameter #0 ("target velocity").

It might be that repeated high-speed attempts to put the module into 'velocity mode' is what is causing the issue (I'm totally guessing here though). You could try calling ROR only once when motion starts, and then on successive calls update the 'target velocity' axis parameter directly instead.

It seems that every second call I get this error. But the strange thing is that the motor reacts as expected. This error actually arises in all functions that I call. (STOP, ROL, ROR, ...)

I solved this issue. The problem arises when more then one motor is connected. When you send a message at one motor, all the motors are answering. The status byte of the messages of the wrong motors is always 4 ('wrong value'). I modified the bus class so is works fine now.
'''python
checksum = self._binaryadd(address, command, type, motorbank, value)
msg = struct.pack(MSG_STRUCTURE, address, command, type, motorbank, value, checksum)
        self.serial.write(msg)

        while True:
            rep = self.serial.read(REPLY_LENGTH)

            if len(rep) < 9:
                    raise ValueError('increase the waiting time!!!')

            reply = Reply(struct.unpack(REPLY_STRUCTURE, rep))

            if reply.status == 100 and reply.module_address != address:
                raise ValueError('two messages interfered: only send one message at the time!')

            if reply.module_address == address:
                break

        #print ('Status: ',reply.status, 'Motor: ',reply.module_address,'Command: ',reply.command,'Value: ',reply.value)

        time.sleep(0.01)

        return self._handle_reply(reply)

'''


Use a gembox

We can use a gembox to group all the mgems needed to compile MRuby, this way we can avoid repetition in the build configuration of each platform.

Thank you for this, I wasn't aware of this feature! If you add a CHANGELOG entry I'll be happy to merge

Done @HellRok :)

Sorry, it seems like I used single quotes instead of double quotes in some parts of the code, I can hook you up a quick PR to fix that if you want.

Don't worry, I'll fix it up now. It's an easy mistake to make.
I really should let some parts of the build run automatically on PRs


test: Add an e2e test for the COMPOSE_FILE environment variable in fi…

…nch compose command
The following pull request attempts to modify finch compose command to allow the COMPOSE_FILE environment variable.

https://github.com/runfinch/finch/pull/994

Therefore, this fix adds an e2e test to run finch compose command with the COMPOSE_FILE environment variable.
Issue #, if available: N/A
Description of changes: The details are described in this commit message.
Testing done: Yes

[x] I've reviewed the guidance in CONTRIBUTING.md

License Acceptance
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Thanks for review!!!, @pendo324
I have fixed based on the review, so could you please re-review when you have time ???

Next step is to create a new release of common-tests and bump the version in finch/finch-core. I can take care of that 👍

Thank you !!!
Also, sorry, I assigned the following PR .
Could you please review if the test failure in Windows environment is resolved 🙇

https://github.com/runfinch/finch/pull/994


feat, cli: add fern generator upgrade command

Would be great to add some end-to-end tests: https://github.com/fern-api/fern/blob/f965520030b6a1f99dda0b1caeeabc6fdef5462a/packages/cli/ete-tests/src/tests/add/add.test.ts#L10-L12
I think it should be pretty easy with the runFernCli utility

A basic fern add fern-openapi --group my-group fails now and incorrectly updates the generators.yml. Also pretty tough to revert since there have been changes on top of this


add a query to check if a burner address is valid

Description/Reasoning

MS triggers a confirmation of transaction for every transfer event on EVM chains, even when those transfers are to non-burner addresses. that is, we monitor the contract and even if there is a erc20 transfer within the chain from a user to another user, we trigger an event. [there is not easy way to avoid this.]
We want to skip trying to confirm deposits to non-burner addresses.
Add a query s.t. on input (burner address, evm chain) returns  (true/false if the burner address was created for that chain, destination address if true). The MS will then, given an event to an address, check if it's a valid burner address, and only then try to confirm the deposit to it.

Current Behaviour
Expected Behaviour
Steps to reproduce (for bugs)
Relevant Logs or Files

cc @jack0son


Running detekt on JVM target (in KMP project) doesn't run against the commoon source set

Expected Behavior
When applying Detekt on a KMP project, with commonMain and jvmMain source sets, the task detektJvmMain  should apply rule detection to all source files contributing to the JVM target, that is, both commonMain and jvmMain.
Observed Behavior
In the case mentionned above,  the task detektJvmMain only applies rule detection to the kotlin files in the jvmMain source set, but not the shared files from the commonMain source set.
Steps to Reproduce
Project
Sample project attached : sample.zip.
If not available, just create a simple new KMP project with the following files:
build.gradle.kts
plugins {
    kotlin("multiplatform") version "1.9.22"
    id("io.gitlab.arturbosch.detekt") version ("1.23.5")
}

kotlin {
    jvm {
        jvmToolchain(17)
        withJava()
    }
}

repositories {
    google()
    mavenCentral()
}

src/commonMain/kotlin/CommonClass.kt
class CommonClass {
    val my_property: Int = 0
}

src/jvmMain/kotlin/JvmClass.kt
import java.util.Date

class JvmClass {
    val my_property: Date = Date()
}

Reproduction test

run the detektMetadataMain gradle task

notice it finds (at least) one issue in the CommonClass (i.e.: Variable names should match the pattern: [a-z][A-Za-z0-9]* [VariableNaming])

run the detektJvmMain  gradle task

notice it only finds issues in the JvmClass (at least: Variable names should match the pattern: [a-z][A-Za-z0-9]* [VariableNaming])

Context
Some of our custom rules rely on type resolution which is only availbale when running against the JVM target.
Your Environment

Version of detekt used: 1.23.5
Version of Gradle used (if applicable): 8.4
Gradle scan link (add --scan option when running the gradle task): https://gradle.com/s/u467js5p4dob2
Operating System and version: Mac OS Ventura 13.6.4
Link to your project (if it's a public repository):

I addressed this in a convention plugin that I use.
See https://github.com/detekt/detekt/issues/6634#issuecomment-1996012038 and https://github.com/detekt/detekt/issues/6634#issuecomment-1996216535 for some pros and cons to this approach

Thanks a lot for your insights @eygraber


Add (sticky) footer component

Maybe it's time to add new component for footers / sticky footers into bootstrap 4 out of the box? I can make PR if you agree.

You mean like http://v4-alpha.getbootstrap.com/examples/sticky-footer/ ?

That have a flexbox enabled option that isn't a fixed height?
https://philipwalton.github.io/solved-by-flexbox/demos/sticky-footer/

@cvrebert yes, sticky footer, base footer class, colors, border-radius, whatever else easily configured using variables.

Going to pass on doing that. We have the example, and it'd be nice to have a flexbox version of that eventually, but not adding a new component for that.

Because of bloat or...?

Because it could be done any number of ways and I don't see value in building one particular method into the project. Full or fixed width, rounded or not, attached or spaced away from the viewport, overlaying other content or another pane outside the rest of the content, etc. That kind of stuff is what the examples and extra CSS outside the project are for :).

I agree with you @mdo
I got confused and thought the example was a component. :+1:


Create an abstract Any for JSON/YAML

Merges duplications between YAML::Any and JSON::Any to a common Any abstract struct, which is inherited.
External shards implementing configurations formats will also be able to inherit from it.
Partially related to https://github.com/crystal-lang/crystal/issues/5155
Note: this PR is a background work to implement this POC https://gist.github.com/j8r/2820157756ce55182c85667cb0456800 in a future PR - the ability to deal with dynamic keys:
js = JSON.parse %({"a": {"b": "c"}, "e": [{"o": "p"}, 12, 2]})

p js[["a", "r"]]?  #=> nil
puts js[["e", 0, "o"]] = JSON::Any.new "test"
puts js        #=> {"a" => {"b" => "c"}, "e" => [{"o" => "test"}, 12_i64, 2_i64]}
js.delete ["e",0, "o"]
puts js        #=> {"a" => {"b" => "c"}, "e" => [{}, 12_i64, 2_i64]}

@j8r Sorry, but these PRs should be discussed before implemented.
I personally agree with one of Go's proverb:

A little copying is better than a little dependency

Just because the code is similar, it shouldn't involve a dependency.
So a strong 👎 from me here.

Note that the API and implementations between the Anys aren't the same:
no YAML::Any#as_bool, YAML::Any#as_f32  and JSON::Any#as_bytes
In JSON::Any, there are as_i if @raw.is_a?(Int) and in YAML::Any @raw.as?(Int64).try &.to_i

I think we need more analysis and experiments on (de)serialization before conducting such changes. For example I'd love some attempts at a Serde like solution (from Rust).

I'll create a shard because for now dealing with dynamic mappings isn't possible in the stdlib

Yes, please experiment and return back your results, both successes and shortcomings :)

For information: a reposiotry is created, https://github.com/j8r/dynany


Connection abort by the error "DCID not found"

hi @dtikhonov , i  have a problem when use lsquic to connect to server.
sometimes client  will receive a retry packet, and than raise an error
"conn: Abort connection: DCID not found"
it will cause quic connection broken.
i ever think that whether there is something wrong about this api "iquic_esfi_reset_dcid”。。。
i'm trying to reproduce...
do you have some ideas about it?
lsquic version is 2.7.0

The "DCID not found" error message comes from this code:
    if (cce >= END_OF_CCES(lconn))
    {
        ABORT_WARN("DCID not found");
        return -1;
    }

This is in function on_dcid_change(), called here:
        parse_regular_packet(conn, packet_in);
        if (saved_path_id == conn->ifc_cur_path_id)
        {
            if (conn->ifc_cur_path_id != packet_in->pi_path_id)
                on_new_or_unconfirmed_path(conn, packet_in);
            else if (!LSQUIC_CIDS_EQ(CN_SCID(&conn->ifc_conn),
                                                    &packet_in->pi_dcid))
            {
                if (0 != on_dcid_change(conn, &packet_in->pi_dcid))
                    return -1;
            }
        }

This means that the incoming packet has DCID that this end point has not issued!  This results in a connection error.  This may be a bug in the server.  Do you know which server implementation this is you're connecting to?

Why do you think it's iquic_esfi_reset_dcid()?  Are you sure it's a Retry packet that causes this?
If iquic_esfi_reset_dcid() failed it would be pretty odd.  It would mean that setup_handshake_keys() failed.  This, in turn, means either that there is a memory allocation failure (the likelihood of this is low) or that there is some buffer size mismatch error or an error in calling a BoringSSL function.  There are only a few possible errors.  You should add log messages so that we can see exactly where it fails next time it hits.

this is the detail log. please help to have a look.
dcid_issue.log
this is my analysis.

Question:

if client receive a different DCID packet to cause DCID not equal, is it a normal logic of quic? whether the quic library need to support this case? always to switch packet DCID to connection SCID?
here we can see if receive a packet DCID is not found in local connection lists, lsquic occur a "DCID not found" error. for these packets, quic should abort connection or just ignore them?
in the image, you will find before the last DCID not equal happened, client received a RETIRE_CONNECTION_ID frame, so if the client receive this frame, whether the client will still receive illegal frames(with invalid DCID) in a normal quic flow?
in you option, is it a server quic(my server quic version is quicgo library) error or client quic error or it just a normal case?

some more other question about lsquic

why it generate so much NEW_CONNECTION_ID frame, just to resend new connection id frame?

if receive out-of-order packets, what will happend that lsquic wanted？

Thank you for the logs!  Your analysis is very good -- and the bug is indeed in lsquic.  Here we are tripped up by something unusual that lsquic client does behind the scenes in some circustances.  Explanation follows.
Because all non-deprecated, non-experimental gQUIC and IETF QUIC versions are enabled by default, lsquic has to cope with the odd property of Q046 and Q050 versions.  In these versions, the server never includes a CID into the packets its sends to the client. This means that a client would not be able to differentiate between two connections by CID when a packet is received.  To work around this, lsquic identifies connections by the local port number instead of the connection ID.  It uses the following code to decide which to pick:
static int
hash_conns_by_addr (const struct lsquic_engine *engine)
{
    if (engine->flags & ENG_SERVER)
        return 0;
    if (engine->pub.enp_settings.es_versions & LSQUIC_FORCED_TCID0_VERSIONS)
        return 1;
    if ((engine->pub.enp_settings.es_versions & LSQUIC_GQUIC_HEADER_VERSIONS)
                                && engine->pub.enp_settings.es_support_tcid0)
        return 1;
    if (engine->pub.enp_settings.es_scid_len == 0)
        return 1;
    return 0;
}

(As you can see, Q043 has the same issue when es_support_tcid0 is enabled -- which it is by default.)
Thus, after CID 324D2EE8D711FDF2 is retired, the following incoming packets that have DCID set to this value, the engine should not be able to find matching connection for this packet and the packet should be dropped.  But (and this is the bug!), since connections are identified by the local port number, the engine knows which connection it is for.  The connection logic does not know how to deal with this situation and aborts.
The proper way to fix it is to perform a check -- only when connections are identified by port number -- whether any SCID matches DCID that's in the incoming packet.  If there are no matching SCID, discard the packet immediately and not give it to the connection.  This will mimic the normal behavior when the connections are looked up in a hash by DCID.
To answer your questions:

Yes, getting a different DCID (as long as it is one of the issued CIDs) is normal.  A peer can change DCID at any time.
Explained above.
Explained above.
This is a bug in lsquic.

Generation of SCIDs is just something that happens at the beginning of a connection (here, when Retry has been handled).  A bunch of NEW_CONNECTION_ID frames are generated once at the beginning of the connection, and then the peer is free to use them.  When peer retires CIDs it was issued, this endpoint will issue more CIDs.
lsquic copes with out-of-order packets just fine.  Note that packets 7 and 5 (the first highlighted pair) are not out-of-order: they are in different Packet Number Spaces.

now I only use LSQVER_ID24 on client and server side. So if i want to fix or avoid this bug happened,  Is it ok to force set es_versioins = 1 << LSQVER_ID24 to make the lsquic not use address to find connection( use dcid to find connection)?
lsquic_engine_init_settings(&settings_, flags);
// the following codes is new added to avoid issue happened
settings_.es_versions = 1 << LSQVER_ID24;

Yes, this will work.

Fixed in 2.23.1 -- closing.


Type 'typeof internal' is not a constructor function type.

Hi @pradel thanks for the great package!
I wanted to report an issue when compiling a project using "esModuleInterop": true in the tsconfig.json:
node_modules/node-instagram/lib/stream.d.ts:4:30 - error TS2507:
Type 'typeof internal' is not a constructor function type.
declare class Stream extends EventEmitter {

I am closing the issues in this repo:
The legacy Instagram API is deprecated and will be disabled on June 29, 2020. More information instagram.com/developer.


remove CPU architecture check form installer

Hello,
I tried to install Macad|3D on a M1 Mac in Parallels Desktop, but the installer says:
"This program can only by installed on versions of Windows designed for the follwoing processor architectures: x64" and aborts.
Since Macos brings an x86_64 emulation layer, this warning/error should be removed. Macad runs flawlessly on M1/Parallels once it is installed. (I installed it on a real x86 Windows machine and copied the folder "C:\Program Files\Macad3D" to the mac and everything runs fine!)
BTW: great piece of software!

I am very happy to hear that Macad3d works in principle on an M1 Mac under Parallels Desktop. If the setup is (and remains) the only problem, then this should be easy to solve.
Instead of removing the prompt completely, I would issue a warning which can be confirmed. This should ensure that the user knows that there is a hardware incompatibility and confirms that emulation is taking place in some way.


Support HTTP POST with payload in load test template

Current load test template does not support HTTP POST with payloads. Enable this.

Resolved by #1129


Support direct project references instead of NuGet for fast development loop

Proposal: Support direct project references instead of NuGet for fast development loop
Summary
Using CsWinRT, it should be possible to add a winrt project reference to a test app or unit test project, and directly run the tests or the test app, without having to create a NuGet package.
Rationale
The recommended way of working with CsWinRT is creating an interop assembly and packing that as a NuGet package. That approach is fine for distributing libraries. But for the inner development loop as a library developer, this is very bad. Every time I change a line of source in the library, I have to build a new NuGet package with a new version number, I need to upgrade my unit test or test app projects to use the new NuGet version, only then I can build and run them. If I find out my change is not working, I have to rebuild it all again, create new NuGet, update, only to find out it might still not work. This also makes it difficult to automatically run unit tests on code changes, because the project would still reference and use the previous NuGet package, even if the source has changed.
This leads to horrible developer productivity. It must be possible to change a line of code in your winrt library and immediately launch the debugger or unit test!
Important Notes
If you directly run a test project with only a project reference (instead of NuGet package), you get "class not registered" exceptions. The reason is that the native dll is expected to be in e.g. runtimes\win10-x64\native, but the project reference will place them in the root output folder instead. If you move the native dlls into the corresponding output folders, the app or unit test will run fine.
Open Questions
In my opinion, this should be the default behavior, to allow seamless interop by default. But it could also be realized with a special property to be added in the project file. Not sure if the change would cause problems with existing projects, if it is enabled by default.

Currently blocked by #756.


DataTree.to_dict() method does not behave as expected

What happened?
I am working with DataTree and find it very useful! However, I think I found a bug in the .to_dict() method.
When I build a DataTree from a dict with DataTree.from_dict() and then want to get the dict again with DataTree.to_dict() the resulting dict differs from the original one.
What did you expect to happen?
I expected the two dicts to be the same. Instead, the root node receives a dict entry with an empty Dataset. I argue that empty nodes should not appear in the dict.
Minimal Complete Verifiable Example
import xarray as xr
from xarray.core.datatree import DataTree

example_dict = {"set1": xr.Dataset({"var1": xr.DataArray([1, 2, 3], dims = "time")}),
                "set2": xr.Dataset({"var1": xr.DataArray([7, 8, 9], dims = "time")})}
dt = DataTree.from_dict(example_dict)
dt.to_dict() == example_dict # False
# should be True imo

# or even just
DataTree.from_dict({}).to_dict == {} # False

MVCE confirmation

[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
[X] Complete example — the example is self-contained, including all data and the text of any traceback.
[X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
[X] New issue — a search of GitHub Issues suggests this is not a duplicate.
[X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output
No response
Anything else we need to know?
No response
Environment

:488: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility.

I argue that empty nodes should not appear in the dict.

In the case of the root node you're right that that extra dict entry is redundant. But in general that's not the case: imagine a tree with an empty leaf node - if we dropped that from the dictionary then called DataTree.from_dict we would completely lose that node, creating a tree with a different structure instead of round-tripping. For that reason we can't just have all empty nodes not appear in the dict.
However I suppose we could have all empty "intermediate nodes" not appear in the dict, as they get automatically reconstructed... I'm not sure that's a good idea either though - just having one dict entry per node always is a lot simpler, even though it does look weird in your case.

Basically in order to get round-tripping, the rule has to be either:
a) create one dict entry per node for all nodes, even if empty,
b) create dict entry per non-empty node, plus one for every empty leaf node.

Yes, I see that removing empty leaf nodes is problematic. I also realized that two different dictionaries can lead to the same DataTree:
dict1 ={"set1": xr.Dataset({"var1": xr.DataArray([1, 2, 3], dims = "time")}),
       "set2": xr.Dataset({"var1": xr.DataArray([7, 8, 9], dims = "time")})}
dict2 = {"/": xr.Dataset(),
       "/set1": xr.Dataset({"var1": xr.DataArray([1, 2, 3], dims = "time")}),
       "/set2": xr.Dataset({"var1": xr.DataArray([7, 8, 9], dims = "time")})}
dt1 = DataTree.from_dict(dict1)
dt2 = DataTree.from_dict(dict2)

xr.testing.assert_identical(dt1, dt2)

and therefore, the roundtrip property cannot work universally...
The reason why the creation of empty datasets was a problem for me in the first place is because this makes it harder to apply my functionality that worked on dictionaries of xr.Datasets before to DataTrees. Particularly, the creation of empty datasets in to_dict but also in .subtree or .items() can lead to unexpected behavior when contents of a DataTree should lead to the output of a single Dataset or DataArray, such as maybe #9349 . However, I see now how it might be better to handle the problem there and not in the to_dict method itself.


Links in Docs API section point to wrong URLs

here: http://shamansir.github.io/rpd/sections/api.html
For example, "Network" points to http://shamansir.github.io/rpd/network.html instead of here: http://shamansir.github.io/rpd/sections/network.html

This text also leads to nothing:

You may set a patch title here and, also optionally, define handlers for the events happening inside, this way:

Also, Patch Definition object is not described.

In this section: http://shamansir.github.io/rpd/sections/api.html#rpd-noderenderer
docs/random-on-click node renderer returns:
{ 'click': Kefir.fromEvents(...) }

, while it should be:
{ 'click':
  {  'valueOut': Kefir.fromEvents(...)  }
}

and several errors like this one, are below


Implement selective convolution in 1D CNN

Thanks for your great work! It gives me some thought about my work and I am going to use it to my 1D CNN. Can you please tell me where should I notice to modify? The shape of 2D CNN and 1D CNN is queit different.

Hi, @QueeneTam, thanks for your interest.
I think you can modify most of the components of SelectiveConv2d in a straightforward way, e.g., you may need to change:

BatchNorm2d -> BatchNorm1d
_distributed_channel(): you should adapt (H,W) -> L

One issue I guess is that affine_grid and grid_sample are currently hard to directly adapt to 1d version afaik, as their implementations currently supports only for 4d inputs. They both are for training "shifting" parameters, and you may not need them if it is ok to use our shifting components with random parameters without training: in case of 2d conv, we have observed that this "random shifting" also works fine.

affine_grid

Thanks for you kind-hearted reply, and I will try my best to make it work for my experiment.

Hi, @QueeneTam, thanks for your interest.
I think you can modify most of the components of SelectiveConv2d in a straightforward way, e.g., you may need to change:

BatchNorm2d -> BatchNorm1d
_distributed_channel(): you should adapt (H,W) -> L

One issue I guess is that affine_grid and grid_sample are currently hard to directly adapt to 1d version afaik, as their implementations currently supports only for 4d inputs. They both are for training "shifting" parameters, and you may not need them if it is ok to use our shifting components with random parameters without training: in case of 2d conv, we have observed that this "random shifting" also works fine.

Thanks for you kind-hearted reply, and I will try my best to make it work for my experiment.


Using awswrangler 2.4.0 with glue 2.0 in --additional-python-modules results in Error

Describe the bug
Using awswrangler 2.4.0 with glue 2.0 in --additional-python-modules results in Error.
In an GlueJob error log, I find the message OSError: 'git' was not found
To Reproduce

create a glue Py Spark Job with a py script and Glue Version 2.0
inside the script, I have import awswrangler as aw
the Job has parameter --additional-python-modules awswrangler
I run the Job
Job fails with no module named awswrangler
so far I can see, this did not happen with Version 2.3.0
Workaround so far

pinning the version to 2.3.0 with --additional-python-modules awswrangler==2.3.0

CloudWatch ErrorLog in Detail:
INFO	2021-02-04 08:12:53,574	9223	com.amazonaws.services.glue.PythonModuleInstaller	[main]	Collecting awswrangler  Downloading https://files.pythonhosted.org/packages/28/57/debf8f714d5b1a14ce39a6e8bbd1105beaab8490f96c2673fe04bf27cedb/awswrangler-2.4.0-py3-none-any.whl (171kB)Collecting botocore<2.0.0,>=1.15.49 (from awswrangler)  Downloading https://files.pythonhosted.org/packages/f7/d1/35591aa624c5f450e32eab12fec2b03ad51b7ad656b7fa17acf04b474372/botocore-1.20.1-py2.py3-none-any.whl (7.2MB)Requirement already satisfied: numpy<1.21.0,>=1.18.0 in /home/spark/.local/lib/python3.7/site-packages (from awswrangler)Collecting pg8000<1.18.0,>=1.16.0 (from awswrangler)  Downloading https://files.pythonhosted.org/packages/ed/d9/2a20d2f41d59b7b814f7cb30e8df9bbd288b000ad8a04dbc97d8c92df6df/pg8000-1.17.0-py3-none-any.whlRequirement already satisfied: pymysql<1.1.0,>=0.9.0 in /home/spark/.local/lib/python3.7/site-packages (from awswrangler)Collecting openpyxl~=3.0.0 (from awswrangler)  Downloading https://files.pythonhosted.org/packages/d4/c5/1a5f82b3020bfb27f21b302f96c8ae6a34475070015d1b1e0b197a97e2af/openpyxl-3.0.6-py2.py3-none-any.whl (242kB)Collecting redshift-connector~=2.0.0 (from awswrangler)  Downloading https://files.pythonhosted.org/packages/9a/87/ad4bdda8037bc1e1c36c227309a48321b2c6faaf6b01328a7341738e7424/redshift_connector-2.0.873-py3-none-any.whl (80kB)Collecting pyarrow<3.1.0,>=2.0.0 (from awswrangler)  Downloading https://files.pythonhosted.org/packages/62/d3/a482d8a4039bf931ed6388308f0cc0541d0cab46f0bbff7c897a74f1c576/pyarrow-3.0.0.tar.gz (682kB)    Complete output from command python setup.py egg_info:    Traceback (most recent call last):      File "<string>", line 1, in <module>      File "/tmp/pip-build-3qfyk1xp/pyarrow/setup.py", line 624, in <module>        url='https://arrow.apache.org/'      File "/home/spark/.local/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup        return distutils.core.setup(**attrs)      File "/usr/lib64/python3.7/distutils/core.py", line 108, in setup        _setup_distribution = dist = klass(attrs)      File "/home/spark/.local/lib/python3.7/site-packages/setuptools/dist.py", line 448, in __init__        k: v for k, v in attrs.items()      File "/usr/lib64/python3.7/distutils/dist.py", line 292, in __init__        self.finalize_options()      File "/home/spark/.local/lib/python3.7/site-packages/setuptools/dist.py", line 740, in finalize_options        ep.load()(self)      File "/home/spark/.local/lib/python3.7/site-packages/setuptools/dist.py", line 747, in _finalize_setup_keywords        ep.load()(self, ep.name, value)      File "/tmp/pip-build-3qfyk1xp/pyarrow/.eggs/setuptools_scm-5.0.1-py3.7.egg/setuptools_scm/integration.py", line 26, in version_keyword        dist.metadata.version = _get_version(config)      File "/tmp/pip-build-3qfyk1xp/pyarrow/.eggs/setuptools_scm-5.0.1-py3.7.egg/setuptools_scm/__init__.py", line 173, in _get_version        parsed_version = _do_parse(config)      File "/tmp/pip-build-3qfyk1xp/pyarrow/.eggs/setuptools_scm-5.0.1-py3.7.egg/setuptools_scm/__init__.py", line 119, in _do_parse        parse_result = _call_entrypoint_fn(config.absolute_root, config, config.parse)      File "/tmp/pip-build-3qfyk1xp/pyarrow/.eggs/setuptools_scm-5.0.1-py3.7.egg/setuptools_scm/__init__.py", line 54, in _call_entrypoint_fn        return fn(root)      File "/tmp/pip-build-3qfyk1xp/pyarrow/setup.py", line 542, in parse_git        return parse(root, **kwargs)      File "/tmp/pip-build-3qfyk1xp/pyarrow/.eggs/setuptools_scm-5.0.1-py3.7.egg/setuptools_scm/git.py", line 95, in parse        require_command("git")      File "/tmp/pip-build-3qfyk1xp/pyarrow/.eggs/setuptools_scm-5.0.1-py3.7.egg/setuptools_scm/utils.py", line 151, in require_command        raise EnvironmentError("%r was not found" % name)    OSError: 'git' was not found        ----------------------------------------

Hi @nikwerhypoport , thanks for reporting it.
For platforms without PyArrow 3 support (e.g. EMR):
pip install pyarrow==2 awswrangler
Or specifically for Glue PySpark Job:

Key: --additional-python-modules
Value: pyarrow==2,awswrangler

P.S. Also consider to freeze the Wrangler version for productive environments (e.g. awswrangler==2.3.0)

P.S. Also consider to freeze the Wrangler version for productive environments (e.g. awswrangler==2.3.0)

tested it successfully with --additional-python-modules pyarrow==2,awswrangler==2.4.0
so I have the opportunity to use 2.4.0 features already and can wait, until the arrow issue is fixed in future
Thx


Server-side rendering doesn't work

When using SSR with ReactDOMServer.renderToString I get following exception:
    Invariant Violation: Could not find "store" in the context of "Connect(ConnectedRouterWithContext)". Either wrap the root component in a <Provider>, or pass a custom React context provider to <Provider> and the corresponding React context consumer to Connect(ConnectedRouterWithContext) in connect options.
        at invariant (/home/user/app/node_modules/invariant/invariant.js:40:15)
        at Connect.renderWrappedComponent (/home/user/app/node_modules/react-redux/lib/components/connectAdvanced.js:162:32)
        at ReactDOMServerRenderer.render (/home/user/app/dist/node_modules/react-dom/cjs/react-dom-server.node.development.js:2494:55)
        at ReactDOMServerRenderer.read (/home/user/app/dist/node_modules/react-dom/cjs/react-dom-server.node.development.js:2357:19)
        at Object.renderToString (/home/user/app/dist/node_modules/react-dom/cjs/react-dom-server.node.development.js:2729:25)
        at renderToString (/home/user/app/dist/server/webpack:/src/server/indexServer.js:220:18)

Dependencies:
    "connected-react-router": "6.0.0",
    "react": "16.7.0",
    "react-dom": "16.7.0",
    "react-redux": "6.0.0",
    "react-router-dom": "4.3.1",
    "redux": "4.0.1",

Compoents:
    <Provider store={store}>
      <ErrorCatcher>
        <ConnectedRouter history={history}>
          <Switch>
            <Route path="/recorder/" component={Recorder} />
            <Route path="/" component={Regular} />
          </Switch>
        </ConnectedRouter>
      </ErrorCatcher>
    </Provider>

Rendering client-side only works.

Minimal testcase:
import ReactDOMServer from 'react-dom/server';
import React from 'react';
import { Provider } from 'react-redux';
import { ConnectedRouter, push } from 'connected-react-router';
import { createStore } from 'redux';
import createHistory from 'history/createMemoryHistory';

const history = createHistory();
const store = createStore(() => {});

console.log('DUMP', ReactDOMServer.renderToString((
  <Provider store={store}>
    <ConnectedRouter history={history}>

Hello

    </ConnectedRouter>
  </Provider>
)));

You may want to use <StaticRouter> on the server side.  Here's an example from a repo that adds SSR to a non-ejected CRA starter app.  https://github.com/cereallarceny/cra-ssr/blob/master/server/loader.js

Mee too

Me too, without SSR

same issue when using ssr

I'm using the connected-react-router v6 and still having the issue.

Just moved off of react-router to connected-react-router v6.(.3..2). We have an SSR setup and are running into the same issue.

When using SSR with ReactDOMServer.renderToString I get following exception:
    Invariant Violation: Could not find "store" in the context of "Connect(ConnectedRouterWithContext)". Either wrap the root component in a <Provider>, or pass a custom React context provider to <Provider> and the corresponding React context consumer to Connect(ConnectedRouterWithContext) in connect options.
        at invariant (/home/user/app/node_modules/invariant/invariant.js:40:15)
        at Connect.renderWrappedComponent (/home/user/app/node_modules/react-redux/lib/components/connectAdvanced.js:162:32)
        at ReactDOMServerRenderer.render (/home/user/app/dist/node_modules/react-dom/cjs/react-dom-server.node.development.js:2494:55)
        at ReactDOMServerRenderer.read (/home/user/app/dist/node_modules/react-dom/cjs/react-dom-server.node.development.js:2357:19)
        at Object.renderToString (/home/user/app/dist/node_modules/react-dom/cjs/react-dom-server.node.development.js:2729:25)
        at renderToString (/home/user/app/dist/server/webpack:/src/server/indexServer.js:220:18)

Dependencies:
    "connected-react-router": "6.0.0",
    "react": "16.7.0",
    "react-dom": "16.7.0",
    "react-redux": "6.0.0",
    "react-router-dom": "4.3.1",
    "redux": "4.0.1",

Compoents:
    <Provider store={store}>
      <ErrorCatcher>
        <ConnectedRouter history={history}>
          <Switch>
            <Route path="/recorder/" component={Recorder} />
            <Route path="/" component={Regular} />
          </Switch>
        </ConnectedRouter>
      </ErrorCatcher>
    </Provider>

Rendering client-side only works.

@zdila My solution is following：
In the store:
export const getStoreConfiguration = (preloadedState) => {
  const middlewares = [];

  let history;
  let reducers = {
    init(state, action) {
      return { ...state, ...action.payload };
    },
  };

  if (isClient()) {
    history = createBrowserHistory();
    reducers.router = connectRouter(history);
    middlewares.push(routerMiddleware(history));
  }

  return {
    store: createStore(
      combineReducers(reducers),
      preloadedState,
      composeEnhancer(applyMiddleware(...middlewares))
    ),
    history,
  };
};

isClient
export default () => {
  try {
    if (window && window.document && window.document.createElement) return true;

    return false;
  } catch (err) {
    return false;
  }
}

in the client
const { store, history } = getStoreConfiguration(preloadedState);
<Provider store={store}>
    <ConnectedRouter history={history}>
      <ClientApp />
    </ConnectedRouter>
  </Provider>,

The key is check the client or server, for difference env to create the store and history.

@zsirfs I am using solution provided in https://github.com/supasate/connected-react-router/issues/212#issuecomment-449390810. But still old approach used to work in older versions of connected-react-router (react-router-redux).

@zsirfs I am using solution provided in #212 (comment). But still old approach used to work in older versions of connected-react-router (react-router-redux).

Ok, I am using my solution work well.

I have an older project, but got it working. My newer project would probably work the same.
I had to use StaticRouter server-side 👎, but it worked when I made connectedRouter optional in combineReducers:
const createRootReducer = history => (
  combineReducers({
    collapsibleMenu: collapsibleMenuReducer,
    pageMeta: pageMetaReducer,
    playback: playbackReducer,
    ...(
      history
      && { router: connectRouter(history) }
    ),
    showSongs: showSongsReducer,
  })
)

Then, in my server renderer, I did this:
const context = {}
const store = compose()(createStore)(createRootReducer())

const renderedContent = renderToString(
  <Provider store={store}>
    <StaticRouter
      context={context}
      location={req.url}
    >
      <Pages />
    </StaticRouter>
  </Provider>
)

const renderedPage = renderSite(renderedContent, store.getState())

if (context.url) {
  console.log('redirect');
  res.writeHead(302, { Location: context.url })
  res.end()

} else {
  console.log(req.url);
  res.send(renderedPage).end()
}


chart.metadata.version is required

Hi, While creating Milvus cluster on gcp cloud getting "loading invalid entry for chart "milvus-operator" "" from https://zilliztech.github.io/milvus-operator/: validation: chart.metadata.version is required"
Pls suggest

@chetan-shah01 I checked that the chart.metadata.version for version 0.9.11 is missing, but it won't affect your installation.
I just get it fixed.  Where you have fixed the issue

@chetan-shah01
It looks fine now in my local env, could you still reproduce it ?


chore: Formatting, linting and manual revisions

Hi!
I really like your project, seeing it on LinkedIn.
I wanted to implement some standard practices to keep the code standard and pretty.
Feel free to ask any questions about the changes I made.

Add .gitignore
Run clippy --fix
Implement clippy suggestions
Run cargo fmt
Implement automatic fixes from clippy pedantic
Implement more suggestions by clippy::pedantic

Hey, thanks for the PR, I'll have a look at it when I get some free time.
Also I don't post on LinkedIn, is there a link to the post you saw?

Here is the post: https://www.linkedin.com/feed/update/urn:li:activity:7224757111858036736?utm_source=share&utm_medium=member_ios

Hey,
Thanks a lot for sharing,
I'm unable to view the post though, says "This post cannot be displayed"

Here is a screenshot

Nice, thank you :)

Hey,
Just had a look at this,
Thanks for the changes :)