[SPARK-24497][SQL] Support recursive SQL query

What changes were proposed in this pull request?
This PR adds recursive query feature to Spark SQL.
A recursive query is defined using the WITH RECURSIVE keywords and referring the name of the common table expression within the query.
The implementation complies with SQL standard and follows similar rules to other relational databases:

A query is made of an anchor followed by a recursive term.
The anchor terms doesn't contain self reference and it is used to initialize the query.
The recursive term contains a self reference and it is used to expand the current set of rows with new ones.
The anchor and recursive terms must be joined with each other by UNION or UNION ALL operators.
New rows can only be derived from the newly added rows of the previous iteration (or from the initial set of rows of anchor terms). This limitation implies that recursive references can't be used with some of the joins, aggregations or subqueries.

Please see cte-recursive.sql and with.sql for some examples.
Please note that this PR focuses on the minimal working implementation which means:

SQL recursion is actually loop where the current iteration is computed based on the previous one's result and when an iteration returns no rows the loop is over. The final result is the union of all iteration results. This means that caching intermediate results could speed up the process, but caching was removed from this PR to reduce complexity and can be added back in a follow-up PR.
A common way to stop SQL recursion is using the LIMIT operator to stop computing more than the required number of rows. LIMIT support was removed from this PR to reduce complexity and can be added back in a follow-up PR.
Some relational databases are more relaxed in terms how many anchor and recursive terms can be in a recursion. This PR allows the most simple case and allows only 1-1 of them. A follow-up PR can target to relax this limitation.

Why are the changes needed?
Recursive query is an ANSI SQL feature that is useful to process hierarchical data.
Does this PR introduce any user-facing change?
Yes, adds recursive query feature.
How was this patch tested?
Added new UTs and tests in cte-recursion.sql and with.sql.

@peter-toth , I see the generated physical plan is always "broadcast hash join" , is it expected ?
This is a really cool feature and I am testing it  on some big/large dataset.

Thanks for testing. It shouldn't genererate BHJ with RecursiveReference on
the broadcasted side. I will look into this tomorrow.
On Tue, Feb 26, 2019, 14:54 laskfla<EMAIL_ADDRESS>wrote:

@peter-toth https://github.com/peter-toth , I see the generated
physical plan is always "broadcast hash join" , is it expected ?
This is a really cool feature and I am testing it on some big/large
dataset.
[image: image]
https://user-images.githubusercontent.com/16917562/53417478-edd9e780-3992-11e9-8cbb-f4f251ab91cd.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/spark/pull/23531#issuecomment-467446052, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AG6vQyVZTmRxn4t385dvAiP9gvvpWAL3ks5vRTyPgaJpZM4Z9Ia9
.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/8342/
Test PASSed.

Test build #102854 has started for PR 23531 at commit 4808301.

Test build #102854 has finished for PR 23531 at commit 4808301.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/102854/
Test PASSed.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10866/
Test PASSed.

Test build #105614 has started for PR 23531 at commit cf13d46.

Test build #105614 has finished for PR 23531 at commit cf13d46.

This patch fails Spark unit tests.
This patch does not merge cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(name: String) extends LeafNode
case class RecursiveReference(name: String, output: Seq[Attribute]) extends LeafNode
case class RecursiveReferenceExec(name: String, output: Seq[Attribute]) extends LeafExecNode

Build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105614/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10868/
Test PASSed.

Test build #105616 has finished for PR 23531 at commit 530936c.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(name: String) extends LeafNode
case class RecursiveTable(
case class RecursiveReference(name: String, output: Seq[Attribute]) extends LeafNode
case class With(
case class RecursiveTableExec(
case class RecursiveReferenceExec(name: String, output: Seq[Attribute]) extends LeafExecNode

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105616/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10950/
Test PASSed.

Test build #105695 has started for PR 23531 at commit f90b8c2.

Test build #105695 has finished for PR 23531 at commit f90b8c2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105695/
Test PASSed.

I squashed previous commits and rebased it on master.
Also modified the implementation and took @maryannxue's suggestion into account:

Since Spark does not offer a stateless physical relation, we need all this hack to reset the state. I'm thinking can we go back to the first stateless construct here, i.e., the logical relation and restart each iteration from there? So instead of clearing up the state of each stateful physical nodes, we can create a clean copy for every iteration.

Some missing features compared to PostgreSQL: https://github.com/apache/spark/pull/23531#discussion_r249968739.
You can find ~90 tests in recursion.sql but let me know if you miss something.
@maryannxue @mgaido91 @gatorsmile @viirya @cloud-fan @dongjoon-hyun any feedback is very welcome.

Test build #105724 has started for PR 23531 at commit 0421bb2.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10981/
Test PASSed.

Test build #105724 has finished for PR 23531 at commit 0421bb2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105724/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11159/
Test PASSed.

Test build #105905 has started for PR 23531 at commit 17a3741.

Test build #105905 has finished for PR 23531 at commit 17a3741.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105905/
Test PASSed.

Changed to WIP as I think https://github.com/apache/spark/pull/24831 should be reviewed first.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13031/
Test PASSed.

Test build #107920 has started for PR 23531 at commit 9888da2.

Test build #107920 has finished for PR 23531 at commit 9888da2.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(name: String, cumulated: Boolean) extends LeafNode
case class RecursiveTable(
case class RecursiveReference(
case class With(
case class RecursiveTableExec(
case class RecursiveReferenceExec(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107920/
Test PASSed.

Test build #107937 has started for PR 23531 at commit b082aed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13047/
Test PASSed.

Test build #107937 has finished for PR 23531 at commit b082aed.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107937/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13161/
Test PASSed.

@dongjoon-hyun, @maropu I've rebased this PR on master and enabled the recursion related tests in with.sql (ported from PostgreSQL). Please let me know how to proceed with this.

Test build #108053 has started for PR 23531 at commit 80d2ed9.

ok, I'll review this tomorrow.

Test build #108053 has finished for PR 23531 at commit 80d2ed9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108053/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13202/
Test PASSed.

Test build #108094 has started for PR 23531 at commit bc0c666.

Test build #108094 has finished for PR 23531 at commit bc0c666.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108094/
Test PASSed.

hi, @peter-toth, great work!  just roughly looking over this pr though, my first impression was that this pr is a bit complicated for supporting WITH RECURSIVE as a first step. For better reviews, could you simplify this pr more? As for complicated cases, IMHO its ok to support them step-by-step. Thought?

hi, @peter-toth, great work! just roughly looking over this pr though, my first impression was that this pr is a bit complicated for supporting WITH RECURSIVE as a first step. For better reviews, could you simplify this pr more? As for complicated cases, IMHO its ok to support them step-by-step. Thought?

Thanks a lot @maropu for the review, I really appreciate your feedback! I will try to address you comments soon.
Regarding simplifying the PR, maybe I can remove the support for UNION combinator and maybe the limit handling and add them back in a follow up ticket. Unfortunately I don't see how to simplify this PR otherwise.

yea, I think its ok that the limit optimization will be addressed in follow-ups.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13321/
Test PASSed.

Test build #108221 has started for PR 23531 at commit 84bea75.

Test build #108221 has finished for PR 23531 at commit 84bea75.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108221/
Test PASSed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14293/
Test PASSed.

Test build #109226 has started for PR 23531 at commit cfe0a67.

Test build #109226 has finished for PR 23531 at commit cfe0a67.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109226/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14307/
Test PASSed.

Test build #109238 has started for PR 23531 at commit f13bb40.

Test build #109238 has finished for PR 23531 at commit f13bb40.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109238/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14363/
Test PASSed.

Test build #109296 has started for PR 23531 at commit 5dc624d.

Test build #109296 has finished for PR 23531 at commit 5dc624d.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109296/
Test PASSed.

Test build #109301 has started for PR 23531 at commit 7d7dba9.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14368/
Test PASSed.

Test build #109301 has finished for PR 23531 at commit 7d7dba9.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class RecursiveRelationExec(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109301/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14755/
Test PASSed.

Test build #109705 has started for PR 23531 at commit a834d18.

Test build #109705 has finished for PR 23531 at commit a834d18.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
 case class BasicOperators(queryExecution: QueryExecution) extends Strategy
case class SparkListenerSQLExecutionUpdate(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109705/
Test PASSed.

Test build #109944 has started for PR 23531 at commit f35a784.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14970/
Test PASSed.

Test build #109945 has started for PR 23531 at commit 1def9fa.

Test build #109945 has finished for PR 23531 at commit 1def9fa.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109945/
Test PASSed.

Test build #109944 has finished for PR 23531 at commit f35a784.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public final class JavaDirectKerberizedKafkaWordCount
trait AlterTableStatement extends ParsedStatement
case class DescribeTableExec(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109944/
Test PASSed.

cc @maryannxue

Reviewers, the current state in this PR focuses on the basic recursive functionality. I've removed some of the features (limit support, multiple anchor/recursive terms, ...) that are not required for a minimal implementation and can these will be addressed in follow-up PRs (cc. @maropu).
To avoid the issues came from physical plan reuse for different recursive iterations, each iteration starts off from the logical plan of the recursive part (using the statistics of the previous iteration to generate the best possible plan for the current iteration). This approach resulted that no changes are required in rules like 'ReuseExchangeand in adaptive execution, but comes with a minor change inQueryExecution`.
The reuse of common items between recursive iterations (like non correlated subqueries) are also not handled in this PR. I want to address them in follow-ups, for example subquery reuse between recursive iterations can be easily added based on this other PR: https://github.com/apache/spark/pull/25644
For metrics support I renamed some classes/methods of adaptive query execution (ex. SparkListenerSQLAdaptiveExecutionUpdate -> SparkListenerSQLExecutionUpdate) and I had to add a bit ugly change (https://github.com/apache/spark/pull/23531/files#diff-abfa38a36bdf86c8947ffa7e23e2a7a7R50) to pass QueryExection around.
Suggestions and feedback are welcome.
cc @gatorsmile @cloud-fan @maryannxue @mgaido91 @viirya @hvanhovell

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15277/
Test PASSed.

Test build #110281 has started for PR 23531 at commit 7597ac5.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15279/
Test PASSed.

Test build #110283 has started for PR 23531 at commit 8b76fa2.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15280/
Test PASSed.

Test build #110281 has finished for PR 23531 at commit 7597ac5.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110281/
Test FAILed.

Test build #110283 has finished for PR 23531 at commit 8b76fa2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110283/
Test PASSed.

Test build #110284 has finished for PR 23531 at commit dd165ca.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110284/
Test PASSed.

Test build #110505 has started for PR 23531 at commit 84c61bd.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15480/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/15541/
Test PASSed.

Test build #110565 has started for PR 23531 at commit 732ffa8.

Test build #110565 has finished for PR 23531 at commit 732ffa8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110565/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/15716/
Test PASSed.

Test build #111651 has started for PR 23531 at commit aa38268.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16647/
Test PASSed.

Test build #111651 has finished for PR 23531 at commit aa38268.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111651/
Test FAILed.

Test build #111993 has started for PR 23531 at commit 70c7782.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17007/
Test PASSed.

Test build #111993 has finished for PR 23531 at commit 70c7782.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111993/
Test PASSed.

Test build #114305 has started for PR 23531 at commit e1bad7c.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19145/
Test PASSed.

Test build #114305 has finished for PR 23531 at commit e1bad7c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(cteName: String, cumulated: Boolean) extends LeafNode
case class RecursiveRelation(
case class RecursiveReference(
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase
case class With(
 case class BasicOperators(queryExecution: QueryExecution) extends Strategy
case class RecursiveRelationExec(
case class SparkListenerSQLExecutionUpdate(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114305/
Test FAILed.

Test build #114325 has started for PR 23531 at commit 949a803.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19164/
Test PASSed.

Test build #114325 has finished for PR 23531 at commit 949a803.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(cteName: String, cumulated: Boolean) extends LeafNode
case class RecursiveRelation(
case class RecursiveReference(
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase
case class With(
 case class BasicOperators(queryExecution: QueryExecution) extends Strategy
case class RecursiveRelationExec(
case class SparkListenerSQLExecutionUpdate(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114325/
Test PASSed.

@cloud-fan @dongjoon-hyun @gatorsmile @hvanhovell @maropu @maryannxue @mgaido91 @viirya
I think recursive SQL would be a useful feature. If there is anything I can improve on this PR please let me know. Your feedback is gladly welcome.

Test build #114871 has started for PR 23531 at commit 6d143f5.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19694/
Test PASSed.

Test build #114871 has finished for PR 23531 at commit 6d143f5.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(cteName: String, accumulated: Boolean) extends LeafNode
case class RecursiveRelation(
case class RecursiveReference(
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase
case class With(
case class RecursiveRelationExec(
case class SparkListenerSQLExecutionUpdate(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114871/
Test PASSed.

Test build #118678 has started for PR 23531 at commit 7daa6aa.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23429/
Test PASSed.

Test build #118678 has finished for PR 23531 at commit 7daa6aa.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(cteName: String, accumulated: Boolean) extends LeafNode
case class RecursiveRelation(
case class RecursiveReference(
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase
case class With(
case class RecursiveRelationExec(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118678/
Test FAILed.

Test build #118683 has started for PR 23531 at commit 70ddbbe.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23434/
Test PASSed.

Test build #118683 has finished for PR 23531 at commit 70ddbbe.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(cteName: String, accumulated: Boolean) extends LeafNode
case class RecursiveRelation(
case class RecursiveReference(
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase
case class With(
case class RecursiveRelationExec(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118683/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23435/
Test PASSed.

Test build #118684 has started for PR 23531 at commit f4155a2.

Test build #118684 has finished for PR 23531 at commit f4155a2.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(cteName: String, accumulated: Boolean) extends LeafNode
case class RecursiveRelation(
case class RecursiveReference(
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase
case class With(
case class RecursiveRelationExec(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118684/
Test FAILed.

Test build #118714 has started for PR 23531 at commit b262e3e.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23465/
Test PASSed.

Test build #118714 has finished for PR 23531 at commit b262e3e.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(cteName: String, accumulated: Boolean) extends LeafNode
case class RecursiveRelation(
case class RecursiveReference(
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase
case class With(
case class RecursiveRelationExec(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118714/
Test PASSed.

Any progress for this? I'm looking forward to this feature!

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24530/
Test PASSed.

Test build #119800 has started for PR 23531 at commit b262e3e.

Test build #119800 has finished for PR 23531 at commit b262e3e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedRecursiveReference(cteName: String, accumulated: Boolean) extends LeafNode
case class RecursiveRelation(
case class RecursiveReference(
abstract class UnionBase extends LogicalPlan
case class Union(children: Seq[LogicalPlan]) extends UnionBase
case class With(
case class RecursiveRelationExec(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119800/
Test FAILed.

Test build #119818 has started for PR 23531 at commit 082e486.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24548/
Test PASSed.

Test build #119818 has finished for PR 23531 at commit 082e486.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119818/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24598/
Test PASSed.

Test build #119870 has started for PR 23531 at commit ec75262.

Test build #119870 has finished for PR 23531 at commit ec75262.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119870/
Test FAILed.

Test build #119891 has started for PR 23531 at commit 656995b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24617/
Test PASSed.

Test build #119891 has finished for PR 23531 at commit 656995b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119891/
Test PASSed.

retest this please

Test build #120335 has started for PR 23531 at commit 656995b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25047/
Test PASSed.

Test build #120335 has finished for PR 23531 at commit 656995b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120335/
Test FAILed.

retest this please

Test build #120411 has started for PR 23531 at commit 656995b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25119/
Test PASSed.

Test build #120411 has finished for PR 23531 at commit 656995b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120411/
Test PASSed.

retest this please

Test build #120774 has started for PR 23531 at commit 656995b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25472/
Test PASSed.

Test build #120774 has finished for PR 23531 at commit 656995b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120774/
Test PASSed.

seems like the PR addressed the feedback, can someone review?

Test build #121283 has started for PR 23531 at commit 8205e61.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25969/
Test PASSed.


Issue #44: Fix for XAMLBehaviorsSample solution not referencing nuget package 1.0.2

Hi @davidezordan, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;


feat: support local model path

Description
Supports configuration of the local model path to omit downloading.
Motivation
close #128.
Sometimes there is already a local model, or a network problem connecting to HuggingFace, so we want to use local model instead.
Now we can achieve this by configuring backend_config.pretrained_model_name_or_path, it both supports transformers and vllm backends, for example:
{
    "model": "facebook/opt-1.3b",
    "backend": "transformers",
    "num_gpus": 1,
    "auto_scaling_config": {
        "metric": "concurrency",
        "target": 1,
        "min_instances": 0,
        "max_instances": 10
    },
    "backend_config": {
        "pretrained_model_name_or_path": "/root/users/dblate/facebook/opt-1.3b",
        "device_map": "auto",
        "torch_dtype": "float16"
    }
}

Since the pull request contains many file changes, I'd like to state my points:

In config, model and pretrained_model_name_or_path should be two independent fields

model used to be an identifier, like as internal keys for routers or serverless format file path
pretrained_model_name_or_path used to get the model, it can be a HuggingFace model name, or local path

They can be the same sometimes, but not always

Feel free to modify my code or reject it if necessary.
Type of Change

[ ] Bug fix
[x] New feature
[ ] Breaking change
[ ] Documentation update

Checklist

[x] I have read the CONTRIBUTION guide.
[ ] I have updated the tests (if applicable).
[ ] I have updated the documentation (if applicable).

I will update doc and example later.

Hi,
We noticed that you closed the PR. Could you share your concerns with us?

Reopen

This must be a misoperation... I am dealing with a code conflict.

Okay, could you please request another PR so that we can review and merge?

I have requested a new PR: https://github.com/ServerlessLLM/ServerlessLLM/pull/177, please review at your convenience.


Installer uses incorrect permissions for QuickLook plugin in ~/Library

Hi there,
I installed qlImageSize through Homebrew Cask, the Spotlight plugin worked fine but there was absolutely no change to QuickLook whatsoever.
Upon further inspection, I noticed that in ~/Library/QuickLook/qlImageSize.qlgenerator/ was owned by root with permissions drwx------/700. Upon running sudo chmod 755 qlImageSize.qlgenerator/ && chown -R $(whoami) qlImageSize.qlgenerator/ it worked perfectly.

Dup of #19


Remove version: Jackett.Jackett version 0.21.2114

404 - release deleted
Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/Jackett/Jackett/releases/download/v0.21.2114/Jackett.Installer.Windows.exe
Response Error: The remote server returned an error: (404) Not Found.
(Automated message - build 918)


Deduplicate rows before notification associations unique constraint migration

We had an issue with some duplicates in the certificate_notification_associations table, this update to the migration will automatically delete duplicate rows before creating the unique index.

For others experiencing the same problem, my output from the broken migration:
INFO  [alembic.runtime.migration] Running upgrade<PHONE_NUMBER>de -> 449c3d5c7299, Add unique constraint to certificate_notification_associations table.
Traceback (most recent call last):
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
psycopg2.IntegrityError: could not create unique index "uq_dest_not_ids"
DETAIL:  Key (notification_id, certificate_id)=(9, 180) is duplicated.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/env/bin/lemur", line 11, in <module>
    load_entry_point('lemur', 'console_scripts', 'lemur')()
  File "/app/lemur/lemur/manage.py", line 556, in main
    manager.run()
  File "/app/env/lib64/python3.5/site-packages/flask_script/__init__.py", line 417, in run
    result = self.handle(argv[0], argv[1:])
  File "/app/env/lib64/python3.5/site-packages/flask_script/__init__.py", line 386, in handle
    res = handle(*args, **config)
  File "/app/env/lib64/python3.5/site-packages/flask_script/commands.py", line 216, in __call__
    return self.run(*args, **kwargs)
  File "/app/env/lib64/python3.5/site-packages/flask_migrate/__init__.py", line 95, in wrapped
    f(*args, **kwargs)
  File "/app/env/lib64/python3.5/site-packages/flask_migrate/__init__.py", line 280, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/app/env/lib64/python3.5/site-packages/alembic/command.py", line 254, in upgrade
    script.run_env()
  File "/app/env/lib64/python3.5/site-packages/alembic/script/base.py", line 427, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/app/env/lib64/python3.5/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
    module = load_module_py(module_id, path)
  File "/app/env/lib64/python3.5/site-packages/alembic/util/compat.py", line 145, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 697, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "migrations/env.py", line 75, in <module>
    run_migrations_online()
  File "migrations/env.py", line 68, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/app/env/lib64/python3.5/site-packages/alembic/runtime/environment.py", line 836, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/app/env/lib64/python3.5/site-packages/alembic/runtime/migration.py", line 330, in run_migrations
    step.migration_fn(**kw)
  File "/app/lemur/lemur/migrations/versions/449c3d5c7299_.py", line 24, in upgrade
    op.create_unique_constraint(CONSTRAINT_NAME, TABLE, COLUMNS)
  File "<string>", line 8, in create_unique_constraint
  File "<string>", line 3, in create_unique_constraint
  File "/app/env/lib64/python3.5/site-packages/alembic/operations/ops.py", line 399, in create_unique_constraint
    return operations.invoke(op)
  File "/app/env/lib64/python3.5/site-packages/alembic/operations/base.py", line 319, in invoke
    return fn(self, operation)
  File "/app/env/lib64/python3.5/site-packages/alembic/operations/toimpl.py", line 135, in create_constraint
    operation.to_constraint(operations.migration_context)
  File "/app/env/lib64/python3.5/site-packages/alembic/ddl/impl.py", line 177, in add_constraint
    self._exec(schema.AddConstraint(const))
  File "/app/env/lib64/python3.5/site-packages/alembic/ddl/impl.py", line 115, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
    compiled
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
    raise value.with_traceback(tb)
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/app/env/lib64/python3.5/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) could not create unique index "uq_dest_not_ids"
DETAIL:  Key (notification_id, certificate_id)=(9, 180) is duplicated.
 [SQL: 'ALTER TABLE certificate_notification_associations ADD CONSTRAINT uq_dest_not_ids UNIQUE (notification_id, certificate_id)'] (Background on this error at: http://sqlalche.me/e/gkpj)

Coverage remained the same at 60.935% when pulling 0f2e30cdae07f154c5b5809dc6e5ccee1c5e2158 on intgr:deduplicate-before-unique-migration into 0b39d0fa34ea8e7801e00b9369f156c05386eb38 on Netflix:master.


feat: allow stamping in repository name

This could be used to take the username of the person running the build.
Fixes #46

@alexeagle this is unblocked.

I believe this was fixed as a side-effect of the repository name moving into the repotags attribute which is already stampable.

I believe this was fixed as a side-effect of the repository name moving into the repotags attribute which is already stampable.

Per my https://github.com/bazel-contrib/rules_oci/issues/46#issuecomment-1718150022, we lost this ability.


Resources - Cluster logging and metrics

As for signals from the operator itself I think we have a couple of things already:

logs: output in JSON format and should be straightforward to use.
metrics: currently we only have a few metrics that shows how many of the different resource types the operator creates.
additional columns when listing humio clusters: https://github.com/humio/humio-operator/blob/master/pkg/apis/core/v1alpha1/humiocluster_types.go#L77-L79

If we want to keep log shipping and metric collection out of this project (which I agree with) and just keep them in the existing helm chart, I'm not sure how much more we have to fix on this one.
Perhaps we should consider closing this issue? @jswoods @schofield

works for me


explicit TLS disable

Aditya's branch includes a change to make the disabling of TLS explicit, rather than inferred by the absence of the -tls-cert and -tls-key command-line arguments to lightwalletd. This provides more security as it prevents the user from unknowingly not using TLS; the user must explicitly specify that TLS will not be used. Not required, but a good safety measure.

Turns out this was already there, in both versions, implemented independently (hackworks had the name -very-insecure, Aditya's repo had the name -no-tls. I'm merging these two as part of addressing review comments for https://app.zenhub.com/workspaces/wallet-5db8aa0244512d0001e0968e/issues/zcash-hackworks/lightwalletd/143 so I will not have a separate PR for this, will be part of https://github.com/zcash-hackworks/lightwalletd/pull/149.


python_linux_dbg_native.py27.test.unit.beta._face_interface_test.DynamicInvokerBlockingInvocationInlineServiceTest failing on Linux Master

https://grpc-testing.appspot.com/job/gRPC_master_linux/1701/testReport/junit/(root)/python_linux_dbg_native/py27_test_unit_beta__face_interface_test_DynamicInvokerBlockingInvocationInlineServiceTest/
Errors/Failures:
[91munit.beta._face_interface_test.DynamicInvokerBlockingInvocationInlineServiceTest.testSuccessfulUnaryRequestUnaryResponse[0m
[1mtraceback:[0m
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/var/local/git/grpc/src/python/grpcio_tests/tests/unit/framework/interfaces/face/_blocking_invocation_inline_service.py", line 94, in testSuccessfulUnaryRequestUnaryResponse
    request, test_constants.LONG_TIMEOUT, with_call=True)
  File "/var/local/git/grpc/py27/local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py", line 322, in __call__
    self._request_serializer, self._response_deserializer)
  File "/var/local/git/grpc/py27/local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py", line 208, in _blocking_unary_unary
    raise _abortion_error(rpc_error_call)
LocalError: LocalError(code=StatusCode.UNIMPLEMENTED, details="Method not found!")

P0? How reproducible?

P0 because it's happening on Master. It's certainly flaky.

How certain are you about "certainly"? Any idea of the flake rate? This issue smells like a duplicate of issue 9435 and if it is then rather than two tests that each have different problems it means that there's some very-low-rate spurious UNIMPLEMENTED/"Method not found!" defect happening across at least all Python tests.
I'd like to know what is the method is that is not found, but I'm wary of echoing in the details string possibly-hostile content sent by the client as "the method". Maybe I'll write some sanitization; our tests only ever use methods containing letters, forward slashes, and periods (I think).

I'm only technically correct, in that it flaked (once): https://grpc-testing.appspot.com/job/gRPC_master_linux/1701/testReport/junit/(root)/python_linux_dbg_native/py27_test_unit_beta__face_interface_test_DynamicInvokerBlockingInvocationInlineServiceTest/history/
However undesirable that is, it seems to have happened only once in a while. I'm removing my zealous P0. Would it be possible to echo the missing method from the test, as opposed to modifying the library code, which as you say would be too revealing?

It wouldn't be possible because (1) the problem is server-side and almost all the tests control the system under test from the client side and (2) the next time the problem is observed (which could be weeks given the time between issue 9435 and this issue, the only two reports) it could come up in any test, not just either of the two tests in which it's been observed.
I think I'll add server-side sanitization. Sit tight.

:+1: thanks!

Not seen in quite a while, closing.

Just saw this again:
https://sponge.corp.google.com/target?id=5664c3f3-315a-4dd2-8f6a-2fdceeb1f5f9&target=github/grpc/python_linux_opt_native&searchFor=&show=FAILED&sortBy=STATUS

Not seen in a long time & test also seems removed by  https://github.com/grpc/grpc/pull/14702.


Add a newline at the end of migration file

According to POSIX standard, a line should end with newline. This PR fixes no-newline at end of file.

Thanks!

:+1:


Svg import (#341)

SVG import #341

imports paths with additional coordinates after M (move-to) command
also imports paths with additional coordinates after L, A, C, S, Q and T commands
fixed a bug where coordinates with no leading 0 and no space between them were not being read correctly

I'm just figuring out how to build and run tests so should have some soon, but I have used the Playground to run this code
var makerjs = require('makerjs');
var m = makerjs.importer.fromSVGPathData("M2.873 11.297V4.142H1.699L0 5.379v1.137l1.64-1.18h.06v5.961h1.174Zm3.213-5.09v-.063c0-.618.44-1.169 1.196-1.169.676 0 1.174.44 1.174 1.106 0 .624-.42 1.101-.807 1.526L4.99 10.553v.744h4.78v-.99H6.643v-.069L8.41 8.252c.65-.724 1.237-1.332 1.237-2.27C9.646 4.849 8.723 4 7.308 4c-1.573 0-2.36 1.064-2.36 2.15v.057h1.138Zm6.559 1.883h.786c.823 0 1.374.481 1.379 <IP_ADDRESS>7-.55 1.216-1.421 1.21-.77-.005-1.326-.419-1.379-.953h-1.095c.042 1.053.938 1.918 2.464 1.918 1.478 0 2.642-.839 2.62-2.144-.02-1.143-.922-1.651-1.551-1.714v-.063c.535-.09 1.347-.66 1.326-1.678-.026-1.053-.933-1.855-2.359-1.845-1.5.005-2.317.88-2.348 1.898h1.116c.032-.498.498-.944 1.206-.944.703 0 1.206.435 1.206 <IP_ADDRESS>-.504 1.106-1.2 1.106h-.75v.96Z");
this.models = m.models;

It should write out "123"

Thank you very much! 🥇


Remove remaining PostGIS functionality

Date: Mon, 4 Jun 2018 15:16:02 +0100
From: Ian Lewis
To: Dr Abraham Martin Campillo
CC: Jon Warbrick
Subject: postgresql tfcweb postgis tidyup
For when we're done with the SmartPanel rollout...
I think at this stage combining Django and PostGIS is an unnecessary complication. From inspecting the code it seems we're barely using PostGIS in prod code, if at all (I see the tfc_gis app, and the 'gis_location' field in transport.Stop).
Migrating a Django database with PostGIS in it is problematic (although I'm sure fixable), e.g. I just hit this error:
https://stackoverflow.com/questions/18643998/geodjango-geosexception-error.
So unless I'm mistaken, I think we should plan for the PostGIS removal from tfc_web.

It is still used to find out which bus lines and bus stops have some defined areas: https://smartcambridge.org/transport/areas/
These areas are not squared (like in a map), but instead defined by a polygon (like the area of Cambridge in this case: https://smartcambridge.org/transport/area/1/). Using PostGIS is the only efficient way to find which objects (bus stops) are in this polygon.


Maybe we should look for "fdfind" and use it if it exists

Since its called fdfind on Debian based distributions, maybe we should look for this binary and use it if it exists.
Or at least give a helpful error message if fd doesnt exist. I think right now there is only a silent fail if it doesnt exist, which would make Ubuntu users think the plugin doesnt work.
It's possible to set the fd binary to something else in settings but maybe people don't look there and just assume the plugin fails. We should at least give a helpful error message. :)

Started working on this in a separate branch:
https://github.com/linux-cultist/venv-selector.nvim/tree/check_for_fd_and_fdfind

Plugin will do this now, closing issue.


Not work in Raspbian(Raspberry Pi 2)

Hi,
It did not work with the latest version.
Linux ARM supported?
System info:
Linux raspberrypi 3.18.12-v7+ #782 SMP PREEMPT Tue Apr 28 19:54:13 BST 2015 armv7l GNU/Linux
jdk-8-oracle-arm-vfp-hflt
Error log:
[3.8.2]
java.lang.Exception: No native library is found for os.name=Linux and os.arch=arm
at org.sqlite.SQLiteJDBCLoader.loadSQLiteNativeLibrary(SQLiteJDBCLoader.java:284)
at org.sqlite.SQLiteJDBCLoader.initialize(SQLiteJDBCLoader.java:65)
at org.sqlite.core.NativeDB.load(NativeDB.java:53)
at org.sqlite.core.CoreConnection.open(CoreConnection.java:136)
at org.sqlite.core.CoreConnection.(CoreConnection.java:66)
at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:21)
at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:23)
at org.sqlite.SQLiteConnection.(SQLiteConnection.java:45)
at org.sqlite.JDBC.createConnection(JDBC.java:114)
at org.sqlite.JDBC.connect(JDBC.java:88)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
[3.8.9]
java.lang.UnsatisfiedLinkError: org.sqlite.core.NativeDB._open(Ljava/lang/String;I)V
at org.sqlite.core.NativeDB._open(Native Method)
at org.sqlite.core.DB.open(DB.java:161)
at org.sqlite.core.CoreConnection.open(CoreConnection.java:218)
at org.sqlite.core.CoreConnection.(CoreConnection.java:74)
at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:24)
at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:23)
at org.sqlite.SQLiteConnection.(SQLiteConnection.java:45)
at org.sqlite.JDBC.createConnection(JDBC.java:114)
at org.sqlite.JDBC.connect(JDBC.java:88)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)

The problem is Rapsberry is actually requires armhf (hard-float), but jdk retuns os.arch=arm. So wrong library is loaded. We need to resolve this, first.
Thanks for reporting.

Released <IP_ADDRESS>, which fixes this problem.


MCUNet_V2 imbalanced memory usage distribution

Great work!
Could I get some detail of figure1(imbalanced memory usage distribution) in MCUNetV2, like which hardware and compiler are used for the base case? Model inference lots of data and then average them or just inference one data point. How to measure and compute a diagram. If there is a code for measurement would be fantastic.

Hi, thanks for your interest in our work. For Figure 1, we are actually analyzing MobileNetV2. We used analytic profiling, which is hardware and compiler agnostic. You can check the first paragraph of the Experiment section for details. But in short, such profiling counts the input and output activation size (for multi-branch structure, you need to consider all the concurrent memory usage).
We calculate the memory usage for batch size=1. The memory usage is the same for the same input size, so we do not need to average.

Thanks for answering my question!
The peak memory usage is the maximum feature map size in the block. Am I correct? Could you demonstrate the calculation for peak memory usage in MobileNetV2 since I can't get the same result as figure 1?
Additionally, there are only 17 bottlenecks in MobileNetV2. Is Block0 indicate the first regular convolution?

Hi, sorry for the delay. Yes, Block 0 indicates the first convolution.
Here is a code piece to reproduce the results in Figure 1:
import torch
import torch.nn as nn
import numpy as np
import torchvision

DATA_TYPE = 'int8'
INPLACE_DW = DATA_TYPE == 'int8'

assert DATA_TYPE in ['int8', 'fp']

def record_in_out_size(m, x, y):
    x = x[0]
    m.input_shape = list(x.shape)
    m.output_shape = list(y.shape)
    # we discard the batch dimension for now
    m.input_size = np.prod(m.input_shape[1:]).item()
    m.output_size = np.prod(m.output_shape[1:]).item()

def add_io_hooks(m_):
    # let's register the hook for all
    m_.register_forward_hook(record_in_out_size)

def estimate_conv(m):
    assert isinstance(m, nn.Conv2d), type(m)
    if INPLACE_DW and m.in_channels == m.out_channels == m.groups:  # inplace depthwise from MCUNetV1
        return m.input_size + np.prod(m.output_shape[2:])  # add a buffer with size equal to one channel
    else:  # normal conv
        return m.input_size + m.output_size

def estimate_mbconv(m):
    assert len(m.conv) in [3, 4]
    if DATA_TYPE == 'fp':  # with float point, we can use per channel compute for depthwise part
        if len(m.conv) == 3:
            return m.conv[0][0].input_size + m.conv[1].output_size
        else:
            return m.conv[0][0].input_size + m.conv[2].output_size
    else:  # normal implementation for int8 quantized case
        if m.use_res_connect:
            if len(m.conv) == 3:
                return max([
                    estimate_conv(m.conv[0][0]),
                    estimate_conv(m.conv[1]),
                ])
            else:
                return max([
                    estimate_conv(m.conv[0][0]),
                    estimate_conv(m.conv[1][0]) + m.input_size,
                    estimate_conv(m.conv[2])
                ])
        else:
            if len(m.conv) == 3:
                return max([
                    estimate_conv(m.conv[0][0]),
                    estimate_conv(m.conv[1])
                ])
            else:
                return max([
                    estimate_conv(m.conv[0][0]),
                    estimate_conv(m.conv[1][0]),
                    estimate_conv(m.conv[2])
                ])

def nelem2kb(n):
    # converting number of elements to kB according to the precision
    if DATA_TYPE == 'fp':
        n = n * 4  # 4 bytes per floating point
    return n / 1024

def estimate_model(model, x):
    model.eval()
    model.apply(add_io_hooks)

    with torch.no_grad():
        _ = model(x)

    memory_usage = list()

    # now estimate the memory of the backbone
    # first conv
    memory_usage.append(estimate_conv(model.features[0][0]))
    # blocks
    for b in model.features[1:-1]:
        memory_usage.append(estimate_mbconv(b))

    # now convert everything into kB
    memory_usage = [nelem2kb(m) for m in memory_usage]

    print('memory usage in kB:', [round(m, 1) for m in memory_usage])  # kB

    import matplotlib.pyplot as plt
    plt.bar(range(len(memory_usage)), memory_usage)
    plt.show()

if __name__ == '__main__':
    net = torchvision.models.mobilenet_v2()
    sample_input = torch.randn(1, 3, 224, 224)
    estimate_model(net, sample_input)

Thanks for sharing.
One more question. For the use_res_connect case in function estimate_mbconv,

Hi, the first conv is sharing the same input as the residual connection, so we do not need to count the input activation twice.
For the last conv, it shares the same output as the residual connection.
We are currently cleaning and will release some MCUNet-v2 related code here: https://github.com/mit-han-lab/mcunet.


add tag "default" for options

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind feature
What happened:
User provided options can use desc tag for description, similarly, we should also provide a default tag for them to add flag default.
Anything else we need to know?:
Right now, default value is set in config function, i.e.
// Path returns a configurer to set metrics path.
func Path(path string) nirvana.Configurer {
	if path == "" {
		path = "/metrics"
	}
	return func(c *nirvana.Config) error {
		wapper(c, func(c *config) {
			c.path = path
		})
		return nil
	}
}

// Namespace returns a configure to set metrics namespace.
func Namespace(ns string) nirvana.Configurer {
	if ns == "" {
		ns = "nirvana"
	}
	return func(c *nirvana.Config) error {
		wapper(c, func(c *config) {
			c.namespace = ns
		})
		return nil
	}
}

What we want is
// Option contains basic configurations of metrics.
type Option struct {
	// Namespace is metrics namespace.
	Namespace string `desc:"Metrics namespace" default:"nirvana"`
	// Path is metrics path.
	Path string `desc:"Metrics path" default:"/metrics"`
}

/assign @kdada
/priority p2

This feature is useless.
When a user passes an option to nirvana command, the command can't distinguish if a field should use default value.

/close

As discussed offline, you can distinguish this using pointer value, this is the way Kubernetes deals with default vs no-value.
I'm fine with closing this for now if there is no immediate use case.

It may be not suitable with our scene. Fields in Option are managed by pflag, That means no field can be nil (Of caurse we can write more codes to bypass it). In addition, a pointer field is not easy to use.
e.g.
type Option struct {
    Enable *bool
}

// Usage:
enable := true
&Option{
    Enable: &enable,
}


Localkriging3d

I have added a test case for 3D local kriging that reproduce the results of kt3d program of the GSLIB toolset. The dataset are public (details in the comments of the test).

It is really nice that we can reproduce GSLIB's kt3d results here :+1:
My main concern with making this a test in our test suite as opposed to an example in a Jupyter notebook is that we may want to improve our own implementation in the future (e.g. adding parallel features), in which case the test may start to fail. I would like to separate (1) the reproduction of theoretical results from (2) the reproduction of software results. Ideally the test suite should only contain (1). Even though GSLIB is well-tested and widely used, we shouldn't use it as ground truth. My point is that there are other projects out there that could produce slightly different results, and yet that doesn't mean that our implementation is incorrect. In other words, failing (2) doesn't imply we found a bug. It is just an indication that there is some design space where projects took different approaches that are both valid but produce slightly different answers.
Can I suggest that we move this example to a tutorial in GeoStatsTutorials where you could share the story around the dataset, and how GeoStats.jl can be used to effectively estimate the variables like GSLIB would? We could then implement the missing components as we progress like the rotations you mentioned and other features. This example could also make use of other tools like GslibIO.jl to directly load the datasets in your repo, which would also be beneficial to the community.

You have a point. My suggestion therefore is: Given we are able to reproduce the GSLIB results (at least for this case), we can say that JE results are now valid. Therefore the test case should test the reproduction of these results.
In other words:

I can generate a dataset with the matched results
We test the results from 1)

In an independent work, I will create a notebook trying to reproduce GSLIB results (not just one single case).

Given we are able to reproduce the GSLIB results (at least for this case), we can say that JE results are now valid.

This is not correct if by JE results you mean GeoStats.jl results. As I mentioned, passing this test doesn't imply validity nor correctness of the project. It just means that so far we took similar design decisions to the ones in the implementation of GSLIB, which are not the ground truth. Besides, we don't want to add GSLIB as an indirect test dependency here. It is too cumbersome to update a test like this if it starts to fail.
For the sake of generality and to avoid limiting ourselves to a specific viewpoint based on a specific geostatistics software, let's make this code a commented tutorial. It will be more useful to the community in the long run.


add all Codewars language icons

fixes #7
This commit includes all language icons as used on the Codewars platform. I extracted them as SVG from the used font icomoon.woff where they are present as glyphs. It also contains icons of some languages which are not anymore supported by the platform (GNU Octave, Chapel, Lisp), and languages only supported by Qualified.io (CSS3, HTML5, Sass, ETH).
You don't need to include all the language icons if you find that some existing icons might fit better (e.g. the hexagonal C++ icon or the JS logo instead of the Node.js logo)
As a tip: You can query currently supported languages and their ids from https://www.codewars.com/api/v1/languages/

Thanks for pointing this out. I removed the XML and DOCTYPE declarations, readded the <title> and adjusted the cardbuilder.go such that it all be shown correctly now.
The downside is now that you have to take all new icons because they cannot be positioned anymore by the viewBox (this differs now for many logos) and since I had to transform="scale(1,-1)" all icons (they were stored upside down for some reason in the font) the x and y offset had to be adjusted.


How can I reuse the function that iv made as a command?

i have the following code:
pass_config = click.make_pass_decorator(Config, ensure=True)

@click.group()
init(config):
    pass

@init.group()
@pass_config
def a(config):
    pass

@a.command('print')
@pass_config
def print_a(config):
    print "yes"

@init.group()
@pass_config
def b(config):
    pass

@b.command('print')
@pass_config
def print_a_again(config):
    print_a(config)

i made :init as my entry point and name it as MYCLI in setup,py
then i run MYCLI b print -  i got console error:
Traceback (most recent call last):
  File "C:\python27\Scripts\CHAINCLITEST-script.py", line 9, in <module>
    load_entry_point('CHAINCLITOOL==1.0', 'console_scripts', 'CHAINCLITEST')()
  File "C:\Python27\lib\site-packages\click\core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "C:\Python27\lib\site-packages\click\core.py", line 644, in main
    rv = self.invoke(ctx)
  File "C:\Python27\lib\site-packages\click\core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Python27\lib\site-packages\click\core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Python27\lib\site-packages\click\core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Python27\lib\site-packages\click\core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "C:\Python27\lib\site-packages\click\decorators.py", line 64, in new_func

    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "C:\Python27\lib\site-packages\click\core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "d:\myCLI.py", line 56, i
n print_a_again
    print_a(config)
  File "C:\Python27\lib\site-packages\click\core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "C:\Python27\lib\site-packages\click\core.py", line 631, in main
    args = list(args)
TypeError: 'Config' object is not iterable

if i remove the decorator above print_a, then everything will be fine.
Could anyone help? ... Why this is happening?

I think when you make a function a command, that's it, it can only be used via the command line. Typically these functions will just do some minor CLI-related logic and then dispatch to a function that does the real work.
So I would suggest moving any reusable logic into separate, regular functions so you can call them elsewhere.

Thanks @nchammas

I first used 'baker', then 'mando' for my command line interface helper, and both allow the same function to be used at the command line and via an API.  This was nice to have.
I would like to use 'click', because of other features, but this one missing feature I have come to rely on.
Here is what I thought of:

Use different function names.  Easiest, but not preferred since I have already established the current names as being available from the command line and from within the Python API.

Use a multiple dispatch library.  Another dependency is a concern.  Haven't tried yet, but some deep stuff would have to go on in the back ground which makes me a little nervous.

An option in 'click.command' to map command line arguments to the function arguments.  So something like:
@click.command(from_function_signature=True)
def func(path, overwrite=True):
print(path, overwrite)

Instead of being specified, the 'from_function_signature' would automatically create the parameter entries:
@click.argument('path')
@click.option('--overwrite', type=click.BOOL)
Is there any other work around?
Is something like 3 a viable idea to be added to click?  If so I may take a stab at making it happen.
I also use *args and **kwds in a couple functions which might be difficult to handle.

Option 1 is the only realistic one. Exposing your click function 1:1 is a very bad idea because those might rely on e.g. click closing files, click's special objects being passed etc. You can use the name parameter on click.command to choose something other than the function name as your command name, and can suffix the functions with _cmd.

In 1. fashion, could may be click store and expose original decorated function, so for above example I could use something like func._original if I want to use it directly via it's original python interface?
Then it would be cool if it's __doc__ was populated with documentation of the parameters in one of the styles (configurable eg numpy etc) so that function was a full fledged function without developers needing to duplicate the signature and docs if to maintain it separately.

Click does store and expose the original function: command.callback
See #1054 for explanation

Thank you @davidism !!

Click does store and expose the original function: command.callback
See #1054 for explanation

It does not expose just the original function in command.callback. For example if you have used the decorator @pass_obj then it modifies the original function and stores that under callback and it would essentially be impossible to call the function outside of the CLI.

pass_context is not the same thing. It's a regular decorator, presumably the function requires a context if it's decorated with it.


Build Five Programs - Step 8

Describe the Issue
I could execute the file without getting the permission denied error as the task said it would. I am not sure if I did have permission beforehand or maybe it is disabled on the challenge's end. I think it is worth to have a look into it just in case.

 You can run it by executing the file and it will default to bash. Execute your script with ./questionnaire.sh. You will get a permission denied error."
Screenshots
No response
System

Device: Desktop PC
OS: Win 10 x64 - 10.0.19045
Browser: Firefox
Version: Ver. 130.0.1

Additional context
No response

I didn't encounter such issue. Are you doing this tutorial on Gitpod, or locally?

I am doing it on gitpod VM using vs code browser. I can continue the
tutorial, however. It was strange to have a different output than it
should.
El mié., 25 de sep. de 2024 23:45, Krzysztof G. @.***>
escribió:

I didn't encounter such issue. Are you doing this tutorial on Gitpod, or
locally?
—
Reply to this email directly, view it on GitHub
https://github.com/freeCodeCamp/freeCodeCamp/issues/56297#issuecomment-2375988623,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APNLLGAKE5DQVNCQSTLWA53ZYONPBAVCNFSM6AAAAABO3X5JKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZVHE4DQNRSGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

Following the steps in a brand new workspace, I also cannot reproduce this. Going to close as CNR.


CLOUDP-132962: Datadog integration is disabled in cloudgov (address comments)

Proposed changes
Datadog integration is disabled in cloudgov (followup from https://github.com/mongodb/mongodb-atlas-cli/pull/1417)

Jira ticket: CLOUDP-132962

Closes #[issue number]
Checklist

[ ] I have signed the MongoDB CLA
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added any necessary documentation in document requirements section listed in CONTRIBUTING.md (if appropriate)
[ ] I have addressed the @mongodb/docs-cloud-team comments (if appropriate)
[ ] I have updated test/README.md (if an e2e test has been added)
[ ] I have run make fmt and formatted my code

Further comments

@gssbzn


Chaise config naming inconsistencies

It was brought up that almost all of the chaise config properties use camel case starting with a lower case letter (camelCase). There are 2 properties that do not, SystemColumnsDisplayCompact and SystemColumnsDisplayDetailed.

We certainly won't have different configuration parameters differing
only by case. Should we make chaise case-insensitive in its
interpretation of the config...?

case-insensitive makes sense.. We will proceed with camel case for documentation though.

From: Karl Czajkowski<EMAIL_ADDRESS>Sent: Friday, July 10, 2020 4:55 PM
To: informatics-isi-edu/chaise<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: Re: [informatics-isi-edu/chaise] Chaise config naming inconsistencies (#1979)
We certainly won't have different configuration parameters differing
only by case. Should we make chaise case-insensitive in its
interpretation of the config...?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/informatics-isi-edu/chaise/issues/1979#issuecomment-656942718, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AADFIDB5ZEJJLV73RGERS7TR26S75ANCNFSM4OXAFXGQ.

#1943 is not directly related to this but can be done together. Both are related to chaise-config.

Both, SystemColumnsDisplayCompact and SystemColumnsDisplayDetailed, have been updated to use proper camel case in the code and documentation.
Chaise config properties that are a part of chaise-config.js and the catalog chaise config annotation including configRules are case-insensitive.


Fix logging

If STP_BRIDGE.logLineStarting is not set to true when logging is enabled, WriteChar doesn't initialize properly, and the assert at stp_log.cpp line 85 trips as a result.

Thanks, Szabolcs.


strings containing decimal places of ".0" are now correctly converted…

… to integers

Codecov Report
Base: 81.88% // Head: 81.93% // Increases project coverage by +0.05% :tada:

Coverage data is based on head (0f20b91) compared to base (c0122fc).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #134      +/-   ##
==========================================
+ Coverage   81.88%   81.93%   +0.05%
==========================================
  Files          54       54
  Lines        2478     2486       +8
==========================================
+ Hits         2029     2037       +8
  Misses        449      449

Impacted Files
Coverage Δ

...ls/datasets/annex_a/lds_annexa_clean/converters.py
100.00% <100.00%> (ø)

...ools/datasets/s903/lds_ssda903_clean/converters.py
100.00% <100.00%> (ø)

tests/annex_a/test_converters.py
100.00% <100.00%> (ø)

tests/s903/test_converters.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Add PrestaShop test

Coverage decreased (-0.5%) to 99.485% when pulling 833f706b4f4a231b27f050abb8a21a40cc96a2df on reedy:reedy-patch-2 into 44362c7c8aace5c52bd20d917b87e6c1d2d8c5c6 on cssjanus:master.

We have a narrower test case.. :)


Depth Maps cannot Supervise Training

Hi, thanks for your released code. I'm interested in the your methods that training with depth map. I tried to train an auto-encoder network, where the encoder and decoder are both pointnet. The input and output are point cloud. If I train the network with chamfer distance, it works well. But if I replace the loss function with L1 distance between depth maps, it cannot work. Do you have any suggestions?

Hi, please check my answer in this issue here.

Hi! I assume the issue was resolved, so I am closing the issue. Feel free to reopen if you still have questions.
Thanks.


Missing screenshots of IDEs in the root readme doc should be fixed

I believe the intent initially by @vhvb1989 in https://github.com/Azure/azure-sdk-for-cpp/pull/3614 was to add images for the IDEs to showcase as examples, but they are missing/were never added. We should add them or restructure the doc to remove them.
We see placeholders today, like:

-IMAGE HERE Visual Studio-

https://github.com/Azure/azure-sdk-for-cpp#cmake-project--vcpkg---manifest-mode
cc @ronniegeraghty

No longer relevant to the current document

Agreed, this issue no longer applies after the doc changes made by @ronniegeraghty in https://github.com/Azure/azure-sdk-for-cpp/pull/5339


gzip compressor should be registered automatically

Currently this needs to be done manually...
Title
gzip compressor should be registered automatically
Motivation
.gz files work out of the box, gzip files need to manually registed...
Checklist
Before you create the PR, please make sure you have:

[x ] Picked a concise, informative and complete title
[ x] Clearly explained the motivation behind the PR
[ ] Linked to any existing issues that your PR will be solving
[ ] Included tests for any new functionality
[ x] Checked that all unit tests pass

Workflow
Please avoid rebasing and force-pushing to the branch of the PR once a review is in progress.
Rebasing can make your commits look a bit cleaner, but it also makes life more difficult from the reviewer, because they are no longer able to distinguish between code that has already been reviewed, and unreviewed code.

How common is .gzip extension? Wikipedia page mentions .gz only. I've never seen .gzip files in the wild.

Oh, good question. I've seen it multiple times. Not sure if it is being used often. But I don't think I'm the only one: https://fileinfo.com/extension/gzip

Np 😉


Syntax highlighting component

You can see on all available dev blog sites sections where code samples get provided.
Goal of this component is a easy way to make code snippets look good with minimal usage effort.
It should receive at least one prop which should be the language prop.
The code which should get highlighted should be received via a slot.

I would like to try to work on it

@marichka-offen
Go for it 💪🏻

Hi @r4pt0s, I find myself unable to complete this component. Sorry for the late notice.

No worries @juancwu


Token is expired

Hey
I guess we have an issue:
~/Desktop ❯ bonbast
Error: token is expired. Try again later.

update the project

Update to the latest version.
pip3 install -U bonbast

@hzonuz The issue shouldn't be related to homebrew. I'm currently using it with homebrew myself. Does it work with Python for you?
Also, your IP might be blocked.

No, it seems doesn't work. I don't think that my ip is blocked. Maybe it caches sth at the first time.


[Hub Generated] Review request for Creator to add version stable/2023-07-01

This is a PR generated at OpenAPI Hub. You can view your work branch via this link.
Data Plane API - Pull Request

API Info: The Basics
Most of the information about your service should be captured in the issue that serves as your API Spec engagement record.

Link to API Spec engagement record issue:

Is this review for (select one):

[ ] a private preview
[ ] a public preview
[ ] GA release

Change Scope
This section will help us focus on the specific parts of your API that are new or have been modified. Please share a link to the design document for the new APIs, a link to the previous API Spec document (if applicable), and the root paths that have been updated.

Design Document:
Previous API Spec Doc:
Updated paths:

Viewing API changes
For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.
Suppressing failures
If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
Swagger-Suppression-Process
to get approval.
❔Got questions? Need additional info?? We are here to help!

 Contact us!
The Azure API Review Board is dedicated to helping you create amazing APIs. You can read about our mission and learn more about our process on our wiki.

💬 Teams Channel
💌 email

Click here for links to tools, specs, guidelines & other good stuff
Tooling

Open API validation tools were run on this PR. Go here to see how to fix errors
Spectral Linting
Open API Hub

Guidelines & Specifications

Azure REST API Guidelines
OpenAPI Style Guidelines
Azure Breaking Change Policy

Helpful Links

Azure DevTools Wiki

Checks stuck in `queued` state?
If the PR CI checks appear to be stuck in `queued` state, please add a comment with contents `/azp run`.
This should result in a new comment denoting a `PR validation pipeline` has started and the checks should be updated after few minutes.

@johanste @stewartadam This is not a breaking change, I only want to change the description, but somehow the module validation keeps failing complaining no "schema" defined for the response. However, if I remove the ref and copy the same thing over, it start complaining about the break change.

The reason for the breaking change notification is because client codegen can be impacted by the rename of the type (which is what inlining would mean). I'm pretty sure that our tooling allows you to put the description next to the $ref rather than inlining it - and if so, that is what I would recommend to avoid the breaking change.

The reason for the breaking change notification is because client codegen can be impacted by the rename of the type (which is what inlining would mean). I'm pretty sure that our tooling allows you to put the description next to the $ref rather than inlining it - and if so, that is what I would recommend to avoid the breaking change.

Hi Johan, if I don't do it inline and left "ref" there, the ModelValidation will failed due to "missing schema in the response". What is your suggestion in this case?

Are you trying to affect the description (documentation) of the responses, or are you just trying to change the description of the /conversions action? If the latter, you should not have to make a change to the responses. If the former, you should also not copy the response over but put the updated description field next to the $ref.
Can you provide a link to the error you are getting when not copying the responses over?

Are you trying to affect the description (documentation) of the responses, or are you just trying to change the description of the /conversions action? If the latter, you should not have to make a change to the responses. If the former, you should also not copy the response over but put the updated description field next to the $ref.
Can you provide a link to the error you are getting when not copying the responses over?

Yes, if I keep the change only on the description, this is the ModelValidationError that appeared: https://github.com/Azure/azure-rest-api-specs/runs/18913067538

@konrad-jamrozik, I see no reason why this would introduce a model violation error. Can we override the check?

@konrad-jamrozik, I see no reason why this would introduce a model violation error. Can we override the check?

@johanste this check is owned by @scbedd. Full list of owners on Teams, here.

@konrad-jamrozik, I see no reason why this would introduce a model violation error. Can we override the check?

@johanste this check is owned by @scbedd. Full list of owners on Teams, here.

Ack notification. I'll check this today.

This is definitely a bug with oav. Here is the issue. My intent is to get to it this week. I will sign off for the model validation failure for now while I code a fix.

This is a failure that was discovered last week
I will sign off for the ModelValidation failure. They are false positives.

Hi, @johanste @scbedd , thanks for helping on the approval and confirming the issue. Could you help me merge this PR? It shows on my side the merging is still blocked and I am not authorized to merge this PR.

/azp run

I believe that the APIView error may be resolved by an emitter upgrade. Merge wise, I need to find the right ARM contact.

@scbedd since you added the Approved-ModelValidation tag, why does it continue to block on this?

@scbedd since you added the Approved-ModelValidation tag, why does it continue to block on this?

The model validation is erroneous, it shouldn't be erroring. However, the check will still fail even if I add the approved-ModelValidation tag. There's no connection of the tags to model or semantic validation.
The two files being touched also don't have readmes associated with them, so there's no easy way to just add a suppression and ignore them.

The model validation is erroneous, it shouldn't be erroring. However, the check will still fail even if I add the approved-ModelValidation tag. There's no connection of the tags to model or semantic validation.

That's weird because if the CI is blocked on BreakingChangeReviewRequired, then adding the BreakingChangeApproved tag actually satisfies the CI and causes it to pass. It would see that ModelValidation would do the same thing. So this needs a force merge?
@lmazuel can you do the honors?

@scbedd @tjprescott I'll have a look into the suppression-via-label not working:

https://github.com/Azure/azure-sdk-tools/issues/7455

/azp run

@scbedd @tjprescott running /azp run fixed the suppression via the label. Looks like it was just a flake.


Sometimes cursor drawn does not match the cursorStyle that is set

After the 1.8.0 release, the VSCodeVim extension is not always rendering the correct cursor.
I am using: console.log(vscode.window.activeTextEditor.options.cursorStyle); to verify cursor style
In some cases, it prints vscode.TextEditorCursorStyle.Block (which is what I want), but what is rendered looks to be vscode.TextEditorCursorStyle.Line.
So there is a mismatch between vscode.window.activeTextEditor.options.cursorStyle, I set it to exactly what I want, I query the value of vscode.window.activeTextEditor.options.cursorStyle and it is what I set it to, however what is drawn is NOT what I set it to.
It is easiest to reproduce in settings.json currently, however we are seeing it sporadically in other files as well..

Related #17513

Was this fixed? It seems to be different/fixed on insiders

@xconverge Even though right now it looks reasonable in Insider, I'll say let's keep an eye on it. Basically right now we are saving a lot of stuff to settings file and whenever the settings file change, the options cache on active text editor will get flushed. The racing of our Vim's internal cursor update and Code's internal cursor style update might be the root cause.

Possibly 67924dec07b503fafbe6e7128333ae15202e3ef9 improved things a lot

Duplicate of #17513


Support for Azure.configure().authenticate(File credentialsFile)

equivalent to Azure.authenticate(File credentialsFile)

Can one of the admins verify this patch?


AC2: Account Management -- Security Question Screen

Trigger
If the user decides to "View all accounts", the Security Question is asked.
Logic

If answered correctly, all accounts are shown.
If answered incorrectly default, duress and archived accounts are shown.

Archived accounts should show in a different format as they will generally be with zero value and nothing can be done for them.

@antuz123 are we doing duress mode before 2.0? This issue should go together wit duress mode.


Send message out of terminal without open messer first

Hi there.
I would like to create a script that automatically sends a message without me having to run messer first. Is it already possible to do this in the current version of messer? I was thinking of something like: messer message "reciever" Hello World! or similar. This command should then open messer, send the message and then close messer again.
Greetings Apop

Thanks for the suggestion @Apop85 , I would call it a duplicate of #58  though, I hope you agree


Mappers and settings

Hi,
Have you experimented with different mappers and settings to generate the bam file? How important is accurate mapping for CliqueSNV? Do you have any recommendations for illumina reads?
Thanks, Jon

Hi Jon,
This is a good question because accurate mapping is important since it can generate some systematic errors. For Illumina reads, for samples from viruses, we reduced mismatch penalty because mutations in viruses are more expected than in other organisms. As of other parameters, we used default values. We also tried to use tablet for raw quality estimation of alignment by controlling visually dependence of coverage on parameters.

Thanks. We're testing a little bit the difference between Bowtie2 and Tanoti. Tanoti is very sensitive and designed for fast evolving viruses. It looks like we get fewer haplotypes predicted with Tanoti compared to Bowtie2, but these are very prelimiary results.

Thank you for sharing that! If you believe that Tanoti is more accurate than Bowtie2, then it makes sense why it produces less haplotypes on Tanoti alignment. The reason could be the systematic errors as I mentioned. However, it is only an intuition, to get better answer, it may require a more thorough analysis. Thank you!


⚠ Bofrost.be has degraded performance

In 779e3cd, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1781 ms

Resolved: Bofrost.be performance has improved in d77de05.


Allocate CSR address for minthresh

Assigned minthresh CSR to 0xm47, where m is the nibble encoding the privilege mode (M=0x3, S=0x1, U=0x0).


Error message, no content in GUI, and crashes on Windows 10

Describe the bug
I get the error message: "Error Detected: Please Restart Miles" in the top left corner of the GUI (see screenshot) every time I launch it. There is no other content appearing in the GUI (according to the screenshots there should be a chat conversation in the GUI). The PowerShell (admin) window on the other hand does show the python output, including the chat conversation.
I believe it is prudent to mention that something may have gone awry during the installation because following the Run sequence as layed out under the installation instructions doesn't work since the 'Scripts' folder ended up in the 'Miles-env' folder, and NOT in the 'Miles-V2' folder. I have been starting Miles by navigating to the folder where the Scripts are and activating the virtual environment there and then navigating to the path specified in the instructions and starting it from there.
P.S. I have no idea if that is relevant or not to the issues I'm experiencing as I'm not proficient in this stuff.
_Open PowerShell as Admin
Navigate to Project Directory:
cd C:\Miles-env\M.I.L.E.S\Miles-V2
Activate Virtual Environment:
.\Scripts\Activate
Start Project with npm:
npm start_
I have also encountered the following python error (this was the second "crash" I encountered after having Miles installed for only an hour or so):
Python Output: Listening for prompt... Speak now.
Python Output: User:
Python Error: Traceback (most recent call last):
File "C:\Miles-env\M.I.L.E.S\Miles-V2\main.py", line 1198, in
Python Error:     main()
File "C:\Miles-env\M.I.L.E.S\Miles-V2\main.py", line 1154, in main
Python Error:     _, skip_wake_word = reply(query)  # reply now returns a tuple
Python Error: ^^^^^^^^^^^^
File "C:\Miles-env\M.I.L.E.S\Miles-V2\main.py", line 889, in reply
Python Error:     response_content = ask(question)
Python Error:         ^^^^^^^^^^^^^
File "C:\Miles-env\M.I.L.E.S\Miles-V2\main.py", line 783, in ask
Python Error:     print("User:", question)
File "C:\Users\GTSA - Infinity\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
Python Error:     return codecs.charmap_encode(input,self.errors,encoding_table)[0]
^^^
Python Error: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u0131' in position 15: character maps to
Python script exited with code: 1
Screenshots

Desktop (please complete the following information):

OS: Windows 10

Is there any way I can fix this? I really love the idea behind Miles and really want it to work.

Seems that Miles or the speech recognition accidentally provided a character that cannot be processed in python.
To fix this, delete the file named conversation_history.txt.
Then go ahead and try to start Miles. Let me know if this fixes the issue.

It did not solve the problem. The GUI is still behaving the same way and Miles again crashed or stopped working after I asked it to play something on Spotify/or change the volume.
This is what it looks like when I start Miles.

This is what it looked like before it crashed.

The spotify error might be related to Spotify's own servers for their API being down, Miles does not have the capacity to request changes that it is not allowed to use, they are simply not available.
That spotify error happens when you try to request a change from a part in Spotify's API that you are not permitted to use or have not asked for permission to use. Given that the app does not request any changes outside of what I ask permission for it is unlikely that this is an issue on my part, at least for spotify issues.
The GUI not responding because of that warning that appears shouldn't happen, but there is no way for me to prevent it, so you can edit the following lines to fix the issue inside renderer.js, keep in mind that any errors will no longer appear on the GUI, and instead will only be in the terminal window:
BEFORE:
function processMessage(message, isError) {
    if (isError) {
        setStatus(message, 'status-error', 'fas fa-exclamation-triangle');
        return;
    }
    // Check if an error has been detected previously
    if (isErrorDetected) {
        console.log("Blocking message due to prior error.");
        return; // Exit early
    }
    let messageClass = '';

    // Detect and wrap LaTeX with spans for MathJax processing
    message = message.replace(/\\\(.*?\\\)/g, function(match) {
        return `<span class="mathjax-latex">${match}</span>`;
    });

AFTER:
function processMessage(message, isError) {
    if (isError) {
        setStatus(message, 'status-error', 'fas fa-exclamation-triangle');
        return;
    }
    // Comment out or remove these lines
    // if (isErrorDetected) {
    //     console.log("Blocking message due to prior error.");
    //     return; // Exit early
    // }
    let messageClass = '';

    // Detect and wrap LaTeX with spans for MathJax processing
    message = message.replace(/\\\(.*?\\\)/g, function(match) {
        return `<span class="mathjax-latex">${match}</span>`;
    });


When running az I get No such file or directory

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Related command

Describe the bug
Installing azure cli on Mac M1 and after installation is complete, and I run the az command, it returns
/opt/homebrew/bin/az: line 2: /opt/homebrew/Cellar/azure-cli/2.36.0/libexec/bin/python: No such file or directory

To Reproduce
Followed all the steps under installing azure-cli for MacOS. The only difference being I used ibrew instead of brew since I have to use the rosetta terminal for all my applications.
Expected behavior

Environment summary

Additional context

I install ibrew but I can't reproduce this. It works as expected.
Why the installed version is 2.36.0 rather than 2.38.0 on your computer?

That is an interesting question:
When I run az, it shows up in /opt/homebrew/Cellar/azure-cli/2.36.0/
But when I try to uninstall azure-cli, it does it from this loc: /usr/local/Cellar/azure-cli/2.38.0

I resolved it by uninstalling azure-cli from arch:amd 64 and only having it under i386