[SPARK-16609] Add to_date/to_timestamp with format functions

What changes were proposed in this pull request?
This pull request adds two new user facing functions:

to_date which accepts an expression and a format and returns a date.
to_timestamp which accepts an expression and a format and returns a timestamp.

For example, Given a date in format: 2016-21-05. (YYYY-dd-MM)
Date Function
Previously
to_date(unix_timestamp(lit("2016-21-05"), "yyyy-dd-MM").cast("timestamp"))

Current
to_date(lit("2016-21-05"), "yyyy-dd-MM")

Timestamp Function
Previously
unix_timestamp(lit("2016-21-05"), "yyyy-dd-MM").cast("timestamp")

Current
to_timestamp(lit("2016-21-05"), "yyyy-dd-MM")

Tasks

[X] Add to_date to Scala Functions
[x] Add to_date to Python Functions
[x] Add to_date to SQL Functions
[X] Add to_timestamp to Scala Functions
[x] Add to_timestamp to Python Functions
[x] Add to_timestamp to SQL Functions
[x] Add function to R

How was this patch tested?

[x] Add Functions to DateFunctionsSuite
Test new ParseToTimestamp Expression (not necessary)
Test new ParseToDate Expression (not necessary)
[ ] Add test for R
[x] Add test for Python in test.py

Please review http://spark.apache.org/contributing.html before opening a pull request.

Test build #69645 has started for PR 16138 at commit b0f1ced.

Test build #69645 has finished for PR 16138 at commit b0f1ced.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69645/
Test FAILed.

Can you use "SPARK-16609" instead of "Spark-16609"?

Test build #69647 has started for PR 16138 at commit 9a10a34.

Test build #69647 has finished for PR 16138 at commit 9a10a34.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69647/
Test FAILed.

Test build #69648 has started for PR 16138 at commit ecde7b7.

Test build #69649 has started for PR 16138 at commit 89dccbd.

Test build #69648 has finished for PR 16138 at commit ecde7b7.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69648/
Test FAILed.

Test build #69650 has started for PR 16138 at commit c17c0bd.

Test build #69651 has started for PR 16138 at commit 5bf5002.

Test build #69649 has finished for PR 16138 at commit 89dccbd.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class ToDate(child: Expression) extends UnaryExpression with ImplicitCastInputTypes

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69649/
Test PASSed.

Merged build finished. Test PASSed.

Test build #69650 has finished for PR 16138 at commit c17c0bd.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69650/
Test FAILed.

Merged build finished. Test FAILed.

Test build #69651 has finished for PR 16138 at commit 5bf5002.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69651/
Test FAILed.

This will be very convenient!  Looking forward to the whole patch.  For SQL I think you should look at RuntimeReplaceable.  Also, we might consider having a default version that doen't take a pattern (and parses something standard such as 2016-12-01T02:03:01Z)

Test build #69713 has started for PR 16138 at commit 8837bdb.

Test build #69713 has finished for PR 16138 at commit 8837bdb.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class ToTimestamp(left: Expression, right: Expression, child: Expression)

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69713/
Test FAILed.

Test build #69737 has started for PR 16138 at commit 8e8339c.

Test build #69738 has started for PR 16138 at commit f1e075c.

Test build #69739 has started for PR 16138 at commit 760650c.

Test build #69737 has finished for PR 16138 at commit 8e8339c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69737/
Test FAILed.

Test build #69738 has finished for PR 16138 at commit f1e075c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class ParseToDate(left: Expression, right: Expression, child: Expression)
case class ParseToTimestamp(left: Expression, right: Expression, child: Expression)

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69738/
Test FAILed.

Test build #69739 has finished for PR 16138 at commit 760650c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69739/
Test FAILed.

and I found the error, I shouldn't be overriding the DataType.

Test build #69747 has started for PR 16138 at commit 399a9c4.

Now that my outputs are correct (in format), there's a new problem. The types are still wrong.
scala> /////////// DETAILS

scala> // Schema

scala> spark.range(1).select(to_timestamp(lit("2016-12-31"), "yyyy-MM-dd")).schema
// res4: org.apache.spark.sql.types.StructType = StructType(StructField(to_timestamp('2016-12-31', 'yyyy-MM-dd'),StringType,true))

scala> spark.range(1).select(to_date(lit("2016-12-31"), "yyyy-MM-dd")).schema
// res5: org.apache.spark.sql.types.StructType = StructType(StructField(to_date('2016-12-31', 'yyyy-MM-dd'),StringType,true))

Those should be DateType and TimestampType respectively.
However if we inspect the expression directly, it seems that we do have the correct type.
scala> import org.apache.spark.sql.catalyst.expressions.ParseToDate
scala> val x = new ParseToDate(lit("hello").expr, lit("world").expr)
x: org.apache.spark.sql.catalyst.expressions.ParseToDate = to_date(hello, world)

scala> x.dataType
res9: org.apache.spark.sql.types.DataType = DateType

It seems to be getting lost somewhere.

Test build #69747 has finished for PR 16138 at commit 399a9c4.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69747/
Test FAILed.

Merged build finished. Test FAILed.

More details are here: https://gist.github.com/anabranch/7a42292593976878eb14e2d86a9966d4
This is completely perplexing to me.

Test build #69807 has started for PR 16138 at commit 602025f.

Test build #69807 has finished for PR 16138 at commit 602025f.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69807/
Test FAILed.

Test build #69822 has started for PR 16138 at commit 8eb3391.

Test build #69826 has started for PR 16138 at commit f76f91a.

Test build #69822 has finished for PR 16138 at commit 8eb3391.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69822/
Test FAILed.

Test build #69826 has finished for PR 16138 at commit f76f91a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class ParseToDate(left: Expression, format: Expression, child: Expression)
case class ParseToTimestamp(left: Expression, format: Expression, child: Expression)

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69826/
Test FAILed.

Test build #70997 has started for PR 16138 at commit 1eb2ad0.

Test build #70997 has finished for PR 16138 at commit 1eb2ad0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70997/
Test PASSed.

Test build #71029 has started for PR 16138 at commit 8c00745.

Test build #71030 has started for PR 16138 at commit a2539a6.

Test build #71031 has started for PR 16138 at commit d11063b.

I believe now why my previous implementation did not work.
My implementation originally looked like this:
case class ParseToTimestamp(left: Expression, format: Expression, child: Expression)
  extends RuntimeReplaceable {

  def this(left: Expression, format: Expression) = {
  this(left, format, Cast(UnixTimestamp(left, format), TimestampType))
}

  override def checkInputDataTypes(): TypeCheckResult = {
    if (left.dataType != StringType) {
      TypeCheckResult.TypeCheckFailure(s"TO_TIMESTAMP requires both inputs to be strings")
    }
    TypeCheckResult.TypeCheckSuccess
  }

  override def flatArguments: Iterator[Any] = Iterator(left, format)
  override def sql: String = s"$prettyName(${left.sql}, ${format.sql})"

  override def prettyName: String = "to_timestamp"
  override def dataType: DataType = TimestampType
}

This implementation with a simple example would fail.
import org.apache.spark.sql.functions._

val ss1 = "2015-07-24 10:00:00"
val ss2 = "2015-07-25 02:02:02"
val df2 = Seq((ss1), (ss2)).toDF("ss")

df2.select(to_timestamp(col("ss"))).show

This throws a
org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to dataType on unresolved object, tree: 'ss

A Trace log level shows that the columns are resolved however the error originates after analysis and during CheckInputDataTypes. This function seeks to analyze the left input, however because this column is input into a RuntimeReplaceable function the relevant, and resolved, tree is actually the child argument - left remains unresolved (and therefore throws the above error).
I believe this to be the root cause and that has in turn showed me that I do not need to perform input validation for this function in the first place. Since I only wrap functions, they are performing the exact same input validation that I would be. Since no new logic is implemented, there's no point in redundantly validating something that will be validated again anyways, especially when the system won't let me.

Test build #71032 has started for PR 16138 at commit 7a13290.

Test build #71032 has finished for PR 16138 at commit 7a13290.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71032/
Test FAILed.

Test build #71029 has finished for PR 16138 at commit 8c00745.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71029/
Test FAILed.

Test build #71030 has finished for PR 16138 at commit a2539a6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71030/
Test FAILed.

Merged build finished. Test FAILed.

Test build #71031 has finished for PR 16138 at commit d11063b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71031/
Test FAILed.

Test build #71810 has started for PR 16138 at commit 806e19b.

Test build #71810 has finished for PR 16138 at commit 806e19b.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71810/
Test FAILed.

@felixcheung could you give me some pointers on these R functions? I don't quite know if I am registering them correctly and they're failing my builds.

Test build #71811 has started for PR 16138 at commit acee26a.

Test build #71811 has finished for PR 16138 at commit acee26a.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71811/
Test FAILed.

Merged build finished. Test FAILed.

Test build #71812 has started for PR 16138 at commit ea14009.

Test build #71812 has finished for PR 16138 at commit ea14009.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71812/
Test FAILed.

Merged build finished. Test FAILed.

Test build #71813 has started for PR 16138 at commit bd4d3aa.

Test build #71813 has finished for PR 16138 at commit bd4d3aa.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71813/
Test FAILed.

Could you make sure your code doc is starting with #' instead of #? It's not obvious what the error is but it might be related to parsing / alignment.

@felixcheung Just tried that it doesn't seem to work.
Here's the strange thing, it should follow the exact same structure as unix_timestamp except that I don't have a signature that takes 0 parameters (i.e., unix_timestamp L3031.
Now if I copy and paste the unix_timestamp section and change the names, it will compile. However as soon as I remove the first function (like L3031, that takes no parameters). It will fail again with the same error.
I also tried removing everything and just creating one function at a time.
just this code will work.
#' to_date
#'
#' Converts the column into DateType, optionally with a format. Returns null if input is invalid.
#' Specify formats according to: http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html
#'
#' @param x Column to parse.
#' @param format string to use to parse x Column to DateType.
#'
#' @rdname to_date
#' @name to_date
#' @family datetime_funcs
#' @aliases to_date,Column-method
#' @export
#' @examples
#' \dontrun{
#' to_date(df$c, 'yyyy-MM-dd')
#' }
#' @note to_date(Column, format) since 2.2.0
setMethod("to_date",
          signature(x = "Column"),
          function(x) {
            jc <- callJStatic("org.apache.spark.sql.functions", "to_date", x@jc)
            column(jc)
          })

However if I add any other parameters to that method (or signature). It will fail.

you would need to update the signature here https://github.com/apache/spark/blob/master/R/pkg/R/generics.R#L1255
this affects the signature matching

@felixcheung yup! Thanks!

Test build #71873 has started for PR 16138 at commit 53a84c1.

Test build #71873 has finished for PR 16138 at commit 53a84c1.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71873/
Test FAILed.

Merged build finished. Test FAILed.

Test build #71875 has started for PR 16138 at commit 682f03e.

Test build #71875 has finished for PR 16138 at commit 682f03e.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71875/
Test FAILed.

Test build #71878 has started for PR 16138 at commit 3b95da1.

Test build #71878 has finished for PR 16138 at commit 3b95da1.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71878/
Test FAILed.

Test build #71879 has started for PR 16138 at commit 2d36993.

Test build #71879 has finished for PR 16138 at commit 2d36993.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71879/
Test FAILed.

Merged build finished. Test FAILed.

@felixcheung Thank you for your feedback! Small request, can you tell me if my R test case is sufficient for this? It doesn't seem like there is extensive R testing right now for virtually any function.
Obviously tests will pass soon, running into strange edge cases.

@cloud-fan - Reynold referred me to your for this test failure.
My two tests are failing because Hive tests allegedly cover something like this.
SELECT to_date('2001-10-30 10:30:00', '')

However, Hive doesn't support passing in multiple parameters to to_date as specified in the language manual. The only instance I see for to_date with multiple parameters is when it talks to ORACLE DB as the metastore although I don't know the code base (I grepped for to_date and tried to find instances of this occurring).
It seems like this test case should not be running in the first place. Can you advise on any suggestions you might have?

Test build #71913 has started for PR 16138 at commit 8fa4bfb.

Test build #71913 has finished for PR 16138 at commit 8fa4bfb.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71913/
Test FAILed.

@anabranch right, since there is little work done on the R side and this is mostly a pass through, just checking if the functions are callable without error like the test you have added should be fine for R.
The gate for functional correctness would be more with the Scala tests in this case.

The failed test is udf5.q in HiveComparisonTest, but I can't find SELECT to_date('2001-10-30 10:30:00', '') in it, can you point out where we write such a test?

@cloud-fan The error i see is the one in this test case
What it states at the top
# Converted SQL query string:
SELECT `gen_attr_0` AS `to_date('2001-10-30 10:30:00', '')` FROM (SELECT to_date('2001-10-30 10:30:00', '') AS `gen_attr_0`) AS gen_subquery_0

# Original HiveQL query string:
SELECT to_date('2001-10-30 10:30:00')

# Resolved query plan:
Project [to_date(2001-10-30 10:30:00, ) AS to_date('2001-10-30 10:30:00', '')#13365]

I don't really understand the hive failure either.

Test build #72301 has started for PR 16138 at commit a2d0221.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72301/
Test FAILed.

retest this please

Test build #72311 has started for PR 16138 at commit a2d0221.

Test build #72311 has finished for PR 16138 at commit a2d0221.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class ParseToDate(left: Expression, format: Option[Expression], child: Expression)

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72311/
Test FAILed.

Test build #72394 has started for PR 16138 at commit 479243f.

Test build #72394 has finished for PR 16138 at commit 479243f.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72394/
Test FAILed.

Test build #72396 has started for PR 16138 at commit e00b446.

Test build #72396 has finished for PR 16138 at commit e00b446.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72396/
Test FAILed.

Test build #72397 has started for PR 16138 at commit da16c66.

Test build #72397 has finished for PR 16138 at commit da16c66.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72397/
Test FAILed.

Merged build finished. Test FAILed.

this is the CRAN check error:
* checking for missing documentation entries ... WARNING
Undocumented S4 methods:
  generic 'to_date' and siglist 'Column,missing'
  generic 'to_timestamp' and siglist 'Column,missing'
All user-level objects in a package (including S4 classes and methods)
should have documentation entries.
See the chapter 'Writing R documentation files' in the 'Writing R
Extensions' manual.
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... WARNING
Objects in \usage without \alias in documentation object 'to_date':
  '\S4method{to_date}{Column,missing}'

Objects in \usage without \alias in documentation object 'to_timestamp':
  '\S4method{to_timestamp}{Column,missing}'

Functions with \usage entries need to have the appropriate \alias
entries, and all their arguments documented.
The \usage entries must correspond to syntactically valid R code.
See the chapter 'Writing R documentation files' in the 'Writing R

you probably change it to @alias to_date,Column,missing-method

@felixcheung thanks, made those changes :). Hopefully this will start passing sometime :P

Test build #72422 has started for PR 16138 at commit c3063a2.

Test build #72422 has finished for PR 16138 at commit c3063a2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72422/
Test PASSed.

Merged build finished. Test PASSed.

@cloud-fan great questions.
I thought that was strange too. However this is the current behavior as well as Java SimpleDateFormat's behavior. I did not implement that logic.
For example, today:
spark.sql("""
   SELECT
   cast(
	   cast(
	   	unix_timestamp('2016-30-12', 'yyyy-MM-dd')
	   as timestamp)
   as date)
   """).take(5)
// res4: Array[org.apache.spark.sql.Row] = Array([2018-06-12])

Which seems wrong to me, but that's the logic that exists. The issue seems to be that unix_timestamp returns an incorrect value when it should just return null. The source is the behavior of SimpleDateFormat.
scala> import java.text.SimpleDateFormat
// import java.text.SimpleDateFormat

scala> val x = new SimpleDateFormat("yyyy-MM-dd")
// x: java.text.SimpleDateFormat = java.text.SimpleDateFormat@f67a0200

scala> x.parse("2016-30-12")
// res5: java.util.Date = Tue Jun 12 00:00:00 PDT 2018

Manual Correctness tests:
Python
>>> from pyspark.sql.functions import to_date, to_timestamp, lit
>>> spark.range(1).select(to_date(lit('2016-01-02'),'yyyy-MM-dd').alias("dt")).collect()
# [Row(dt=datetime.date(2016, 1, 2))]
>>> spark.range(1).select(to_timestamp(lit('2016-01-02'),'yyyy-MM-dd').alias("dt")).collect()
# [Row(dt=datetime.datetime(2016, 1, 2, 0, 0))]

R
> collect(select(people, to_date(lit('2016-01-02'), 'yyyy-MM-dd')))
  to_date('2016-01-02', 'yyyy-MM-dd'
#1                         2016-01-02
#2                         2016-01-02
#3                         2016-01-02
> collect(select(people, to_timestamp(lit('2016-01-02'), 'yyyy-MM-dd')))
  to_timestamp('2016-01-02', 'yyyy-MM-dd')
#1                               2016-01-02
#2                               2016-01-02
#3                               2016-01-02
> select(people, to_timestamp(lit('2016-01-02'), 'yyyy-MM-dd'))

Since this is an existing behavior, this PR LGTM

LGTM - merging master.

@anabranch
I met a failure case when using to_date, like this:
 spark-sql> select func(to_date('2017-08-25 12:00:00'));

func is a custom udf which extends org.apache.hadoop.hive.ql.udf.generic.GenericUDF.
And this following exception happened:
Error in query: No handler for Hive UDF 'xxx.xxx.func': java.lang.UnsupportedOperationException: Cannot evaluate expression: to_date(2017-08-25 12:00:00, None);

I locate this problem. When resolving function func(to_date('2017-08-25 12:00:00'))(at this time Spark has already finished the resolving to func's child to_date('2017-08-25 12:00:00')),  it will call func's FunctionBuilder that is defined in HiveSessionCatalog.makeFunctionBuilder).
In it, the GenericUDF instance will be forced to check input data type. During this process, All of the child catalyst expression which are the parameters of the subclass of GenericUDF, need to be mapped to ObjectInspector(see details: HiveInspector.toInspector).  Inside, the child expression need to be evaluated though eval() firstly, if expression doesn't match with Literal.
def toInspector(expr: Expression): ObjectInspector = expr match {
    case Literal(value, XXXType) =>
        //...
    // ideally, we don't test the foldable here(but in optimizer), however, some of the
    // Hive UDF / UDAF requires its argument to be constant objectinspector, we do it eagerly.
    case _ if expr.foldable => toInspector(Literal.create(expr.eval(), expr.dataType))
}

In my case, the child expression is to_date('2017-08-25 12:00:00') that is the instance of ParseToDate (extends RuntimeReplaceable). When calling its eval(),  we call the Unevaluable's eval() that will throw this exception mentioned above.
trait Unevaluable extends Expression {

  final override def eval(input: InternalRow = null): Any =
    throw new UnsupportedOperationException(s"Cannot evaluate expression: $this")

  final override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode =
    throw new UnsupportedOperationException(s"Cannot evaluate expression: $this")
}

Considering that we barely use the second parameter pattern, I remap to_date to ToDate again to fix the problem.
So, does my case relate to this PR?

I don't think this is necessarily my call but you're effectively writing a
hive udf, not a Spark one that depends on this. Writing a Spark UDF would
allow this to work just fine and the expression to be evaluated. Your work
around seems fine in your local version though.
On Fri, Aug 25, 2017 at 12:23 AM, Yann Byron<EMAIL_ADDRESS>wrote:

@anabranch https://github.com/anabranch
I met a failure case when using to_date, like this:
spark-sql> select func(to_date('2017-08-25 12:00:00'));
func is a custom udf which extends org.apache.hadoop.hive.ql.udf.
generic.GenericUDF.
And this following exception happened:
Error in query: No handler for Hive UDF 'xxx.xxx.func': java.lang.UnsupportedOperationException: Cannot evaluate expression: to_date(2017-08-25 12:00:00, None);
I locate this problem. When resolving function func(to_date('2017-08-25
12:00:00'))(at this time Spark has already finished the resolving to func's
child to_date('2017-08-25 12:00:00')), it will call func's
FunctionBuilder that is defined in HiveSessionCatalog.makeFunctionBuilder
https://github.com/apache/spark/blob/v2.2.0/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala
).
In it, the GenericUDF instance will be forced to check input data type.
During this process, All of the child catalyst expression which are the
parameters of the subclass of GenericUDF, need to be mapped to
ObjectInspector(see details: HiveInspector.toInspector
https://github.com/apache/spark/blob/v2.2.0/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala).
Inside, the child expression need to be evaluated though eval() firstly,
if expression doesn't match with Literal.
def toInspector(expr: Expression): ObjectInspector = expr match {
case Literal(value, XXXType) =>
//...
// ideally, we don't test the foldable here(but in optimizer), however, some of the
// Hive UDF / UDAF requires its argument to be constant objectinspector, we do it eagerly.
case _ if expr.foldable => toInspector(Literal.create(expr.eval(), expr.dataType))
}
In my case, the child expression is to_date('2017-08-25 12:00:00') that
is the instance of ParseToDate
https://github.com/apache/spark/blob/v2.2.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
(extends RuntimeReplaceable
https://github.com/apache/spark/blob/v2.2.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala).
When calling its eval(), we call the Unevaluable
https://github.com/apache/spark/blob/v2.2.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala's
eval() that will throw this exception mentioned above.
trait Unevaluable extends Expression {
final override def eval(input: InternalRow = null): Any =
throw new UnsupportedOperationException(s"Cannot evaluate expression: $this")
final override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode =
throw new UnsupportedOperationException(s"Cannot evaluate expression: $this")
}
Considering that we barely use the second parameter pattern, I remap
to_date to ToDate again to fix the problem.
So, does my case relate to this PR?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/spark/pull/16138#issuecomment-324843845, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABkQBwa1lkNT81uAWDV3ZRSRz8XqApPHks5sbnaMgaJpZM4LDrz0
.


Validate vsphere datacenter config at api level

Issue #, if available:
Description of changes:
Move vsphere datacenter config validation to api level
Testing (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Is the vsphere datacenter config validation still happening at the same place? Is the PR/commit description accurate?

/lgtm

/approve


cdr: for the council

@icepuma I made some changes to this. I don't think we need a council. I'd like anyone to be able to submit frictions and CDRs and to vote.
So let's lay that bedrock and take it from there?

lgtm, merge it


Encoding error at conversion

Hi Ajay !
I have a .bin sentence embedding file generated as word2vec .bin format, and am trying to use the magnitude converter utility to convert it to a .magnitude file.
I believe a .bin generated with fastText should work for conversion, as mentioned in your documentation, but I'm hitting an encoding error in the file attached.
Could you shed some light on this, since there's no encoding option for the converter ?
Is there a solution ?
Thanks !
Jessica T
magnitude_encoding_error.txt

Actually fastText .bin's are not supported unfortunately only word2vec .bins. Is it easy enough to convert these to the .vec format Facebook also releases? We support those. If not, I can look into figuring out how to support these in the future.

Hi Ajay,
Sorry I took a while to get back to you about this.
It seems they only published this .bin file, which I can only read with sent2vec in python.
I tried reading it with other gensim libraries, hoping I might be able to convert it to a word2vec bin instead of a fastText .bin, but no success so far :/
I'll keep digging and will let you know a workaround if I find one, in case anyone else tries this !
Cheers
JT


First release

This will be tagged as 0.1.0. Nothing new to test. Just wanted to have an official PR that we can refer to later if needed.

Looks OK to me!
The run_simulations notebook I made has been made obsolete with the new Bokeh functionality, so we might choose to omit that file.
Under NREC_VM_setup, platform_dev branch is used in several scripts instead of main. I guess that's OK since NREC is mostly for development anyways, and if it's a problem we can fix it later.

Thanks. I replaced platform_dev with main in relevant parts of docs and scripts.
We can keep run_simulations for now for reference and remove it later.

Great idea! We just need to be aware that some functionalities that appear to be included have not been implemented yet, e.g., some settings that are listed in the default settings file do not actually change the model configuration. Moreover, multiple functionalities and edge cases have not been tested thoroughly. I'm not entirely sure what the convention for releases is, but I would prefer if the release tag clarifies this in some way. For example, we could use something like v0.1.0-dev.


69 Window Test => Full screen is hanging/ Second monitor modus not working

Windows 10 / VS2019
Steps to Reproduce:
Start 69 Window Test -> try all modus -> the second Monitor is not working
On v4 it is working only once (than it is no longer shown)

Mybe glfw bugs, I will check next week, my home no double display

please check whether the latest commit fix this issue

Works now. Thanks.
BTW: I works also on an third monitor too

wow, you have so many monitors


Add support for deleting machines that failed creation

If a machine failed creation because of a user misconfiguration, then
when the user corrected their mistake a new machine would get created
but the old machine would not get removed. This change introduces
support for deleting machines without user interaction if the machine
failed creation. In addition, this change makes it possible for a user
to force deletion of a machine if the machine cannot be deleted for some
reason.
This change also cleans up the machine-status controller. A type is
created to simplify the return of a few functions.
A special note: conditions are added here to the infrastructure machine
objects. Based on some code here, it seems that they were always
intended to exist but were leftout.
Issue:
https://github.com/rancher/rancher/issues/34914

https://github.com/rancher/rancher/issues/34945


how to install for L5?

how to install for L5?

+1 what about support for L5?

+1

+1

+1

+1

any update?

Please see my fork. I am using this for Laravel 5. (verified stripe only)

An update to L5 would be great!

@bart you may use my fork - https://github.com/bryceadams/laravel-billing

I could but you didn't test braintree so using the original package would be better. Last but not least including braintree should be easy and possible without using any third party package than braintree itself.

+1

We recommend the official Laravel Cashier package for Laravel 5 users.


Does GSA layer support backward with caches?

Feature request
For some reason I want to run backward with caches. However, I got the following error with GSA (Everything is fine without caches)
  File "/home/wangfan/.local/lib/python3.8/site-packages/torch/_tensor.py", line 521, in backward
    torch.autograd.backward(
  File "/home/wangfan/.local/lib/python3.8/site-packages/torch/autograd/__init__.py", line 289, in backward
    _engine_run_backward(
  File "/home/wangfan/.local/lib/python3.8/site-packages/torch/autograd/graph.py", line 769, in _engine_run_backward
    return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
  File "/home/wangfan/.local/lib/python3.8/site-packages/torch/autograd/function.py", line 306, in apply
    return user_fn(self, *args)
  File "/home/wangfan/.local/lib/python3.8/site-packages/fla/utils.py", line 12, in wrapper
    return fn(ctx,
  File "/home/wangfan/.local/lib/python3.8/site-packages/fla/ops/gsa/chunk.py", line 1071, in backward
    q, k, v, s, g, ok, p, hk, hv, Av, hk0, hv0 = ctx.saved_tensors
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [4, 16, 32, 128]], which is output 0 of torch::autograd::CopyBackwards, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

I have not much idea about this problem. I guess this is because FLA do not support backward with caches.
Motivation
N/A
Your contribution
N/A

Could you provide minimal reproduction code?

Could you provide minimal reproduction code?

A minimal reproduction code:

# Segement-wise Back-Propagation with GSA

import torch
from fla.layers.gsa import GatedSlotAttention
from fla.layers.gla import GatedLinearAttention
from fla.models.utils import Cache

bsz = 4
hidden_size = 256
num_heads = 8
num_slots = 16
seq_len = 1024
seg_len = 128
lr=1.0e-3
use_cache=True # Will be OK if it is False

seg_num = (seq_len - 1) // seg_len + 1

encoder = GatedSlotAttention(
          hidden_size=hidden_size,
          num_heads=num_heads,
          num_slots=num_slots,
          layer_idx=0)

optimizer = torch.optim.Adam(encoder.parameters(), lr=lr)

encoder = encoder.to('cuda')
inputs = torch.randn(bsz, seq_len, hidden_size, device='cuda')
outputs = torch.randn(bsz, seq_len, hidden_size, device='cuda')

if(use_cache):
    cache = encoder.init_state(bsz)
    kvs = Cache.from_legacy_cache([cache])
else:
    kvs = None

b = 0
optimizer.zero_grad()
for seg_id in range(seg_num):
    y, _, new_cache = encoder(inputs[:, b:(b+seg_len)], use_cache=use_cache, past_key_values=kvs)
    loss = torch.sum((y - outputs[:, b:(b+seg_len)]) ** 2)
    b += seg_len
    loss.backward()
    if(use_cache):
        kvs.update(new_cache, 0)

optimizer.step()

@WorldEditors Hi, check out the commit https://github.com/sustcsonglin/flash-linear-attention/commit/39172ad1e9d9a58dbe9b51e88943b475ec907a5a and the following scripts:

# Segement-wise Back-Propagation with GSA

import torch

from fla.layers.gsa import GatedSlotAttention
from fla.models.utils import Cache

bsz = 4
hidden_size = 256
num_heads = 8
num_slots = 16
seq_len = 1024
seg_len = 128
lr = 1.0e-3
use_cache = True  # Will be OK if it is False

seg_num = (seq_len - 1) // seg_len + 1

encoder = GatedSlotAttention(
    hidden_size=hidden_size,
    num_heads=num_heads,
    num_slots=num_slots,
    layer_idx=0)

optimizer = torch.optim.Adam(encoder.parameters(), lr=lr)

encoder = encoder.to('cuda')
inputs = torch.randn(bsz, seq_len, hidden_size, device='cuda')
outputs = torch.randn(bsz, seq_len, hidden_size, device='cuda')

if (use_cache):
    cache = encoder.init_state(bsz)
    kvs = Cache.from_legacy_cache([cache])
else:
    kvs = None

b = 0
optimizer.zero_grad()
for seg_id in range(seg_num):
    for state in kvs.states:
        for i in state:
            i.detach_()

    y, _, new_cache = encoder(inputs[:, b:(b+seg_len)], use_cache=use_cache, past_key_values=kvs)
    loss = torch.sum((y - outputs[:, b:(b+seg_len)]) ** 2)
    b += seg_len
    loss.backward()
    if (use_cache):
        kvs = new_cache

optimizer.step()

Key updates:

avoid in-place copy of newly passed states
detach the states every time before the forward

Issue solved, great work!


How to highlight keywords?

How to highlight keywords?

Here's a codesandbox that does it: https://codesandbox.io/s/ol7yp20966
something like this:
<template slot-scope="{ suggestion }">

bolden (text, suggestion) {
  var regex = new RegExp("(" + text + ")", "gi");

  return suggestion.replace(regex, "$1");
}

Here's a codesandbox that does it: https://codesandbox.io/s/ol7yp20966
something like this:
<template slot-scope="{ suggestion }">

bolden (text, suggestion) {
  var regex = new RegExp("(" + text + ")", "gi");

  return suggestion.replace(regex, "$1");
}

Wow，Thank you very much，it works! Haha


Jesse: Watch tanzilmatjarplay.com

Jesse requests the watch of the watch_keyword tanzilmatjarplay\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

tanzilmatjarplay\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Spevacus in Charcoal HQ


Add condition of not being under Windows to reuse file descriptor

Given that, under Windows, socket.fileno's output "can't be used where a file descriptor can be used" (socket.fileno docs), we need to add the condition of not being in Windows to be able to reuse the file descriptor when the app is reloaded
Closes pallets/werkzeug#993
Closes pallets/flask#2194

Thanks, could you rebase against the 0.12-maintenance branch and add a changelog entry?

@untitaker can you tell me if the rebase was correctly done? I think everything is ok :speak_no_evil:

You need to use rebase -i such that you can edit out my first commit which
doesn't belong in this PR.
On Sun, Mar 12, 2017 at 07:44:01AM -0700, Antonio Ossa wrote:

@untitaker can you tell me if the rebase was correctly done? I think everything is ok :speak_no_evil:
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/pallets/werkzeug/pull/1081#issuecomment-285949102

Done :ok_hand:

I'm not sure if tests are possible, but they'd be useful.

I'm not sure how to test that. Because in the end we will be testing if the WIN constant works... seems weird to me.
I'll edit the last commit to add your suggestion

Nevermind, I think we need to set up CI for Windows.

The entry was changed to:

Fix crash of reloader (used on debug mode) on Windows (OSError: [WinError 10038]). See pull request #1081

You can merge this now and, once you have a CI for Windows, email me or something and I'll try to create a proper test for this :grin:

@untitaker I have no idea why the build failed, but seems like is not related to my changes

Thanks!


FixFix hudi-catalog get file split error (#18644)

cherry-pick #18644
Describe your changes.
Proposed changes
hudi-common depends on parque-avro, but the dependency scope is provide.  When we use hudi-catalog, HoodieAvroWriteSupport will be called. This method depends on parque-avro, so it will generate ClassNotFound
Problem summary
Describe your changes.
add parque-avro dependency
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.68 seconds
stream load tsv:          429 seconds loaded<PHONE_NUMBER>9 Bytes, about 166 MB/s
stream load json:         21 seconds loaded<PHONE_NUMBER> Bytes, about 107 MB/s
stream load orc:          61 seconds loaded<PHONE_NUMBER> Bytes, about 17 MB/s
stream load parquet:          30 seconds loaded 861443392 Bytes, about 27 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230414093848_clickbench_pr_129100.html


Babel ENV Issue Caused Only by Standard VS Code Extension

What version of this package are you using?
Newest as of 9/8/21
What operating system, Node.js, and npm version?
MacOS/12 LTS
What happened?
When StandardJS extension is installed or enabled, I received a pointless error at the start of every file about envs.
What did you expect to happen?
Nothing
Are you willing to submit a pull request to fix this bug?
no
Here is the error I get at the top of my file:
Parsing error: [BABEL] : Using babel-preset-react-app requires that you specify NODE_ENV or BABEL_ENV environment variables. Valid values are "development", "test", and "production". Instead, received: undefined. (While processing: "")standard
I don't even understand why standard is doing this. The old extension had issues with decorators, and this is happening ever since the upgrade. Please advise how I can get standard back to a useful formatting tool instead of a whining extra linter that I don't need.

I'm getting it to shut up about the non-exsistent error by utilizing a deprecated lib for now. Changed:
{
  "standard": {
    "parser": "@babel/eslint-parser"
  }
}

from docs at https://github.com/standard/standard#how-do-i-use-experimental-javascript-es-next-features
to:
{
  "standard": {
    "parser": "babel-eslint"
  }
}

I guess it doesn't matter because only standard is using this old deprecated library in my project, but whatever went into not making this break with the old eslint parser should probably be updated. I'm guessing standard is just doing too much. Most of us probably don't want standard looking for this type of error anyway.


Investigate forced proportional adds

If pool state changes quickly it can make it difficult to add liquidity to the pool.

I think we more deeply understand the issues involved in this one now. The original problem was that the SDK helper function was rounding up which could result in amountsIn above the user's balance during the simulation of the tx. This has now been fixed in #906


Implement Cross Resource Referencing for resource types

Signed-off-by: soorena776<EMAIL_ADDRESS>
Description of your changes

Fixes #948
This PR converts the existing script based installation of WordPress workload example, to gitops model. Now the user can install the entire workload in two steps:

Install AWS secure connectivity configuration, and create cloud specific resources by running

 kubectl apply -k cluster/examples/workloads/kubernetes/wordpress/aws/gitops/aws-nfra

Install the application specific resources  by running

 kubectl apply -k cluster/examples/workloads/kubernetes/wordpress/aws/gitops/wordpress-dev

Checklist

I have:

[x] Run make reviewable to ensure this PR is ready for review.
[x] Ensured this PR contains a neat, self documenting set of commits.
[ ] Updated any relevant documentation, examples, or release notes.
[ ] Updated the RBAC permissions in clusterrole.yaml to include any new types.

@prasek @hasheddan @negz PTAL!


OSX Info Page

From Duke :
Even Windows has its own info page. All OSX gets is some videos talking about Apple hardware and not the actual operating system. My suggestion is to have an actual OSX info page.

Probably not going to happen.  Both were given extremely minimal effort as this is not a Windows or macOS server.  If you want to make one, feel free to submit it.

Just the bullet points would be all you would need to submit; I can take care of everything else.

You can't actually use macOS on non-Apple hardware, so in my opinion the videos are relevant.

The current info page is titled macOS so I just think it should actually be about macOS. If people want to know about macOS currently there is nothing for that. Those videos also only cover certain hardware.

From Faith | Am Girl! :
This went nowhere and was left open, closing.


⚠ 366.ru has degraded performance

In b983836, 366.ru (https://366.ru) experienced degraded performance:

HTTP code: 200
Response time: 1625 ms

Resolved: 366.ru performance has improved in 4e9b6f6.


Can I login without password or authentication

I want to use terminal by my user when i link Wetty that without password,
what can i do?
or give me some proposal?

0.0

SSH keys. Look up password less ssh key setup in google.

Thanks,but i don't know how to use SSH key to do.
so i try another way to do
XD

@VagrantPi
First add passwordless users:
useradd -m -p U6aMy0wojraho john, explanation: http://askubuntu.com/questions/14222/how-to-add-a-guest-account-without-a-password
and use ssdd_config like this:
# Package generated configuration file
# See the sshd_config(5) manpage for details

# What ports, IPs and protocols we listen for
Port 22

# Forces the execution of the command specified by ForceCommand, ignoring any
# command supplied by the client and ~/.ssh/rc if present. The command is
# invoked by using the user's login shell with the -c option.  This applies
# to shell, command, or subsystem execution.  It is most useful inside a
# Match block. The command originally supplied by the client is available in
# the SSH_ORIGINAL_COMMAND environment variable. Specifying a command of
# “internal-sftp” will force the use of an in-process sftp server that
# requires no support files when used with ChrootDirectory.
ForceCommand /post_login.sh

# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress <IP_ADDRESS>

Protocol 2
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key

#Privilege Separation is turned on for security
UsePrivilegeSeparation yes

# Lifetime and size of ephemeral version 1 server key
KeyRegenerationInterval 3600
ServerKeyBits 1024

# Logging
SyslogFacility AUTH
LogLevel INFO

# Authentication:
LoginGraceTime 120
PermitRootLogin yes
StrictModes yes

RSAAuthentication yes
PubkeyAuthentication yes
#AuthorizedKeysFile     %h/.ssh/authorized_keys

# Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes

# For this to work you will also need host keys in /etc/ssh_known_hosts
RhostsRSAAuthentication no

# similar for protocol version 2
HostbasedAuthentication no

# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication
#IgnoreUserKnownHosts yes

# To enable empty passwords, change to yes (NOT RECOMMENDED)
PermitEmptyPasswords yes

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Change to no to disable tunnelled clear text passwords
#PasswordAuthentication yes

# Kerberos options
#KerberosAuthentication no
#KerberosGetAFSToken no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

X11Forwarding yes
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no

#MaxStartups 10:30:60
#Banner /etc/issue.net

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin yes".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

In wetty.js I added this in the socket.on scope:
       // dirty login... 	socket.emit('input', "Piet\r");          setTimeout(function() { 		socket.emit('input', "Paulusma\r");	         }, 200);
It's very unsave (password in a plain text html file). But its just a docker password for local testing.....


Add cluster autoscaler

Noodling around trying to see how easy/hard it is to run the cluster autoscaler.
There's a couple of FIXME/TODOs I'd like to address before merging.
I haven't actually tried to configure the autoscaler itself yet; just get it up and running with the appropriate permissions.

This needs a rebase. But no-one else can do it until Phil is back unless we want to re-raise the PR. ;-)

This PR is a mess, opening a new one


Basic Auth support

:lock: Basic Auth support from Spring Security and can be set via env
:pencil: Add documentation for Basic Auth configuration and usage via
docker
project_management#114
Changes

Add Configuration for Basic Auth with Spring Security
Add Docker env variables for Basic auth usage
Add documentation for EHRbase and Docker image usage

Related issue
Closes https://github.com/ehrbase/project_management/issues/114
Additional information and checks

[x] Pull request linked in changelog

The integration test are all falling. please disable auth for Integration tests

I was not able to successfully authenticate using the credentials. I used the examples provided in the readme and also checked with the security config (additionally set my own via environment variables). We might need to clarify if the problem is on my end.


Only 50 last tasks shown for workflows with many tasks

We have a workflow which generates tasks in mistral using the with-items syntax and since this needs to be carried out for a large number of item we get a very large workflow. Looking at this workflow in the webui I noticed that it appears that when a workflow has to many tasks only the last 50 are shown in the webui. While all the information is still available via the cli it might make troubleshooting these very large workflow more difficult.

Thanks for the report and sorry for the late reply.
Indeed, we're limiting a single query to /executions endpoint to 50 entries to keep UI responsiveness at bay and also prevent browser JSON parser from hanging (which is sadly the case even at this day and age).
How big of a workflow execution are we talking about in your case? Is is just a few executions that goes missing or are we talking about hundreds?

Sorry for taking so long to come back to you on this - I've been on vacation. We currently have a workflow which generates about 500 tasks... This might not be the best way to go about things, but except for not showing up in the web-ui, it appears to be working.
I do realize that this kind of workflow might be kind of niche, so feel free to close this issue if you think that it is out of scope.

Yes, 500 tasks per workflow execution invalidates most of our UX assumptions on the spot. Loading them all in one go is out of the question, even simple "show more" approach to pagination would quickly kill the browser with sheer amount of DOM nodes required.
I wonder, how it works for you in CLI? How long it takes to st2 execution get such a workflow? Are you using grep to sort through them or just doing it by eye?
All in all, it seem to me that you need an entirely different set of tools to work with such a huge workflows. At the moment we're starting to explore things we can improve for v2 API and this issues seem like an interesting direction we should look into.
As for an immediate action, we need to at least let user know there's more tasks in this workflow than UI is showing. I'll look into it.

It's not that fast, but acceptable in the CLI. Takes about 7 s to display. And we do use grep to sort through them for troubleshooting.
Sounds like a good idea to add some type of notice that not all tasks are showing - left me a bit confused until I figured out only 50 of them were showing... :)

I'm hitting this same limitation using Stack Storm 2.6.0.  We have 75 actions that get executed, some are in retry loops, and only the last 50 are displayed under the parent workflow in UI.


[Chore] Add mim extras_requires in setup.py

Motivation
Add mim extras_requires in setup.py
Modification

setup.py

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Move numba.dppl into a standalone extension/package

This ticket is to track the initial set of tasks/ideas to create a numba-dppy extension for numba.

[ ] Can we move the numba.dppl sub-module to its own repo called numba-dppy?
[ ] Can we create a new package called numba-dppy? How to resolve dependency on numba?
[ ] Add a dependency to our fork of numba on the new package.
[ ] Intel GPU functionalities are enabled only when the new package is present.
[ ] How should the new package be used? Directly imported by users or indirectly by numba.
[ ] Figure out how to structure and run numba-dppy tests.
[ ] Set up a CI for numba-dppy.

[ ] Changes to numba-dppy should not break any numba test cases when the package is used.

The above tasks relate to only changes that are compartmentalized into the numba.dppl sub-module, and do not include changes that we have made to code numba. To handle the changes to core numba, we should do the following:

[ ] All changes should be logically grouped into separate patches that are needed for our numba fork that is part of oneAPI.
[ ] oneAPI's numba should be built with these patches.
[ ] Any further changes should also be added in the same way.

Once these initial changes are understood, we will start working on minimizing the need of patches by :

[ ] move them cleanly into numba.dppy
[ ] upstream them
[ ] create more plug-ins

@PokhodenkoSA @reazulhoque I am created this high-level task to track our work to isolate our changes into a standalone package. I jotted down the initial design I had in mind.

I just saw #76. @PokhodenkoSA close one of the tickets please.

closing in favor of #76


Adding shadow robot repos to documentation index for distro melodic

I'd like shadow robot repos to be indexed and documented on ros.org.

when do you think this will merged?


Dashboards: following Span Links works in unexpected ways

While preparing a demo for .NET9's experimental connection tracing feature, I discovered that after clicking View on the span link, the UI is showing the wrong span (keeps visualizing the request span):

I would like to see the linked connection_setup span expanded with it's subspans. To get there quickly, I tried to paste the spanID into the filter input box, but it filters by name by default, so eventually the quickest solution was to find the correct connection_setup span manually by scrolling down and using the browser's search function:

This feels like a bug, or if it works like this by design, the UX is clearly sub-optimal.

I feel the same about this. I want it to navigate to that trace when I click it, not just updated the side panel...


Option to discard burn-in for HMC/MH

Not sure if I'm looking at this correctly, but from the implementations of AbstractMCMC.transitions_init! and AbstractMCMC.transitions_save!, it looks like if a sampler has an n_adapts attribute, then the first n_adapts samples are discarded as burn-in (which is the case in, for instance, HMCDA and NUTS).
I think in practice it's useful to be able to discard burn-in samples for HMC as well, but that doesn't seem to be an option currently. Was there some reason for this? If having the ability to discard burn-in samples is a desirable feature, is this something that can be easily implemented?

I agree, I have been wondering about this for a while. IMO this is such a common use case that maybe one should support and implement it in the generic AbstractMCMC interface instead of Turing only. All algorithms in Turing (and other packages that implement the AbstractMCMC interface) would then get this functionality automatically.

It's pretty easy to add IMO. Maybe we could have something like
chain = sample(model, spl, 1000, burnin = 100)

To keep the last 900 samples. Not sure I like that keyword argument though.

@devmotion

IMO this is such a common use case that maybe one should support and implement it in the generic AbstractMCMC interface instead of Turing only.

I like this idea.
@cpfiffer

Not sure I like that keyword argument though.

My personal preference would be nburn. This is what the following PPLs use:

STAN: warmup
TFP: num_burnin_steps
Pyro: warmup_steps
Numpyro: num_warmup

In TFP, you can specify both a number of burn-in samples to discard (from the beginning), and a number of iterations to adapt sampler parameters (e.g. in HMCDA and NUTS). In STAN, Pyro, and Numpyro, I believe you can only specify the number of samples to burn and afterwards sampler tuning parameters won't be further adapted. I was more thinking of the latter at first.
@devmotion's suggestion would be more like what is done in TFP, which I think is a little more flexible, and probably more sensible. I suppose if we have an optional burnin argument in sample, some checks are needed to ensure there are no conflicts with nadapt in some samplers (like NUTS)?

IMO nburn is not descriptive enough. I'd suggest something clearer such as discard, ndiscard, discard_initial, or discard_samples. I guess it would be preferable to keep it separate from specific considerations for HMC since this is a general approach when sampling Markov chains.


Support { __proto__: prototype } pattern for V8 compatibility option

V8 supports setting the prototype of an object literal with the inclusion of an entry for the key __proto__. This works regardless of the value of __proto__ on Object.prototype.

I am sorry, I don't understand what you request. __proto__ property of object literal sets the prototype of the literal already:
$ js
> var myProto = {};
> var myObject = { __proto__: myProto };
> Object.getPrototypeOf(myObject) === myProto
true

I am sorry, I don't understand what you request. __proto__ property of object literal sets the prototype of the literal already:
$ js
> var myProto = {};
> var myObject = { __proto__: myProto };
> Object.getPrototypeOf(myObject) === myProto
true

This won't work:
delete Object.prototype.__proto__;
{ __proto__: null };

It would help if you described the exact steps to reproduce the problem (including the description of the actual versus the expected behavior). I still fail to see what does not work:
$ js
> delete Object.prototype.__proto__;
true
> var o = { __proto__: null };
> Object.getPrototypeOf(o)
null

It would help if you described the exact steps to reproduce the problem (including the description of the actual versus the expected behavior). I still fail to see what does not work:
$ js
> delete Object.prototype.__proto__;
true
> var o = { __proto__: null };
> Object.getPrototypeOf(o)
null

You mean the command line options?

You mean the command line options?

I mean that it is not clear at all what you are complaining about. You failed to describe the difference between the actual and the expected behavior of any relevant test-case.

You mean the command line options?

I mean that it is not clear at all what you are complaining about. You failed to describe the difference between the actual and the expected behavior of any relevant test-case.

Expected:
> delete Object.prototype.__proto__;
true
> Object.getPrototypeOf({ __proto__: null });
null

Actual:
> delete Object.prototype.__proto__;
true
> Object.getPrototypeOf({ __proto__: null });
[object Object]

The last line of the actual output might have been {} instead. I'm going by memory.

The behavior that you describe as expected is the one that I see also as actual. Maybe you should verify the actual behavior by trying it locally instead of "going by memory".


Implement 'to have no calls satisftying' assertion

See #32.

Coverage decreased (-2.4%) to 90.22% when pulling f5b393f510d2243af82172195a51b40f704422de on feature/toHaveNoCallsSatisfying into 280e7018966936fa65da0fcea9115c893b60d2c1 on master.

Needs some more tests, I know :)

Coverage increased (+0.8%) to 93.399% when pulling cd199043f91e51447253023a348fe1ff5e83db7a on feature/toHaveNoCallsSatisfying into 280e7018966936fa65da0fcea9115c893b60d2c1 on master.

Thanks!
While you're at it, could you take a look at the very similar https://github.com/unexpectedjs/unexpected-sinon/pull/40 as well?

Released in 10.6.0.


Feature/results refactor

Doesn't seem like the parallelization helped speed up the alternative and alternative comparison result computes...

you might need to run the test in commandline and see if there are any sync lock issues due to the nature of your loop.

debug with a profiler in a commandline app i mean

understood. this PR has logic I'd like to get in main now so I will merge and come back to issue #318 ASAP


Release network microsoft.network official 2023 04 01 2

ARM (Control Plane) API Specification Update Pull Request
PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

Purpose of this PR
What's the purpose of this PR? Check all that apply. This is mandatory!

[x] New API version. (Such PR should have been generated with OpenAPI Hub, per this wiki doc.)
[ ] Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
[ ] Update existing version to fix swagger quality issues in S360.
[ ] Other, please clarify:

edit this with your clarification

Due diligence checklist
To merge this PR, you must go through the following checklist and confirm you understood
and followed the instructions by checking all the boxes:

[x] I have reviewed the general guidance on the spec PR review process: https://aka.ms/specprreview.
[ ] I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data-plane related specifications.
[ ] I commit to follow the Breaking Change Policy.
[ ] I have reviewed following Resource Provider guidelines, including
ARM resource provider contract and
REST guidelines (estimated time: 4 hours).
I understand this is required before I can request review from an ARM API Review board.

ARM API changes review

If you want for the ARM team to review this PR, you must add the ARMReview label.
The automation may automatically add the ARMReview label, if appropriate.
If this happens, proceed according to guidance given in GitHub comments also added by the automation.

Breaking change review
If you have any breaking changes as defined in the Breaking Change Policy,
follow the process outlined in the High-level Breaking Change Process doc.
Getting help

For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
For additional help, see https://aka.ms/azsdk/support/spectools.

Taken 5 PRs from NRP partners and those PRs already got ARM approval:
https://github.com/Azure/azure-rest-api-specs/pulls?q=is%3Apr+base%3Arelease-network-Microsoft.Network-official-2023-04-01-2+is%3Aclosed

@rkmanda could you please help with the review of NRP PR for swagger release. We are already delayed for this release. Your help is appreciated.

Attached previous discussion history for record


Synced to chain error handling

Is your feature request related to a problem? Please describe.
Other clients (e.g. lnd) have specific handling if they recognize that their chain backend is not synced to chain e.g. allowing certain calls like generic getinfo, but blocking others throwing a SyncedToChain error. Also, they expose the synced_to_chain status via a generic health call.
Describe the solution you'd like
Something similar for connext.
Describe alternatives you've considered
External services like xud doing the health check of the backend via the forwarded rpc with e.g. eth.syncing and not calling certain endpoints. Not a great option imho.

This is to avoid ugly situations like with the current non-vector connext client being completely unresponsive if e.g. the eth provider is not fully synced:
bash-5.0# curl -H 'Content-Type: application/json' http://localhost:5040/config
curl: (7) Failed to connect to localhost port 5040: Connection refused

If connext client stays up and responds with a Eth provider not synced to chain error, services like xud can take care of e.g. switching to another eth provider.


Custom random module for use in JIT mode

This merge adds a custom Parcels-specific random module to the kernel API that works similar to the math module and is available in JIT kernels or through pure python. It provides the functions:

random(): Random float between 0 and 1
uniform(low, high): Random float between low and high
randint(low, high): Random integer between low and high

The merge also adds a set of basic arithmetic tests that covers the core arithmetic and logical functionalities of the JIT engine. It is also worth noting that the random module uses the basic C rand()/srand() API, so I make no guarantees about the quality of the RNG sequence. Once this becomes an issue we can incorporate other RNGs, like PCG, but for now plain old C avoids licensing issues.

Looks really good! Happy for this to land, if the minor issue with the double 'grid' in kernel.py is sorted

Cool, thanks, I just pushed the fix. Let's wait for travis to finish before we merge.


Screen Marker Merge

Would it be possible to merge multiple screen markers together so you can view/hide multiple screen markers at once. I use like 4-5 at a time when doing bank standing skills and it would be awesome if it were possible to do this.

Duplicate of #3205


Reorganize delayed submission with operation-specific data and expose tag mask

Add new class to organize operation-specific data, making members optional depending on what a specific transfer operation requires and checking their validity at construction time.
Expose tag mask to tag and multi-buffer tag APIs.

Thanks @wence- !

/merge


Updated example for xPfxImport

Updated example on Readme to include the mandatory items required by the DSC Resource

This change is

@ITGuyOU,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

Thanks @ITGuyOU - I'll merge this one after the other PR #58 is merged.


NoSuchFieldError: USE_PUSH_API

Hi,
I updated a friends Debian 12 box today which loaded Debian 12.8 and new Jitsi Packages from the stable Jitsi.org APT source.
After rebooting i get
JVB 2024-11-13 14:04:13.900 INFORMATION: [1] JitsiConfig.<clinit>#47: Initialized newConfig: merge of /etc/jitsi/videobridge/jvb.conf: 1,application.conf @ jar:[file:/usr/share/jitsi-videobridge/j](file:///usr/share/jitsi-videobridge/j)
itsi-videobridge.jar!/application.conf: 1,system properties,reference.conf @ [jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/reference.conf](jar:file:///usr/share/jitsi-videobridge/jitsi-videobridge.jar!/reference.conf): 1,reference.conf @ jar:[file:/usr/share/ji](file:///usr/share/ji)
tsi-videobridge/lib/jitsi-media-transform-2.3-174-gd011ddf7.jar!/reference.conf: 1,reference.conf @ [jar:file:/usr/share/jitsi-videobridge/lib/jitsi-media-transform-1.0-261-g7b0aeed.jar!/reference](jar:file:///usr/share/jitsi-videobridge/lib/jitsi-media-transform-1.0-261-g7b0aeed.jar!/reference)
.conf: 1,reference.conf @ [jar:file:/usr/share/jitsi-videobridge/lib/ice4j-3.0-35-gadca67b.jar!/reference.conf](jar:file:///usr/share/jitsi-videobridge/lib/ice4j-3.0-35-gadca67b.jar!/reference.conf): 1,reference.conf @ jar:[file:/usr/share/jitsi-videobridge/lib/ice4j-3.2-4-g1373788.ja](file:///usr/share/jitsi-videobridge/lib/ice4j-3.2-4-g1373788.ja)
r!/reference.conf: 1
JVB 2024-11-13 14:04:13.917 INFORMATION: [1] ReadOnlyConfigurationService.reloadConfiguration#51: loading config file at path /etc/jitsi/videobridge/sip-communicator.properties
JVB 2024-11-13 14:04:13.919 INFORMATION: [1] JitsiConfig.<clinit>#68: Initialized legacyConfig: sip communicator props (no description provided)
JVB 2024-11-13 14:04:14.162 INFORMATION: [1] JvmMetrics.<init>#36: Detected GC type G1
JVB 2024-11-13 14:04:14.169 INFORMATION: [1] MetricsUpdater.addUpdateTask#45: Scheduling metrics update task with interval PT5S.
JVB 2024-11-13 14:04:14.184 INFORMATION: [1] JitsiConfig$Companion.reloadNewConfig#94: Reloading the Typesafe config source (previously reloaded 0 times).
JVB 2024-11-13 14:04:14.199 INFORMATION: [1] MainKt.main#81: Starting jitsi-videobridge version 2.3.174-gd011ddf7
JVB 2024-11-13 14:04:14.215 INFORMATION: [1] UlimitCheck.printUlimits#109: Running with open files limit 65000 (hard 65000), thread limit 65000 (hard 65000).
JVB 2024-11-13 14:04:14.216 SCHWERWIEGEND: [1] MainKt.main$lambda$0#60: An uncaught exception occurred in thread=Thread[main,5,main]
java.lang.NoSuchFieldError: USE_PUSH_API
        at org.jitsi.videobridge.MainKt.startIce4j(Main.kt:227)
        at org.jitsi.videobridge.MainKt.main(Main.kt:84)
        at org.jitsi.videobridge.MainKt.main(Main.kt)

I restore Proxmox Snapshot so he cann use Jitsi again, but I am looking for a solution to the above to keep his system updated.
Any hints, even temporary workarounds are greatly appreciated.
Thank you.

You have several ice4j in your folder:
/usr/share/jitsi-videobridge/lib/ice4j-3.0-35-gadca67b.jar
/usr/share/jitsi-videobridge/lib/ice4j-3.2-4-g1373788.jar
The latest jvb stable builds come with https://github.com/jitsi/jitsi-videobridge/blob/cc337a2abe033a4a2785ee6ef6f0303c1ca2fc9a/pom.xml#L109
Drop the other one.
Please, when you have questions or problems use the community forum before opening new issues, thank you.


Remove 'Container Compliance'

I ran into this project which, I first thought it's in the wrong category according to the description, is no longer active.

@devhkr - the obsolete message mention the oscap-docker [1] tool which provides the same functionality

Obsolete!
This project is obsolete, the functionality is provided by the oscap-docker tool that ships with OpenSCAP.

Instead of remove the project, I suggest to replace it - what did you think?
[1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Security_Guide/sect-Using_OpenSCAP_with_Docker.html

@devhkr it looks like https://github.com/OpenSCAP team provides docker compliance functionality as part of their https://github.com/OpenSCAP/openscap project.
See commit msg on https://github.com/OpenSCAP/container-compliance/commit/6b6090a9b3cdb05aa42b34b276f5625a36e88e9c
Here are steps of running it on RHEL: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Security_Guide/sect-Using_OpenSCAP_with_Docker.html
Wonder if we should not remove but update this project info because security is important and we have very few compliance resources as part of awesome-docker list.

Thanks for your feedbacks @agebhar1 @dmitrytokarev
Yeah, had to read the notice again and again what it could mean. Then, also found out as you both correctly did that functionality is provided by oscap-docker.
OpenSCAP-daemon seems to be the right project?

@devhkr, oscap-docker is part of OpenSCAP - see @dmitrytokarev's comment :wink:
See also 5.2. Scanning of Docker containers and images using oscap-docker

@agebhar1 thanks for the hint, again.
OpenSCAP provides several tools and
Yes, oscap-docker is part of the OpenSCAP base tool.
Ok then, should I fix the project info?

from my point of view - yes
Thanks @devhkr for this update!

Thanks @dmitrytokarev for your review.
Yes, I was going to add a link to the usage section. However, the documentation is quite huge. For starters, the first two sections (introduction and installation guide) are relevant in order to understand and to use the oscap-docker tool. Maybe, replace the current link to the documentation site?

I gave it a second thought...
The link to the docs is this: https://github.com/OpenSCAP/openscap/blob/maint-1.2/docs/manual/manual.adoc#scanning-of-docker-containers-and-images-using-oscap-docker
but it'd be pain to maintain, so probably will just leave this as is - let curious minds find the docs they need! 😸

@dmitrytokarev @agebhar1 @devhkr Good team work 👍


UI 🎉(Admin Dashboard)

Admin should be able to:

Add a podcast title
Add a podcast author
Add a podcast file (audio)
Submit podcast

Dev Note
Create a form with three inputs of the title, author (type text), and file (type file). With the submit button.

i want to work on this feature

i want to work on this feature

That will be nice, you can go ahead and work on it.


Media Folder - Cannot read property 'length' of undefined

Version information

Laravel: v5.8.37
Voyager: v1.3
PHP: 7.1.33
Database: MySQL 5.7.29

Description
This is more of a problem than a bug. I've been tasked with fixing a voyager install. The /admin/media loads fine but when some (not all) folders are selected I get the following console error and I get a spinning loading icon:
voyager-assets?path=js%2Fapp.js:1 TypeError: Cannot read property 'length' of undefined
at a.eval (eval at Ka (voyager-assets?path=js%2Fapp.js:1), :3:8100)
at a.e._render (voyager-assets?path=js%2Fapp.js:1)
at a.r (voyager-assets?path=js%2Fapp.js:1)
at pn.get (voyager-assets?path=js%2Fapp.js:1)
at pn.run (voyager-assets?path=js%2Fapp.js:1)
at hn (voyager-assets?path=js%2Fapp.js:1)
at Array. (voyager-assets?path=js%2Fapp.js:1)
at Xe (voyager-assets?path=js%2Fapp.js:1)
Screenshot of error: https://imgur.com/BbXqsxf
Additional context
I've moved the build away from a stand alone server to AWS and the problem persists. The build was originally Voyager 1.2.X and I updated it to 1.3 in hopes of fixing this issue, with no luck. After extensive google-foo, messing with file permissions and a long review of issues on this gitlab, I've been unable to find anyone with a similar issue or a something that lead me to a solution. I've looked through all error logs that I'm aware of and have not found anything relevant or related. I'd really appreciate it if someone could even just give me a hint as to what the issue might be.
Thank you, whoever you are, for taking a moment and reading this.
Steps To Reproduce
Sadly, I don't think its possible without the specific build I'm working with.
##Update
I've continued to bash my face on this and I've found something relevant. I can open and view a folder in /admin/media just fine until the folder contains a .PNG file, and that's when it throws the above error.

##Fix
On the off chance someone stumbles upon this with the same issue, open up /vendor/tcg/voyager/resources/views/media/manager.blade.php and look for the following bit around line 141:

 1" class="right_none_selected">
the value of 'selected_files.lengh' can be null, and is where the error comes from. Create a copy of manager.blade.php to /resources/views/vendor/voyager/media (you may need to create new directories to do this). This file will now override the vendor file, I've changed the above line to the following:

I hope this helps someone!

@Peabo83 I don't really think that if selected_files can be null your change is a solution, you have the same if just 4 lines below the one you are changing.

 1" class="right_none_selected">

@{{ selected_files.length }} {{ __('voyager::media.files_selected') }}

I don't really see how selected_files can be undefined since it's initialized as empty array and reset as empty array, only operation done on it is push.
I can't reproduce the problem with a PNG, what's the result of ajax call to /admin/media/files when you enter that folder?


Vitualenv filelock logging needs to be toned down

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
342 unchanged lines in 25 files lost coverage.
Overall coverage decreased (-4.3%) to 76.763%

Files with Coverage Reduction
New Missed Lines
%

src/main/python/pybuilder/pip_utils.py
1
94.12%

src/main/python/pybuilder/pluginloader.py
1
87.1%

src/main/python/pybuilder/plugins/python/remote_tools/init.py
1
27.4%

src/main/python/pybuilder/terminal.py
1
75.47%

src/main/python/pybuilder/extern/init.py
2
85.9%

src/main/python/pybuilder/plugins/core_plugin.py
3
88.06%

src/main/python/pybuilder/remote/init.py
3
76.17%

src/main/python/pybuilder/plugins/python/coverage_plugin.py
4
51.83%

src/main/python/pybuilder/execution.py
5
91.44%

src/main/python/pybuilder/install_utils.py
5
87.88%

Totals

Change from base Build<PHONE_NUMBER>:
-4.3%

Covered Lines:
4707

Relevant Lines:
5968

💛  - Coveralls


adding the introduction setion of the workshop

Issue #, if available:
Description of changes:
Testing (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

assign to @jonahjon

/lgtm

/cherrypick release-0.8


⚠ Bofrost.lu has degraded performance

In 0d1f7f5, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1120 ms

Resolved: Bofrost.lu performance has improved in c1e2c6a after 21 minutes.


[Bug report]: Ledger Nano S (NOT PLUS) and Firefly 1.7 = Cannot Stake or Send Transactions

Platform
Desktop
Version
1.7.0
Description
Since the Upgrade to Firefly 1.7.0 i cannot send or Stake a Wallet with 300 Transactions.
I was able to Stake 2 Wallets with a few Transactions after multiple trys.
Was Firefly 1.7 tested with Ledger Nano S Accounts? Or were only Ledger Nano S plus Part of the Tests?
Soon the Staking begins and multiple Guys cannot Stake their funds, so this should be a prioritized Topic.
Expected behaviour
That Firefly would let me send and Receive Transactions ;-) like in the previous versions.
Actual behaviour
Problem persists.
also restarts, another Computers etc. are helping. Some friends with the same Hardwareconfiguration (Ledger Nano S) are facing the same Problems.
Can the issue reliably be reproduced?
Yes
Steps to reproduce the issue

Upgrade Firefly 1.6X (all working) to Firefly 1.7
Try to Stake or Send a Transaction with a Wallet with 300 Transactions in Combination with a Ledger Nano S.

Operating System
Windows
Diagnostics
Version 1.7.0
Language: en
Currency: USD
Node selection: Automatic
Platform: win32
Platform Version: 10.0.22621
Platform Architecture: x64
CPU Count: 24
Total Memory: 32488.7 MB
Free Memory: 17727.9 MB

Developer Profile
No
Profile Type
Hardware (Ledger)
Staking
Previously staked
Crash Status
None of the above
Error Messages
"Something went wrong" are sometimes displayed. But the Error Logs are empty.
Error Logs
Error Log:

The Error Log is empty.

Duplicate declaration

[X] I have searched the issues tracker this issue and there is none

Code of Conduct

[X] I agree to follow this project's Code of Conduct

We developed after many Testings a Workaround:

Downgrade to Firefly 1.6.5
Reconfigure Firefly to use a Private Node (a whole Part of the Problem seems Timeouts on Wallets with >100 Adresses). A good Private Node is way faster then the default Public IF Nodes.
Send MIOTA to another Account
If a error occured (after multiple Trys), Send the Coin per Receive-Address each individually until all Coins transfered.
Upgrade to Firefly 1.7 and stake the Amount from the new Wallet.

Hey,
do you know if the problem will be fixed in the long run? Because I have exactly the same problem. And even despite the fact that I only have 11 transactions. What I also noticed is that it's very difficult to connect my Ledger to Firefly - you could almost say it connects kind off randomly after waiting for quite a while. Was or is this also the case for you?


Fix formatting of watched_keywords.txt lines 205 and 206

A couple of lines in watched_keywords.txt had broken formatting; line 205 had a trailing space (I'm not sure if it's a big deal or not, but I removed it), and line 206 had two spaces instead of a tab (which caused an error message when loading the keywords).

Codecov Report

Merging #865 into master will decrease coverage by 0.17%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #865      +/-   ##
==========================================
- Coverage   63.73%   63.56%   -0.18%
==========================================
  Files           6        6
  Lines        1762     1762
==========================================
- Hits         1123     1120       -3
- Misses        639      642       +3

Impacted Files
Coverage Δ

findspam.py
82.39% <0%> (-0.77%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d015612...4e49d2a. Read the comment docs.

@codecov-io That's because the error handler doesn't get run

!!/approve


JavaScript: Teach IncompleteSanitization to flag incomplete path sanitizers.

This would have flagged https://github.com/prahladyeri/http-live-simulator/commit/8e85a1be562248d0d616c0e5092a3d71bbf5fe5f. No result or performance changes on default.slugs, so I suggest we forgo a change note.
An interesting extension could be to flag .replace(/\/..\//g, ""), which runs afoul of /./.././, but that perhaps falls under the purview of the proposed multi-character sanitization query.

An interesting extension could be to flag .replace(/\/..\//g, ""), which runs afoul of /./.././, but that perhaps falls under the purview of the proposed multi-character sanitization query.

I think you mean /\/\.\.\//g? 😄
Anyway, LGTM but an evaluation or query console run to check for FPs would be nice.

I think you mean (...)

Don't I just. Fortunately we have a query for that.

an evaluation (...) would be nice.

I did a full evaluation on default.slugs, and there were no result changes.

Fortunately we have a query for that.

Nit picking: we do not have a query for that, we only flag unescaped hostname dots in relation to host names.
We could add a query that looks for unescaped double-dots in regexps, but I have a hard time coming up with a scenario where that wouldn't be flagged by a multi-character sanitization query instead.

Nit picking: we do not have a query for that


Saving output as tscn

I did a little addition to my workflow by letting an output node to save the result as a .tscn file. It is useful to me and I believe it should be helpful for more people. I can send a pull request if deemed worthwhile.

It would definitely be useful. How does it work? Did you added a tool script on the output node and exported a file path where the output is saved?

I (unknowingly) sent the change in pr #40 .


feat: [IDP-3071]: Added a docs on how to create the catalog-info.yaml

Description

Please describe your changes: Added a docs on how to create the catalog-info.yaml

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
PRs must meet these requirements to be merged:

[x] Successful preview build.
[x] Code owner review.
[x] No merge conflicts.
[x] Release notes/new features docs: Feature/version released to at least one prod environment.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66700a3341e4c3280df812a1--harness-developer.netlify.app


chore: remove extra action tp deploy site

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.006%) to 91.271%

Totals

Change from base Build<PHONE_NUMBER>:
0.006%

Covered Lines:
11820

Relevant Lines:
12458

💛  - Coveralls


Fix Oracle specific

This is FIX proposed and tested for PDI-19547
Unit Testing needs to be added

:x: Build failed in 1m 28s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core

:exclamation: No tests found!
Errors:
Filtered log (click to expand)

script returned exit code 1

:information_source:  This is an automatic message

:x: Build failed in 2h 24m 55s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core

:exclamation: No tests found!
Errors:
Filtered log (click to expand)

script returned exit code 1

:information_source:  This is an automatic message