[SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

What changes were proposed in this pull request?
This updates Parquet to 1.10.0 and updates the vectorized path for buffer management changes. Parquet 1.10.0 uses ByteBufferInputStream instead of byte arrays in encoders. This allows Parquet to break allocations into smaller chunks that are better for garbage collection.
How was this patch tested?
Existing Parquet tests. Running in production at Netflix for about 3 months.

Test build #89362 has started for PR 21070 at commit 4df17a6.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2327/
Test PASSed.

Test build #89362 has finished for PR 21070 at commit 4df17a6.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89362/
Test FAILed.

Could you share the performance number?

@rdblue, BTW mind fixing the title to [SPARK-23972][...] ...? It's actually written in the guide.

Test build #89408 has started for PR 21070 at commit 8446b40.

Test build #89408 has finished for PR 21070 at commit 8446b40.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89408/
Test FAILed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2353/
Test PASSed.

Merged build finished. Test PASSed.

Test build #89409 has started for PR 21070 at commit d724811.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2354/
Test PASSed.

Test build #89409 has finished for PR 21070 at commit d724811.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89409/
Test FAILed.

Test build #89412 has started for PR 21070 at commit 839cf28.

Test build #89415 has started for PR 21070 at commit 27a66d8.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2359/
Test PASSed.

Test build #89415 has finished for PR 21070 at commit 27a66d8.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89415/
Test FAILed.

This PR should include changes to ParquetOptions.scala to expose the LZ4, ZSTD and BROTLI parquet compression codecs, or else spark users won't be able to leverage those parquet 1.10.0 features.

@scottcarey I agree that's important. Perhaps it could be done as a follow-up PR?

I tested this with the addition of some changes to ParquetOptions.scala, but this alone does not allow for writing or reading zstd compressed parquet files, because it is using reflection to acquire hadoop classes for compression which are not in the supplied dependencies.
From what I can see, anyone that wants to use the new compression codecs are going to have to build their own custom version of spark.  And probably with modified versions of hadoop libraries as well, including changing how the native bindings are built.... because that would be easier than updating the whole thing to hadoop-common 3.0 where the required compressors exist.
Alternatively, spark+parquet should avoid the hadoop dependencies like the plague for compression / decompression.  They bring in a steaming heap of dependencies and possible library conflicts and users often have versions (or CDH versions) that don't exactly match.
In my mind, parquet should handle the compression itself, or with a light-weight dependency.
Perhaps it can use either the hadoop flavor, or if that is not found, another one, or even a user-supplied one so that it works stand-alone or from inside hadoop without issue.
Right now it is bound together with reflection and an awkward stack of brittle dependencies with no escape hatch.
Or am I missing something here, and it is possible to read/write with the new codecs if I configure it differently?

Test build #89531 has started for PR 21070 at commit 5a78030.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2453/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2455/
Test PASSed.

@rdblue
The problem with zstd is that it is only in Hadoop 3.0, and dropping that jar in breaks things as it is a major release.  Extracting out only the ZStandardCodec from that and recompiling to a 2.x release does not work either, because it depends on some low level hadoop native library management to load the native library (it does not appear to use  https://github.com/luben/zstd-jni).
The alternative is to write a custom ZStandardCodec implementation that uses luben:zstd-jni
Furthermore, if you add a o.a.h.io.codecs.ZStandardCodec class to a jar on the client side, it is still not found -- my guess is there is some classloader isolation between client code and spark itself and spark itself is what needs to find the class.  So one has to have it installed inside of the spark distribution.
I may take you up on fixing the compression codec dependency mess in a couple months.  The hardest part will be lining up the configuration options with what users already expect -- the raw codecs aren't that hard to do.

I backported the Hadoop zstd codec to 2.7.3 without much trouble. But either way, I think that's a separate concern. This is about getting Parquet updated, not about worrying whether users can easily add compression implementations to their classpath.

This is about getting Parquet updated, not about worrying whether users can easily add compression implementations to their classpath.

Yes, of course.
My hunch is that someone else will read the release notes that spark 2.3.0 supports zstandard, and parquet 1.10.0 supports zstandard, then realize it doesn't work in combination and end up here.  So I feel that this is the right place to discuss the state of these features until there is another more specific place to do so.
The discussion here has been useful to get closer to understanding what further tasks there may be.  If there are any follow-on issues, the discussion can move there.
I would love to be able to test this with my full use case, and give it a big thumbs up if it works.  Unfortunately my only motivation for this upgrade is access to ZStandard, and I'm not as excited to say 'works for me if I don't use new parquet codecs'.

Test build #89531 has finished for PR 21070 at commit 5a78030.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89531/
Test FAILed.

Test build #89533 has finished for PR 21070 at commit 5fca3ce.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89533/
Test FAILed.

Test build #89600 has started for PR 21070 at commit 40a9cdd.

Test build #89600 has finished for PR 21070 at commit 40a9cdd.

This patch fails from timeout after a configured wait of `300m`.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89600/
Test FAILed.

Retest this please.

Test build #89644 has started for PR 21070 at commit 40a9cdd.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2539/
Test PASSed.

Test build #89644 has finished for PR 21070 at commit 40a9cdd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89644/
Test PASSed.

Can we run a TPCDS and show that this upgrade doesn't cause performance regression in Spark? I can see that this new version doesn't have perf regression at parquet side, just want to be sure the Spark parquet integration is also OK.

@cloud-fan, is there a Jenkins job to run it?

No. But there are a bunch of open source projects to help you easily run TPCDS on Spark with parquet files, e.g. https://github.com/maropu/spark-tpcds-datagen

Okay, I don't have the time to set up and run benchmarks without a stronger case for this causing a regression (given the Parquet testing), but other people should feel free to pick this up.

@cloud-fan since there’s probably quite some time before this lands in a release, what do you think about merging this now if it’s ready, and filing the perf jira as a blocker against 2.4? My guess is that Spark will want to move to 1.10 at some point no matter what, so doing it this way has the advantage of giving plenty of time for any other issues to shake out before the next release.

Based on the previous upgrade (e.g., https://github.com/apache/spark/pull/13280#issuecomment-223022415), we hit the performance regressions when we upgrade Parquet and we did the revert at the end. That might be the major reason why we have not upgraded to 1.9 yet.
Jumping to 1.10.0 from 1.8.2 is big and risky especially when it was recently released. Even if we merge it now, we might still revert it if we hit ANY performance regression. That is why we prefer to do the performance testing before we do the merge.
Not sure whether the community can help the performance tests? cc @maropu You did many performance tests in TPC-DS in the past release. Could you help this too? If possible, we also need to do the micro-benchmark like what @dongjoon-hyun did for ORC.

cc @liancheng @yhuai @rxin

Ok, thanks for the context. I've worked on other projects where it's sometimes ok to take a small risk of a perf hit on trunk as long as the community is committed to addressing the issues before the next release. If the norms here are not to absorb any risk on trunk, that also seems reasonable.

I'll check performance changes w/ this pr in days and put the result in https://issues.apache.org/jira/browse/SPARK-24070

Parquet is the default file format. Thus, the impact could be huge. We need to be more cautious. Since Parquet 1.10 was officially released this month. Thus, we might need more careful review for such a big version upgrade.
@rdblue Could you summarize the major benefits brought by this upgrade from 1.8 to 1.10?

There are two main reasons to update. First, the problem behind SPARK-17213 is fixed, hence the new min and max fields. Second, this updates the internal byte array management, which is needed for page skipping in the next few versions.

Regarding SPARK-17213, we already enable the pushdown after we upgrading to 1.8.2. Is my understanding right?

The fix for PARQUET-686 was to suppress min/max stats. It is safe to push filters, but those filters can't be used without the stats. 1.10.0 has the correct stats and can use those filters.

@gatorsmile @rdblue I checked the numbers and I found no regression at least in TPC-DS: to check the actual numbers, see https://issues.apache.org/jira/browse/SPARK-24070.

Thank you @maropu!
What resources does the run require? Is it something we could create a Jenkins job to run?

I just run TPCDSQueryBenchmark in an ec2 instance (m4.2xlarge). The job to check performance regression is too heavy, so we just check if tpcds, tpch, and ssb can be correctly compiled in TPCDSQuerySuite, TPCHQuerySuite, and SSBQuerySuite.
BTW, I sometime checks the regression in my repo by myself: https://github.com/maropu/spark-tpcds-datagen.

ok, I'll work on it.

@iddoav Thank you for your feedbacks. It sounds like CDH5.X's spark has a bug, right? Our Apache Spark does not have such an issue based on my understanding.
@rdblue Does that means we can see a better perf number after upgrading to 1.10.0, since it has the correct stats and can use these stats for string filter pushdown?

Yes, it is safe to use push-down for string columns in data written with this version.

@maropu Could you help verify the perf improvement in this string filter pushdown in the microbenchmark? You can also refer to another microbenchmark we did in ORC. https://github.com/apache/spark/blob/master/sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala
If possible, please make a new microbenchmark suite that is extensible to all the data sources.

Is it necessary to block this commit on benchmarks? We know that it doesn't degrade performance from the Parquet benchmarks and TPC-DS run. What do you think needs to be done before this can move forward?

TPC-DS is just an end-to-end perf testing. To ensure no perf regressions, we need micro-benchmark tests. That is just a normal procedure. We  did the same in the ORC upgrade too.

@rdblue Do you see any risk of additional overhead coming from the extra stats? For example, if the data contains very long strings, performing comparison on them to generate stats will be expensive and might not be worth it in certain scenarios.

@mswit-databricks, I wouldn't worry about that. We've limited the length of binary and string fields.
In the next version of Parquet, we're planning on releasing page indexes, which are lower and upper bounds instead of min and max values. That gives us more flexibility to shorten values and avoid the case that you're worried about.

I'll put the benchmark results (the benchmark's currently running) today, just a sec.

Simple scan benchmarks:
code: https://github.com/apache/spark/compare/master...maropu:DataSourceReadBenchmark
master: https://gist.github.com/maropu/a767d21ed1dd047ec2bdca92915dc5c5
this pr: https://gist.github.com/maropu/b9312488ec791caec9711ba64027e3e2
Pushdown benchmarks:
code: https://github.com/apache/spark/compare/master...maropu:UpdateParquetBenchmark
this pr: https://gist.github.com/maropu/e7cbfb8dc73a91806088e159f64e113f
Obviously, the pushdown for strings didn't work and I don't look into why now.
Can you check this? @rdblue

@maropu, are you sure about the INT and FLOAT columns? I think you might have that assessment backwards. Here's the INT results from the PR gist:
SQL Single INT Column Scan:              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
SQL Parquet Vectorized                         149 /  162        105.5           9.5       1.0X
SQL Parquet MR                                1825 / 1836          8.6         116.1       0.1X

And here are the INT results from the master gist:
SQL Single INT Column Scan:              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
SQL Parquet Vectorized                         250 /  292         63.0          15.9       1.0X
SQL Parquet MR                                3175 / 3202          5.0         201.8       0.1X

I think that shows that the PR result was significantly faster, not slower. (The other INT test was about the same.)
Here's the FLOAT column from the PR gist:
SQL Single FLOAT Column Scan:            Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
SQL Parquet Vectorized                         145 /  158        108.8           9.2       1.0X
SQL Parquet MR                                1840 / 1843          8.5         117.0       0.1X

And FLOAT from the master gist:
SQL Single FLOAT Column Scan:            Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
SQL Parquet Vectorized                         261 /  316         60.2          16.6       1.0X
SQL Parquet MR                                3267 / 3284          4.8         207.7       0.1X

Am I reading this incorrectly? I'm considering lower time values and higher rate values to be better.

@maropu, looking at the pushdown benchmark, it looks like ORC and Parquet either both benefit or both do not benefit from pushdown. In some cases ORC is much faster, which is due to the fact that ORC will skip reading pages, not just row groups. But, when ORC benefits from pushdown so does Parquet, for example the Select 1 int row (value = 7864320) case.
I think that you were expecting a string comparison case to have a significant benefit over non-pushdown. But I would only expect that if ORC had a similar benefit. That's because this is dependent on the clustering of values in the file so that Parquet can eliminate row groups. If ORC didn't have a benefit, then I would expect that the data just isn't clustered in a way that helps.
I'm not sure how you're generating data, but I'd recommend adding a sorted column case with enough data to create multiple row groups (or stripes for ORC). That would write data so that you can ignore some row groups and you should see a speed up.
Parquet also supports dictionary-based row group filtering. To test that, make sure you have a column that is entirely dictionary-encoded: pick a small set of values and randomly draw from that set. Then if you search for a value that isn't in that set you should see a speedup. Also make sure that you have parquet.filter.dictionary.enabled=true set in the Hadoop configuration so that Parquet uses dictionary filtering.

@rdblue Aha, thanks for the explanation!

I think that you were expecting a string comparison case to have a significant benefit over
non->pushdown. But I would only expect that if ORC had a similar benefit.
That's because this is dependent on the clustering of values in the file so that Parquet can
eliminate row groups. If ORC didn't have a benefit, then I would expect that the data just
isn't clustered in a way that helps.

The string case had the same test data set (monotonically-increasing ids) with the int case, but we didn't get the benefit of push-down only in the string case. Is the logic to eliminate row groups different between int cases and string cases in spite that we use the same dataset?
https://github.com/maropu/spark/blob/465aa420b1399aba7199aa2868ad6ae58d877d50/sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala#L70

@maropu, I suspect that the problem is that comparison is different for strings: "17297598712" is less than "5" with string comparison.

aha, I see and it might to be true. I'll check the benchmark again.

@rdblue I made the input test date sorted (clustered) though, I didn't get a luck in the string case;
https://gist.github.com/maropu/b7ff6d528547de03a4a5cbabd120e78d
I'm still looking into why (I welcome any suggestion).

@maropu, I'd recommend looking at the Parquet files using parquet-cli to see if you're getting reasonable min/max stats for your string columns. That will give you an idea of when predicate push-down will help.

ok, give me more time to do so.

ok, I got the reason why;

$ ./bin/spark-shell --master=local[1]

scala> import scala.util.Random
scala> :paste
def timer[R](f: => {}): Unit = {
  val count = 5
  val iters = (0 until count).map { i =>
    val t0 = System.nanoTime()
    f
    val t1 = System.nanoTime()
    val elapsed = t1 - t0 + 0.0
    println(s"#$i: ${elapsed /<PHONE_NUMBER>.0}")
    elapsed
  }
  println("Avg. Elapsed Time: " + ((iters.sum / count) /<PHONE_NUMBER>.0) + "s")
}
var dir = "/home/ec2-user/parquet-test-string"
val numRows = 1024 * 1024 * 15
val width = 6
val selectExpr = (1 to width).map(i => s"CAST(value AS STRING) c$i")
val valueCol = monotonically_increasing_id().cast("string")
val df = spark.range(numRows).map(_ => Random.nextLong).selectExpr(selectExpr: _*).withColumn("value", valueCol).sort("value")
df.write.mode("overwrite").parquet(dir)
spark.read.parquet(dir).createOrReplaceTempView("parquetTable")

scala> sql("SET spark.sql.parquet.filterPushdown=true")
scala> timer { sql("select * from parquetTable where value = '0'").collect }
#0: 1.041495043
#1: 0.53017502
#2: 0.468505896
#3: 0.437655119
#4: 0.429151435
Avg. Elapsed Time: 0.5813965026s

scala> sql("select * from parquetTable where value = 0").explain
== Physical Plan ==
*(1) Project [c1#35, c2#36, c3#37, c4#38, c5#39, c6#40, value#41]
+- *(1) Filter (isnotnull(value#41) && (cast(value#41 as int) = 0))
   +- *(1) FileScan parquet [c1#35,c2#36,c3#37,c4#38,c5#39,c6#40,value#41] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/home/ec2-user/parquet-test-string], PartitionFilters: [], PushedFilters: [IsNotNull(value)], ReadSchema: struct<c1:string,c2:string,c3:string,c4:string,c5:string,c6:string,value:string>

scala> timer { sql("select * from parquetTable where value = 0").collect }
#0: 10.656159769
#1: 10.583965537
#2: 10.486018192
#3: 10.475532898
#4: 10.494059857
Avg. Elapsed Time: 10.539147250600001s

scala> sql("select * from parquetTable where value = '0'").explain
== Physical Plan ==
*(1) Project [c1#35, c2#36, c3#37, c4#38, c5#39, c6#40, value#41]
+- *(1) Filter (isnotnull(value#41) && (value#41 = 0))
   +- *(1) FileScan parquet [c1#35,c2#36,c3#37,c4#38,c5#39,c6#40,value#41] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/home/ec2-user/parquet-test-string], PartitionFilters: [], PushedFilters: [IsNotNull(value), EqualTo(value,0)], ReadSchema: struct<c1:string,c2:string,c3:string,c4:string,c5:string,c6:string,value:string>

We do push down the predicate value = '0' into the parquet filter though, we don't push down the predicate value = 0. So, this is a spark-side issue for push-down handling.

I looked over the code and I fond this issue could be fix by;
https://github.com/apache/spark/compare/master...maropu:FixFilterPushdown
scala> sql("select * from parquetTable where value = 0").explain
== Physical Plan ==
*(1) Project [c1#35, c2#36, c3#37, c4#38, c5#39, c6#40, value#41]
+- *(1) Filter (isnotnull(value#41) && (cast(value#41 as int) = 0))
   +- *(1) FileScan parquet [c1#35,c2#36,c3#37,c4#38,c5#39,c6#40,value#41] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/Users/maropu/Desktop/parquet-test], PartitionFilters: [], PushedFilters: [IsNotNull(value), EqualTo(value,0)], ReadSchema: struct<c1:string,c2:string,c3:string,c4:string,c5:string,c6:string,value:string>

But, I don't know this is a known issue and this fix is a right approach. cc: @HyukjinKwon

ok, I checked string pushdown worked well;
https://gist.github.com/maropu/e7cbfb8dc73a91806088e159f64e113f

Yup, https://github.com/apache/spark/pull/21070#issuecomment-386793202, just for clarification, the given attribute and literal is castable but they are not being as so, right?
I believe this is a known issue and there were several tries:
One approach was directly casting always and it was reverted (roughly 2 years ago?). Another approach was constant folding at optimizer level but it was rejected as it's too messy. Another approach was directly casting and comparing both values but it was also rejected since it sounded unsafe.
It was a long old story so probably worth double checking the history but I feel sure that I remember this story correctly.
The key point IIUC was that translateFilter should be super conservative and sounds we better need to check every possibility.

@liancheng should also remember all the story too since I talked with him at that time IIRC.

@HyukjinKwon Thanks for the explanation! I feel it's ok to keep this as it it for now. btw, we don't need to document the story somewhere?

For documentation, I think it's fine. it's a trackable history in JIRA although it's a bit hard to search .. Also, there might be some diff we should check since I haven't checked the details of JIRAs and if it's still true.

sql("select * from parquetTable where value = '0'")
sql("select * from parquetTable where value = 0")

Semantically, they are different. The results should be also different. The plan shows whether these predicates are pushed down or not.
Without the upgrade to 1.10.0, we can't get the benefit from the predicate pushdown between the above two queries. I also double checked it in my local environment. Now, it sounds like it is good reason to upgrade the version, since string predicates are pretty common.
@rdblue @maropu Thank you for your great efforts and patience! This upgrade is definitely good to have. We should try to make it in the next release.
@maropu Could you also submit a separate PR for your micro benchmark? Thanks!
cc @cloud-fan @hvanhovell @mswit-databricks Do you have any comment about the implementation of this PR?

@gatorsmile ok, I'll do later.

Test build #90327 has started for PR 21070 at commit 6c9d47b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3007/
Test PASSed.

Test build #90327 has finished for PR 21070 at commit 6c9d47b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90327/
Test PASSed.

LGTM except 2 comments

(While I am here) seems fine to me otherwise.

Test build #90375 has started for PR 21070 at commit 95ecde0.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3045/
Test PASSed.

Test build #90375 has finished for PR 21070 at commit 95ecde0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90375/
Test PASSed.

thanks, merging to master!


Make Taskman service validate Jira token

This PT add Jira token validation for Taskman service.
Closes OSIDB-2203.

Note for the future:
I've submitted a new commit with the cassettes rewritten as I mention in some dailies that were only possible after OSIDB-3029.
There is no changes in code (only some IDs in tests because of cassettes) since the last review you guys did so I'm not requesting a new review.


Please update DRAKE_SUGGESTED_VERSION to v1.27.0

https://drake.mit.edu/release_notes/v1.27.0.html
https://github.com/RobotLocomotion/drake/releases/tag/v1.27.0

https://github.com/RobotLocomotion/drake-ros/pull/340

Duplicate of #342 now, closing this one.

https://github.com/RobotLocomotion/drake-ros/pull/340


Issue if user selects no radar

After launching the app from the command line, if a user selects no radar(s) and immediately clicks the Download/Execute button, sa.radar_list will be empty and thus the for loop is never entered. The results variable is not defined and the app will throw an error.
The fix should either be to (1) make it impossible for the user to click the Download/Execute button without selecting a radar, or  (2) to check that len(sa.radar_list) > 0 above res = call_function(query_radar_files).  (2) is probably a bit easier to implement.

Fixed with this commit


Separate backend and frontend directories

Is your feature request related to a problem? Please describe.
Currently, the backend and frontend directories are coupled together. As you see, the package.json file belonging to the backend is in the root. I think separating them would be a good option for deploying them in the cloud and scaling in the future.
Also, It would be good if the changes were updated in the docs (README.md).

i would love to contribute in this kindly assign it to me

Could you brief your approach?

Frontend and backend are separated already !

i think the front end and backend are already sepearated please cloase the issue


Skip e2e tests on forks

Change-type: patch
Signed-off-by: Kyle Harding<EMAIL_ADDRESS>

I self-certify!

Created draft PR: https://github.com/balenaltd/handbook/pull/73

Created draft PR: https://github.com/balena-io/docs/pull/2508

Created draft PR: https://github.com/product-os/environment-staging/pull/478

Created draft PR: https://github.com/balena-io/balena-api/pull/4086

Created draft PR: https://github.com/balena-os/balena-supervisor/pull/2117

Created draft PR: https://github.com/balena-io/environment-staging/pull/1092

Created draft PR: https://github.com/balena-io/balena-cli/pull/2592

I self-certify!

Created draft PR: https://github.com/balena-os/fatrw/pull/49

Created draft PR: https://github.com/balena-os/balena-engine/pull/337

I self-certify!


Added safe environment

I have added a safe environment conda recipe (environment-safe.yml) and its associated CI test - this contains a frozen environment that is easily solved by conda and can be used everytime we have major inter-dependency issues for a short time, rather than have a completely broken environment; this will, of course, become obsolete in a while, since all the deps are pinned to (at the moment) the latest versions as of August 2019, but it's a life boat in case of emergency and if we keep updating it once say every six months, we should be fine.
see issue #1239 for a typical issue with versioning via conda that completely breaks the cdo use

closes #1239

The referenced issue is already closed.
@valeriupredoi is this still relevant?

@valeriupredoi This looks a bit outdated, is it still relevant?

superseded by modern implementation of conda lock #2193


[FEATURE] Support $NO_COLOR environment variable having an effect.

Is your feature request related to a problem? Please describe.
I use a programming language named Samarium and run it sometimes in a web environment. It would be useful if I could disable the colouring directly from Dahlia as ansi escapes don't show up well on the web.
I suggest a way to disable colouring.
Describe the solution you'd like
Detection of if the environment variable $NO_COLOR is set. If it is, all text should be returned without formatting.
Describe alternatives you've considered
Currently I just use regex to remove it but that is not convenient and slow.
Additional context
https://no-color.org/

Added in 0.2.0


[GOBBLIN-1450] Update test framework version for latest IntelliJ

The recent version of IntelliJ are not fully compatible with old
versions of testng and junit that we use. As a result IDE does not show
test names in the UI correctly.
New version of test frameworks should solve this problem.
We also fix a couple of misconfigurations for TestNG. When the class
is annotated with "@test", TestNG will run all public methods as tests.
We convert a couple of helper methods to private to fix that.
https://issues.apache.org/jira/browse/GOBBLIN-1450

@autumnust , can you take a look at merge?

Codecov Report

Merging #3288 (b81db01) into master (569cdb7) will decrease coverage by 0.07%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #3288      +/-   ##
============================================
- Coverage     46.53%   46.45%   -0.08%
+ Complexity    10028    10020       -8
============================================
  Files          2037     2037
  Lines         79271    79277       +6
  Branches       8839     8841       +2
============================================
- Hits          36886    36829      -57
- Misses        38963    39030      +67
+ Partials       3422     3418       -4

Impacted Files
Coverage Δ

.../metrics/reporter/KeyValueEventObjectReporter.java
0.00% <0.00%> (-60.79%)
:arrow_down:

...metrics/reporter/KeyValueMetricObjectReporter.java
0.00% <0.00%> (-55.56%)
:arrow_down:

.../org/apache/gobblin/metrics/kafka/PusherUtils.java
0.00% <0.00%> (-40.00%)
:arrow_down:

...blin/metrics/reporter/util/KafkaReporterUtils.java
21.87% <0.00%> (-28.13%)
:arrow_down:

...gobblin/iceberg/publisher/GobblinMCEPublisher.java
66.66% <0.00%> (-1.69%)
:arrow_down:

...lin/elasticsearch/writer/FutureCallbackHolder.java
61.42% <0.00%> (-1.43%)
:arrow_down:

...kunit/packer/KafkaTopicGroupingWorkUnitPacker.java
40.65% <0.00%> (-0.34%)
:arrow_down:

.../gobblin/iceberg/writer/IcebergMetadataWriter.java
68.93% <0.00%> (+1.36%)
:arrow_up:

...lin/source/extractor/extract/kafka/KafkaUtils.java
61.11% <0.00%> (+2.28%)
:arrow_up:

...lin/util/filesystem/FileSystemInstrumentation.java
100.00% <0.00%> (+7.14%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 569cdb7...b81db01. Read the comment docs.


Connection hangs if bad SQL query

If you send it invalid SQL, the prqld never responds until it finally hits a panic error.

This isn't happening against a local postgres server

Not happening against local mysql either.


ch02: FileIO - Infinite loop reading csv

back-link: ch02/index.html#console-application
QString.isEmpty() returns true both for cases where it is an empty string value AND when it is null, so performing this check first and using continue will loop forever
This:
    if(line.isEmpty()) {
        continue;
    }
    if(line.isNull()) {
        break;
    }

Should be changed to
    if(line.isNull()) {
        break;
    }
    if(line.isEmpty()) {
        continue;
    }

Hmm. Correct. Did not test it but sounds reasonable. Seems I mixed them up


(docs): update rc.0 guide to note tsconfig entry change

When the tsconfig file names are changed using this guide we also need to update the old app.tsconfig entry to reflect the new file names

sorry, this is already reflected later on in the guide.


Vectorize Rejection Sampling

Improve the performance of Rejection Sampling
I have not benchmark this code, this code is refactored and cleaned up from my previous implementation of Kyber.
It's seem like we need to switch to CircleCI to have ARM-based runner.

[ ] Does this PR change the input/output behaviour of a cryptographic algorithm (i.e., does it change known answer test values)?  (If so, a version bump will be required from x.y.z to x.(y+1).0.)

I tried to rewrite the sampler to meet the layout of Keccak.
It's seem like we can't use the layout friendly to Keccak, this make the loading from sampling very difficult.
The sample will need 3 bytes, to produce 2 coefficients.
If we layout like this:
0 - 1 - 2 -3 - 4 - 5 -6 - 7 | 8 - 9 - a - b - c - d - e - f.
With 0 -> to 7 belongs to keccak_way_1 and 8->f belongs keccak_way_2, then it's not possible to align the byte we need:
Bytes: 0, 1, 2, 3, 4, 5 create 4 coeffcients. Missing one byte added to 6, 7 to complete the sampling.
Similarly, keccak_way_2 also needs one additional byte to complete the sampling.
Thus, the memory layout of Keccak must be All data of keccak_way_1 || All data of keccak_way_2 || ... The interleave will make the sampler more difficult.

We need ARM-runner so build_test can run.  #34

We can just cross compile and emulate using qemu.
See for example here: https://github.com/slothy-optimizer/pqax/blob/main/.github/workflows/tests.yaml.
Do you mind setting that up? We won't have Arm runners any time soon.

I have a tight schedule this week. Perhaps next week I'll set it up.

@cothan @mkannwischer This PR reminds me that we should discuss intrinsics vs. ASM.
As hinted at in the README, my preference is to write 'clean' ASM and automatically micro-optimize it, e.g. using SLOTHY. The main reasons are: (1) Assurance: There is more tooling for the verification of ASM than Intrinsics, at least to my knowledge. Moreover, verification can be split into separate verification of the clean code and the micro-optimizations. (2) Performance: Past experience shows that the best implementations are obtained by writing ASM by hand. Moreover, by splitting development into manual writing of clean code and automatic micro-optimization, development time should not be much higher than when writing intrinsics (This may be false in general if there are a lot of stack spills, but this is not the case for MLKEM).
I would therefore propose that we do not use intrinsics but clean ASM throughout, including this PR. @cothan Would you be willing to port your intrinsics code to ASM? Might be a useful learning experience as well.
Thoughts?

Hi @hanno-becker ,
Good idea. Originally this function was implemented in Intrinsic since it's quicker for me to debug in my VSCode. I Don't mind writing ASM at all. I will port this implementation to ASM. Do we have a Python template to generate ASM that I can take look at?

@cothan I tend to write ASM by hand, but making extensive use of .req and .macro directives to keep things modular and readable. We have plenty of examples in the SLOTHY repo.

Update: I have my ASM working and will update this PR very soon!

Awesome @cothan !

Oh, with macos-latest I thought it will run on arm64. But it's not.


DeprecationWarning: the imp module is deprecated

Looks like the imp module is deprecated in favour of importlib

Hi there,
Thanks for reporting this. Under what circumstances did you receive the deprecation message? Need to replace imp, but also need to remain compatible with older Pythons, so it's not a cutpaste switchover.

This is now on the master branch and will make it into the next release. To be updated when a new release is made, subscribe to https://www.freelists.org/list/mitogen-announce
Thanks for reporting this!


Check for messages over max packet size limit and throw an exception with the message

Currently such messages trigger a BufferOverflowException (see issue #23) which an error handler could handle, but it wouldn't be given the contents of the message itself. I'm proposing to propagate that on for a consumer to do with as they wish.

Build failed due to what looks like a timing issue in an existing test. Proposed fix in #72.

@schilling I'll take a look and review this one, thanks again for your contributions. I merged in #72, could you merge that in and confirm the CI passes?

@schilling I'll take a look and review this one, thanks again for your contributions. I merged in #72, could you merge that into this and confirm the CI passes?

Merged in and the CI passed nicely.

Is there a timeline for inclusion and a publish of the library? It doesn't affect functionality our side at present, but I'd be glad to have an idea of when I should be able to swap over to the upstream library completely.

@schilling sorry for the delay in updating this thread. We should be able to publish a new release sometime in May (we're in the process of changing ownership of that repo team wise). We'll update you once it's done, thanks again for working on that.

No worries. Thank you very much for the update.


[Enhancement] add thread monitor for vCPU thread

report inside display_loop per second about stats of monitored vCPU threads and change CPU affinity & sched policy when triggered by a constraint that thread shall not run on a E-core with higher than 50% cpu usage.

using fn::usched_loop to monitor every thread which has been add to the pool.

using fn::addThreadMonitor to pass qemu pid and add all vCPU threads into the pool.

mutex lock not implemented.

UPDATE:
1. report inside display_loop per second about stats of monitored vCPU threads and change CPU affinity when triggered by a constraint that thread shall not run on a E-core with higher than the pre-set cpu usage(defined by THRESHOLD).
2. using usched_loop to monitor every thread which has been added to the
pool.
3. using add_to_monitor_pool to pass qemu-pid and vCPU threads
into the pool.
4. mutex lock implemented.

UPDATE：

rewrite function updatePID_INFO() to upsert_to_monitor_pool().
add safety checks.
rename variables and functions style.


SARIF Report structure Validation Failing

TL;DR
Current SARIF report is not compliant with the SARIF format. Current report has empty values for some of the key parameters.
SARIF report structure which includes not just the keys, but values should comply with SARIF structure.
This tool SARIF_Validator can help in validating the same.
Expected behavior
informationUri should be non empty.
In general SARIF report should be validated against SARIF_Validator
Observed behavior
"informationUri": ""
informationUri is empty and the generated report is non compliant with SARIF_Validator
Action YAML
N/A

Log output
No response
Additional information
No response

The above bug was closed in this commit : https://github.com/google-github-actions/analyze-code-security-scc/pull/36


Add support for measurements (new feature)

Pretty standard for 3D applications are Lines, Circles, Angles.

Yes that would be useful I agree - are you interested in submitting a PR?


Unable to download VS code extension on my SAW

Version: 1.53.2 (system setup)
Commit: 622cb03f7e070a9670c94bae1a45d78d7181fbd4
Date: 2021-02-11T11:48:04.245Z
Electron: 11.2.1
Chrome: 87.0.4280.141
Node.js: 12.18.3
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.18363
Steps to Reproduce:
While trying to install Azure functions extension on VS code on my SAW I am getting a failure message. When I try to download extension manually it is consistently throwing following error. Is there a way to make it work?
{“$id”:“1”,“innerException”:null,“message”:“Request was blocked due to exceeding usage of resource ‘Count’ in namespace ‘AnonymousId’. For more information on why your request was blocked, see the topic “Rate limits” on the Microsoft Web site (https://go.microsoft.com/fwlink/?LinkId=823950).”,“typeName”:“Microsoft.TeamFoundation.Framework.Server.RequestBlockedException, Microsoft.TeamFoundation.Framework.Server”,“typeKey”:“RequestBlockedException”,“errorCode”:0,“eventId”:3000}

Does this issue occur when all extensions are disabled?: Didn't check

This should be addressed in later releases.


RC.18-f8a68cc messes up styles for IE9 again

I've added quite a few fixes for CSS to support IE9 in version RC.16-9c495da. Tried to update to RC.18-f8a68cc only to see that the styling is all messed up and even my previous fixes do not really do their job anymore. Scrollbars are out of place in random spots on a page, headers are not visible, etc.
Do you guys incorporate or take into account fixes posted here or do you only accept fixes done on the actual codebase?
I've reverted for the time being, but once I get back to this (or later) version I will post my CSS fixes here for others to use.

Alexandery, I migrated to RC 18 and your changes are still working for me.

@alexandry: unless you submit a PR and we merge it, then it isn't in the code base, unless someone else picks up your changes and makes them into a PR.  We do pick things up and make PRs out of them, but for me I don't have IE to be able to test, so I can't easily do that.

@PaulL1 Got it. Never done PRs. I'll see if I can help with the fixes there sooner rather than later. For now, I guess, we'll have to deal with CSS overrides.

@mdsauer Interesting. Wasn't working for me. But I am also using a pagination, which is messing up styles in RC-18. Maybe that was the reason. I just didn't have any free time to figure it out and fix it.

There's a reasonable guide on PRs in https://github.com/angular-ui/ng-grid/blob/master/FIRST_TIMER.md

Can we get a screenshot of the styling issues and a short write-up of what's expected?

@c0bra At this point I think it'd be fare to say that this item should be closed. I've done a lot of code and style changes to make sure the grid works and gets displayed properly on my screens, so I cannot really provide you with screenshots of it's incorrect behavior.

Understood. I'll go ahead and close.


Unify Tuple and Tree types

At the moment each data leaf in the tree is a "tuple" of atomic values ([WireValue]). This makes the top level of those nodes special, in that they can be a heterogeneous collection of data, but everything nested underneath may not be.
In itself this may or may not be a problem. However, the "correct" solution feels like we should leverage the more recent pairs functionality that I added (or more general atom-level tuples, or even algebraic data types if we can) to make the leaf collections we want. Then, we could use named types for both tree leaf nodes and as parts of other, more complex atom types.

Moved to https://gitlab.com/concert/clapi/issues/17


Fix internal link & let zh-tw docs match en docs in the range of "Getting Started"

Fix internal link that still having wrong prefix "docs/" in zh-tw docs.
Let zh-tw docs match en docs in the range of "Getting Started".

Thanks for the contribution!


Updated translation after ICT final pass on November 21

Updated translation after ICT final pass on November 21
Only fr, it, pt_BR tester updated minor translations.
CC @jrafanie

Closing this as it's now stale @Frank-NB did we get these changes in another PR? Sorry, I didn't see this one.

Yes, I think we have got these changes in recent PR,  it is OK to close here.


First debugging steps

@brunobowden ptal
@andrewfunston, let me know if this worked for your protobuf problem on #498

LGTM


Feature/drawer classes

<!-- Include a bulleted list summarizing the main changes you have made in this PR --
Changes proposed in this Pull Request:
- Add class overrides to Drawer

InfoListItem Classes PR needs to be merged before this can be completed; DrawerNavGroups needs to be able to pass class overrides.

I want to merge #52 in prior to this (if it's ready), since it'll probably be easier for you to handle merge conflicts than Huayun

I want to merge #52 in prior to this (if it's ready), since it'll probably be easier for you to handle merge conflicts than Huayun

Sounds good to me!  Work is wrapped up for this PR, until the merging happens.


Rebuild the dragdrop

Rebuild the dragdrop/improve the dragdrop
Maybe just indicate where drop will be?

@Thanood
Experimenting with auto hidden column, and have option in menu to drag it in

Looks like the basic hide/column chooser is working
Just need to clean up menu part before I push new version and test column html in menu and update docs

I love that.. 😃
Just an idea:
Could you restict the drag animation to the x-axis and move it along the header line?

Could you restrict the drag animation to the x-axis and move it along the header line?

@Thanood
But then you cant drag it from then menu, or from header to group panel.
Or did I miss something ?
Well anyways have a lot of clean up to do, and see if I can make it a little more stable/ optimise how I do some of the stuff, little bit messy atm.
Was thinking about just indicate where the drop will be like Ive seen in other grids, but looks like this is working

But then you cant drag it from then menu, or from header to group panel.

I mean only when adding a new column from the menu.


Add Publication download

We need to be able to download and convert publications from CDL elements outside of the a user download.
This is needed both as a better troubleshooting tool, and as a way to save all citation sources for a publication record.
The new functionality allows users to do cdl-elements publications <elements-pub numbers> .  These are downloaded to the cache directory, converted to standard jsonld and if either the global --data= or --tdb= parameters are specified (either via commandline or env variable) then they will also be added to that particular linked database.

Closed with 2c19df8c49d0ada99669693f0c50ceed5eaa041f


Intermittent derper disconnects when transferring large amounts of data

While investigating https://github.com/n0-computer/iroh/issues/1357 I did some manual derper only tests, both mac -> mac and linux -> linux.
When transferring a large amount of data, the derper drops connections to clients in regular intervals, leading to very slow transfers. Below is an example of a derper log.
It seems that the derper gets errors in the frame parsing code, causing it to drop the connection to the client. The connection will then be reestablished from the client side a few seconds later, and the process begins anew.
This could either be the client sending malformed packets and the derper correctly disconnecting them, or the derper being unable to parse some valid packets. In any case it slows down the transfer to a crawl.
2023-08-21T12:09:47.602154Z  WARN iroh_net::derp::client_conn: connection manager for PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814) 34: writer closed in error frame header size<PHONE_NUMBER> exceeds reader limit of 1048576
2023-08-21T12:09:47.602173Z  WARN iroh_net::derp::clients: Could not find client for PublicKey(e0cbae283a789a8b6f9bbe115954807d370ecde72eb8f0b47e3e879b0099df6d), dropping packet
2023-08-21T12:09:47.602178Z  WARN iroh_net::derp::clients: pruning connection PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814)
2023-08-21T12:09:47.633353Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:47.633368Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:47.633373Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:47.633377Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:47.633381Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:47.633385Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:47.738954Z  WARN iroh_net::derp::client_conn: connection manager for PublicKey(e0cbae283a789a8b6f9bbe115954807d370ecde72eb8f0b47e3e879b0099df6d) 29: writer closed in error frame header size<PHONE_NUMBER> exceeds reader limit of 1048576
2023-08-21T12:09:47.738973Z  WARN iroh_net::derp::clients: Could not find client for PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropping packet
2023-08-21T12:09:47.738978Z  WARN iroh_net::derp::clients: pruning connection PublicKey(e0cbae283a789a8b6f9bbe115954807d370ecde72eb8f0b47e3e879b0099df6d)
2023-08-21T12:09:48.226127Z  WARN iroh_net::derp::client_conn: connection manager for PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814) 35: writer closed in error frame header size<PHONE_NUMBER> exceeds reader limit of 1048576
2023-08-21T12:09:48.226151Z  WARN iroh_net::derp::clients: Could not find client for PublicKey(e0cbae283a789a8b6f9bbe115954807d370ecde72eb8f0b47e3e879b0099df6d), dropping packet
2023-08-21T12:09:48.226159Z  WARN iroh_net::derp::clients: pruning connection PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814)
2023-08-21T12:09:48.239733Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.239748Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.277510Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.277524Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.277530Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.277534Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.277538Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.277542Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.472031Z  WARN iroh_net::derp::client_conn: connection manager for PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814) 37: writer closed in error frame header
2023-08-21T12:09:48.472069Z  WARN iroh_net::derp::clients: Could not find client for PublicKey(e0cbae283a789a8b6f9bbe115954807d370ecde72eb8f0b47e3e879b0099df6d), dropping packet
2023-08-21T12:09:48.472081Z  WARN iroh_net::derp::clients: pruning connection PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814)
2023-08-21T12:09:48.518947Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet
2023-08-21T12:09:48.518962Z  WARN iroh_net::derp::server: send packet: no way to reach client PublicKey(874d9a50e9b3449bf06af6aceae81b558f8e6cd023bb419d743713e6d84b1814), dropped packet

Here are some example reasons for a disconnect.
2023-08-21T12:13:58.727053Z  WARN iroh_net::derp::client_conn: connection manager for PublicKey(6fb11ecdbe86b3f46fcd9e2fe751d0fda8a435aceb6a2b8e61cf7164afa6b95c) 2: writer closed in error frame header

2023-08-21T12:14:26.800152Z  WARN iroh_net::derp::client_conn: connection manager for PublicKey(6fb11ecdbe86b3f46fcd9e2fe751d0fda8a435aceb6a2b8e61cf7164afa6b95c) 3: writer closed in error frame header size<PHONE_NUMBER> exceeds reader limit of 1048576

PublicKey(6fb11ecdbe86b3f46fcd9e2fe751d0fda8a435aceb6a2b8e61cf7164afa6b95c) 4: writer closed in error frame header

PublicKey(6fb11ecdbe86b3f46fcd9e2fe751d0fda8a435aceb6a2b8e61cf7164afa6b95c) 5: writer closed in error frame header size 4758350 exceeds reader limit of 1048576

PublicKey(6fb11ecdbe86b3f46fcd9e2fe751d0fda8a435aceb6a2b8e61cf7164afa6b95c) 6: writer closed in error frame header size<PHONE_NUMBER> exceeds reader limit of 1048576

PublicKey(e0cbae283a789a8b6f9bbe115954807d370ecde72eb8f0b47e3e879b0099df6d) 1: writer closed in error frame header size<PHONE_NUMBER> exceeds reader limit of 1048576

I think this is because of a bug in the frame reading code. See https://github.com/n0-computer/iroh/pull/1382 for a minimal fix.

https://github.com/n0-computer/iroh/pull/1386 is a fix that also makes the encode/decode nicer.

I did a manual test. It seems that the issue is gone with the tokio-codec PR.
Doing the manual test is a bit annoying since you have to do a code change to force derper only mode...


Enable GH Discussions

Please?

Why?

Why?

Because it's nice to have it.

@JaydenDev please do

@webdev03 is it really necessary? it's a website...

@webdev03 is it really necessary? it's a website...

YES!


ShinyProxy can be downloaded from https://www.shinyproxy.io/downloads/  It is probably best to start from https://www.shinyproxy.io/getting-started/


What are the recovery expectations of ktra

First: Thanks for working on this. I've been experimenting with it the last couple of days and it's great.
I don't know if this is an issue or simply expected behavior, so I thought I'd ask.
If I start ktra and then publish a few crates, my git repo is updated correctly and the various queries about which crates are available etc all work as expected.
Then: I delete my database (I'm using sled), index and my crates directory. (i.e. all contents created and managed by ktra).
Now, when I start ktra, it happily updates my index and seems to "do the right thing" in terms of populating that with the name of my crates from my git repo. I create a new user and get a token and then when I use that to ask ktra what crates are available I get an empty list.
I know deleting my database, etc... is a silly thing to do, but I sort of feel like it should be possible to recreate (most of) the contents of the ktra data based on the information that is stored in the git repo. So, that's my question really. Should it be possible?

Hi @garyanaplan, thank you for using Ktra.
To come to the point, and unfortunately, it is not possible to restore deleted database and crates data from the index git repository.
A metadata sent by cargo when you hit cargo publish command is similar to the index data but not the same.  The metadata can map onto the index data but not vice versa.
Additionally, the checksum of crate binary data is calculated when it is published so if you remove crates directory you cannot retrieve it too.
Feel free to write a comment if you have more questions 👍.

That's totally fine. I wanted to get a better idea of how things worked and you've made that very clear. Thank you!

@moriturus I have a follow up question.
What's the purpose of git index if it contains only a subset of data published to the database? It sounds redundant as it'd seem that database+crates tars (archives) should be enough but I'm surely missing something?

The git index is how cargo does the dependency resolution, see https://doc.rust-lang.org/cargo/reference/registries.html#index-format


cbonsai: init at 1.0.0

Motivation for this change
It's a fun program.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Looks like someone beat you to it :) https://github.com/NixOS/nixpkgs/commit/75078f712d5c2989549028ce8f13ebe8a8f78a6e

Too bad, it's duplicate of #115062


[Feature] Create JSON Rules Easily

I've created a package to create JSON rules easily.
https://www.npmjs.com/package/durable-rule-converter
This is an initial draft version.  This might help someone.

Thanks a lot for sharing.

This is cool. How does one do a negation in durable rules? Is there a NOT operator?


Time option not working

I have changed the hour but in demo mode the current hour is still shown (not even 7:00 or 6:00)...
I am using the pre-built apk from XDA-Developers, but I don't think they did changes to the code.
Everything else is working great, although an option to show "Priority mode" icon would be cool too.
Thanks

Hi @jahirfiquitiva,
Could you please provide more detail on the issue so we can look into it (i.e. device model name, Android OS version, steps to reproduce). Thanks!

Hello @davidschreiber
I opened the issue about 8 months ago ... I cannot remember the steps to reproduce and honestly don't even remember how I was using your app.
I'm going to close this issue since I cannot provide any real info.

For those coming here and having the same problem (I did), it is not a bug, the command only accepts 4 digits, without :, such as entering 0941, this will change the time to 9:41.


Just a simple question

Why in the earth the plugin is paid on spigot even tho its open source...

Please read this section of the wiki.
And it's not completely open source.

Alright makes sense, ty


LoremIpsumAnonymizer - Add customization options + documentation

1/ Add customization options
LoremIpsumAnonymizer takes for now these options:

'paragraphs': (int) number of paragraphs, default is 1,
'html': (bool) surround each paragraph with <p>, default is false.
'sample_count': (int) how many different values to use (default is 100).

It could be nice to have another option words (int):

If this option is given, the Anonymizer does not fill the anonymized value with a complete paragraph but only with words
The value of the option gives the number of words to generate
This option could not be used associated with paragraphs

2/ Documentation
LoremIpsumAnonymizer should be documented in docs/content/anonymization/core-anonymizers.md

done with #112


Re-enable Android tests via Browserstack

This is straight-forward now since we migrated our tests to Browserstack already.

This was fixed by #2955


Clean up clippy warnings and errors

Huge clean up for the clippy warnings and errors and enable the cargo fmt and cargo clippy checks on Travis CI.

By

https://rust-lang-nursery.github.io/rust-toolstate/
https://rust-lang.github.io/rustup-components-history/x86_64-apple-darwin.html

rustfmt is not available on Rust nightly today on Mac OSX

This should address #43


Popover: gutter - Accept string value

🚀 Feature request
Motivation
👋Hallo! First off, thank you for all for creating this magical thing that is Reakit! It's absolutely wonderful, and it's easily become one of my favourite libraries ❤.
The feature request idea I had to was to enable the gutter option for usePopoverState() to accept a string value.
The underlying library, popper.js, accepts a string:
https://popper.js.org/popper-documentation.html#modifiers..offset.offset
The use case was to enable x offsetting in addition to y.
I recently needed this when I needed to negatively offset a Reakit Menu to the left. I wasn't able to do this with useMenu(), so I had to use CSS negative margins.
Example
An example of this implementation would be something like:
const popover = usePopoverState({ gutter: '-10px 10px' })

When passing it to popper.js, it should translate to:
x: -10px
y: 10px
Possible implementations
To enable this change, we'd need to adjust the options being passed to Popper, here:
https://github.com/reakit/reakit/blob/master/packages/reakit/src/Popover/PopoverState.ts#L168
To maintain backwards compatibility, we'd need to adjust the value based on either the gutter was a string or number.
Lastly, we'll need to update the type.
Thank you so much for your time!

That sounds very reasonable to me!
@ItsJonQ Would you like to work on that?

Briefly looked and just keep in mind that the string option with values separated by spaces will be deprecated in v2 of popper.js. They recommend separate values using a comma, ie. -10px, 10px and not -10px 10px. From Reakit's point of view, it probably doesn't matter because it will just pass along either the number or string to popper.js under the hood.

Would you like to work on that?
@diegohaz Sure thing! I'd love to :)

They recommend separate values using a comma, ie. -10px, 10px and not -10px 10px
Oo! Good catch

Actually, in v2 we are going to accept only a tuple, with [mainAxis, altAxis] values.
here you find some documentation, but consider it may change since v2 is still under development.
https://github.com/popperjs/popper.js/blob/next/docs/src/pages/modifiers/offset.mdx

@FezVrasta Oh! Interesting. That seems like a better API. Thanks for the heads up!
P.S. I'm a big fan fan of Popper.js. Thank you for your amazing work 🎉

@FezVrasta Will v1 accept offset as a tuple before v2 is out? That would help with the migration I guess.
To be safer here, I'd suggest that we introduce a new unstable_ property instead of augmenting gutter, that is already stable:
usePopoverState({ unstable_offset: [-10, 10] });

I imagine the implementation being something like this for now:
const [offsetA = 0, offsetB = gutter] = initialOffset || [];
const offset = `${offsetA}, ${offsetB}`; // [offsetA, offsetB] on Popper v2?

I don’t plan to backport the API to v1, I’m sorry

The fact that Reakit uses Popper.js under the hood should probably be an implementation detail. Maybe Reakit should allow a number and a tuple for the offset. While Reakit uses v1 of Popper.js, it can transform the tuple ([-10, 10]) to the string (-10px, 10px) that Popper.js v1 understands. When v2 is released and Reakit upgrades to it, the transformation can be dropped.

Just ran into this issue myself. @ItsJonQ, how is the PR going? I can make some time to address this today or tomorrow if there's a need.

@FezVrasta Will the v2 API continue to support '%p', '%r', and other special strings, or do you intend to deprecate those in favor of the function case?

The function callback will replace them

@folz Hai there! My apologies! Things have been a bit busy on my end, I haven't gotten around to it yet. You (or anyone else) is more than welcome to submit a PR 🙏

All good! Just wanted to make sure I wouldn't be clobbering anything you were working on if I started something up. Thanks for getting back to me :) I'll take a crack at this and try to have a PR out by EOD.


Added B9PartSwitch as recommendation as its used on some of the new m…

…odels

Thanks for the update!


HDMI-CEC

In Front of Room Displays you advise to buy Displays that support HDMI-CEC. The Logitech Smartdock does not support this feature, see https://support.logitech.com/en_us/product/smartdock/faq#link_Does-SmartDock-support-CEC-for-attached-displays so maybe it is better to rephrase this text. I have now 22 Samsung screens with HDMI-CEC support that I can't use.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f68cba21-b43e-ed44-449c-da752f5a0f53
Version Independent ID: 5a2a2c5e-ad0e-833d-6ab7-74ed26f63d4a
Content: Microsoft Teams Rooms maintenance and operations
Content Source: Skype/SfbServer/manage/skype-room-systems-v2/room-systems-v2-operations.md
Product: skype-for-business-itpro
GitHub Login: @jambirk
Microsoft Alias: jambirk

@officedocsbot assign @scanum

@davgroom @sohailta
Please have a look at this comment.  --jambirk

@davgroom @sohailta Any updates on this?

Not all MTR systems supports CEC.  Some NUC miniPC supports CEC and provides a BIOS setting to enable it. Unfortunately they cannot be controlled by application SW, therefore is not very useful. You may want to consider some external pass through device like the one below, if you intend to use CEC for your system.
https://www.extron.com/article/hdctl100ad

@Polleke66 Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.

@officedocsbot close

@scanum, nope issue has not been resolved, at least to my knowlede.
On the microsoft site there is still a Note that we should buy screens with CEC for SRS systems:

A consumer TV used as a front of room display needs to support/enable the Consumer Electronics Control (CEC) feature of HDMI so that it can switch automatically to an active video source from standby mode. This feature is not supported on all TVs.
https://docs.microsoft.com/en-gb/MicrosoftTeams/room-systems/room-systems-v2-operations

Logitech says the exact opposite:

Logitech SmartDock with Microsoft Surface Pro does not support CEC. It is recommended to use commercial displays with power management capabilities such as schedule on/off times.
https://support.logi.com/hc/en-us/articles/360023367273

@officedocsbot reopen

@officedocsbot open


Autodesk.DWGTrueView version <IP_ADDRESS>

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

You will need to update the folder the manifest is in because of the ID change

Installer flag will not work until the initial exe extracts working files it seems. Winget would likely need to accommodate for this somehow.

TTBOMK: DWG TrueView 2022 does not have the option to perform an unattended install, straight out of the box. Your need to  extract the contents of the DWGTrueView_2022_English_64bit_dlm.sfx.exe file, and only then can you run setup.exe. Unfortunately winget is expecting direct access to the installer and thus cannot handle this senerio.
see: https://silentinstallhq.com/autodesk-dwg-trueview-2022-silent-install-how-to-guide/

TTBOMK: DWG TrueView 2022 does not have the option to perform an unattended install, straight out of the box. Your need to  extract the contents of the DWGTrueView_2022_English_64bit_dlm.sfx.exe file, and only then can you run setup.exe. Unfortunately winget is expecting direct access to the installer and thus cannot handle this senerio.
see: https://silentinstallhq.com/autodesk-dwg-trueview-2022-silent-install-how-to-guide/


Added a paragraph about modern app registration

As some folks have reached out confused about how to register their modern apps using OpenId in Enterprise Applications blade,  Adding a clarification to help them out.

@kalyankrishna1 : Thanks for your contribution! The author, @CelesteDG, has been notified to review your proposed change.

Adding @CelesteDG , @mtillman and @danieldobalian

Hi @kalyankrishna1 - as a PM, you should be submitting changes in the private repositories so the content can be staged and validated by the current validation rules.

For reference in future content PRs:
Read about authors working in the public repo for more information.
If you need help getting set up to work in the private repo, see Set up Git repository locally for documentation.

Hi @kalyankrishna1 - Can you please close this PR and make the changes to the private repo version of the files? When you click the pencil button to edit, please just make sure you add "-pr" to the URL so it's azure-docs-pr rather than just azure-docs when it directs you to GH.
#please-close

Created a PR in my fork as instructed (https://github.com/MicrosoftDocs/azure-docs-pr/pull/75185)


Atom doesn't save the file tree upon closing the program

Prerequisites

[ x  ] Put an X between the brackets on this line if you have done all of the following:

Reproduced the problem in Safe Mode: http://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
Followed all applicable steps in the debugging guide: http://flight-manual.atom.io/hacking-atom/sections/debugging/
Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description
Atom doesn't save the file tree upon closing the program. The application.json file clears every time, even with "Restore Previous Windows on Start" enabled. I have tried completely wiping and re-installing Atom, choosing a different directory, opening it in safe mode, none of those things have solved the issue.
 It doesn't.
Actual behavior: Atom opens with a completely blank editor.
Reproduces how often: 100% of the time.
Versions
Atom    : 1.16.0
Electron: 1.3.13
Chrome  : 52.0.2743.82
Node    : 6.5.0

Additional Information
I am on the latest version of macOS.

Thanks for opening a new issue @RyanIMinnick - cross-referencing your message board post with a lot of the debugging steps you've already tried:
https://discuss.atom.io/t/atom-wont-save-my-default-file-tree/42077

@rsese Yes, sorry, I should have included that.

No problem :v:
Can you confirm if you have Settings > Core > Restore Previous Windows on Start checked?

Yes. I tried it with that turned on, turned off, with "open empty editor" on and off, and now after reinstalling atom restore previous window on start is enabled.

Close Atom

Can you confirm how you close Atom?  Are you clicking the red x in the window title bar or are you quitting the program (e.g. with cmd-q or Atom > Quit in the menu)?

All return the same result @rsese, my file tree cleared. I've tried all that you've mentioned.

Hmm, would you be able to record a screencast or GIF showing these steps?

With an application.json file that has no paths, run cat ~/.atom/storage/application.json
In your project directory start Atom with atom .
run cat ~/.atom/storage/application.json
Close Atom from the menu with Atom > Quit
run cat ~/.atom/storage/application.json

So if you could basically match my GIF, that would be great:

Sorry for the repetition, I know you've gone through a lot of this already based on the message board post but I just want to step back a bit and make sure we're on the same page.

Missed the last step of starting Atom again with just atom, added the step above.

I'll give it a try.

@rsese https://vid.me/8GwF

Great thanks @RyanIMinnick I think I see what's happening - it looks like everything works fine when you get to the point where youAtom > Quit and then start Atom again.  The project you had open is opened again when you restart and the project path in your application.json file is correct to match.
The issue is at around where there's ~22 seconds left in the video, it looks like you close the Atom window with the red x in the window bar rather than quitting Atom (let me know if that's not the case).  When you close the window, Atom is still running but now there's no project to remember, it's remembering this empty state (as reflected in your application.json file).  Max describes this behavior here:
https://github.com/atom/atom/issues/13895#issuecomment-283726843
So basically, if you want Atom to remember the project you had opened, you should quit Atom (with Atom > Quit or cmd-q) rather than just closing the last window.  As Max also mentions, this is macOS specific since on Windows and Linux, if you close the last open Atom window, that quits the app.
So I can reproduce the behavior you're reporting if I close the Atom window rather than quitting Atom, but that's expected behavior.  Let me know if that makes sense of if you have questions @RyanIMinnick.

Okay... it saves the previous session if I close with CMD+Q or Atom > Quit (Both in the hotbar and navbar)... I wonder why it resets on a soft close but on a hard close it doesn't? Weird behavior. Good to know its normal though.

it saves the previous session if I close with CMD+Q or Atom > Quit (Both in the hotbar and navbar)...

Yep that's right :+1:

I wonder why it resets on a soft close but on a hard close it doesn't?

It's because when you soft close so there's no windows but Atom is still open, it's remembering that empty state.  When you quit while a project is open, the last state Atom remembers is with that last open project.

Weird behavior. Good to know its normal though.

Yep I'm glad we figured out what was going on, thanks for your patience!


Year prompt is breaking this generator

I made a note on the relevant commit, but these lines are breaking this generator: https://github.com/jozefizso/generator-license/blob/master/app/index.js#L69-L72
That is not a valid Inquirer prompt.
Either way, I think we're better not making a prompt out of this. It is verbose and probably no one needs to manually define the year. If you really want one, then create an option as it won't clutter the interface.

Ou, I created a really f***ed uped release.
The year option is there to allow it to set from unit tests. I did not anticipate the user would enter it, although in rare cases it is possible they want to set a different year. It is really not necessary to ask everybody about the year.
When I was testing it from command line, it worked fine and did not crash, but I see now that my published version crashes.


[bmap] Fix up

@glennsl  if you are okay we can merge this in?

@glennsl I will merge this in then. Fixed it up per the comments.


Fcrepo 2248: 	Adds missing Content-Length header on HEAD request for non binary obj

Closing this, based on discussion. Most of the changes here were part of https://github.com/fcrepo4/fcrepo4/commit/b06bf6bad2346373225f4b45c7275afa3530f6c7


⚠ Bofrost.be has degraded performance

In fc1d7a2, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1488 ms

Resolved: Bofrost.be performance has improved in 5081b90 after 39 minutes.


Templates are kinda broken with latest Bulma

Its unfortunate but it seems that with the latest versions some templates are broken, new classes seem to have been added in the framework. Have you tried it? :(

@hugows Yes, I have noticed. These templates were developed for bulma 0.1.0. I am working on updating them all to version 0.3.0 in an upcoming release.

Great job btw, I think its a great selling point of the Bootstrap site - and the Bulma guys should put you on their Frontpage!!


Feature request: Provide an ability to select older detekt version to make formatting rules backwards compatible

Currently there is no way to match detekt version between the one used in gradle dependencies and the plugin.
This matters because the formatting rules (ktlint) used in plugin is always latest while the codebase might be using an older version based on the given compatibility table: https://detekt.dev/docs/introduction/compatibility/#tool-versions.
Workaround we have been doing is to manually download an older version of plugin based on the release notes which is not very intuitive.
Requesting a feature where you get to choose detekt version (or formatting rules version specifically) which would somewhat look like this:

Related to https://github.com/detekt/detekt-intellij-plugin/issues/125.
Hm, if the plugin would not bundle the formatting jar I think your usecase could be solved by choosing the formatting jar a s a plugin jar.
I wonder if it is possible to bundle an extra jar in the plugin zip and load it on demand but not on startup of the plugin.
Like an implicit plugin jar when the Enable formatting option is chosen.

your usecase could be solved by choosing the formatting jar a s a plugin jar I tried this but it breaks the plugin completely somehow. Meaning, no issues gets detected on the fly if I choose a suitable formatting jar in plugin jars manually. This could be a bug.

your usecase could be solved by choosing the formatting jar a s a plugin jar

I tried this but it breaks the plugin completely somehow. Meaning, no issues gets detected on the fly if I choose a suitable formatting jar in plugin jars manually. This could be a bug.

Just to be sure, you forked the plugin, removed the formatting dependency and added the formatting jar as a plugin jar?
As the detekt plugin bundles the formatting jar by default, there could be some class conflicts if you just try to add another formatting jar version.


Tabs with icon

If you create a tabbable navigation and add an icon to only one tab, the other tab(s) have an incorrect height and thus the bottom border is shown.
The following example illustrates the issue:
http://jsfiddle.net/ZCZev/
A temporary solution is to add
<i class="icon-blank"></i>

Inside the tab's anchor element and have icon-blank be like:
.icon-blank {
	width: 0px;
}

Not seeing the problem you're mentioning on Chrome 18.

Ah, I see it now. Will look into it. To clarify, the problem appears on hover of the non-active tab.

I've resolved this in 2.0.2-wip by adding line-height: 18px.


Add readonly property modifier everywhere the property isn't modified.

Has a merge conflict. What is the intention behind this?

@fb55 Resolved the merge conflicts and rebased on gdax's master.

Thanks a lot!


Data reconcilation for cached functions works incorrectly under some conditions

Describe the bug
On the first cache revalidation, data is reconciled incorrectly, which causes rerenders.
Example:
https://playground.solidjs.com/anonymous/b6b25440-9d66-467e-94ee-5148484a5bc2
You'll see in the console that items are rerendered on the first revalidation, however it doesn't happen when you continue revalidate the data.
Your Example Website or App
https://playground.solidjs.com/anonymous/b6b25440-9d66-467e-94ee-5148484a5bc2
Steps to Reproduce the Bug or Issue
Check out the example
Expected behavior
Items should be reconciled correctly without rerenders
Screenshots or Videos
No response
Platform

OS: [e.g. macOS, Windows, Linux]
Browser: [e.g. Chrome, Safari, Firefox]
Version: [e.g. 91.1]

Additional context
No response

Thanks.. I missed this. I'm trying to track this one down but its odd.. Somehow on first load we aren't getting the wrapped value.


KAFKA-5783: Add KafkaPrincipalBuilder with support for SASL (KIP-189)

@rajinisivaram, maybe you can review this as well? Once you're happy with it, feel free to merge. I don't need to re-review. Also, I only had a brief look at the tests.

@ijuma @rajinisivaram Thanks for the reviews. I think I've addressed your comments.

retest this please

SUCCESS
7559 tests run, 4 skipped, 0 failed.
--none--

@ijuma @hachikuji has added a null check for client address in the latest commit. I am happy with that. If there are no other concerns, shall I merge this?

Sounds good @rajinisivaram


Cloud land

New Area Suggestion
NOTE: If you wish to present more than one area in this suggestion write each one in a comment and use the following structure
What will this area look like?
We would walk on clouds and there would be some rainbows.
What slimes will exist in this area?
Cloud slime,thunder,water drop,bird and pink slimes thunder and water drop slimes represent the clouds (thunder and rain)
What foods will exist in this new area?
Cloud berry, favorite of: cloud slime and water drop slime
Worm,favorite of: bied slime
Sky pear,favorite of: thunder slime
What is the reason for this area to exist?
This area will exist to add a new type of biome/area rather than just being on the ground.
How do you get to this area?
This area is related to the area i made a couple of minutes ago (space world) you need all the space world slime's plorts in order to use them in an area of statues when you put every plort in its respective statue a portal will open leading to the area.
Materials to be found (Optional)
Cloud essence and water droplets that can be used for additional crafts.
Possible names for the area? (Optional)
Cloud land
Additional context
Add any other context or screenshots about the suggestion here.

Did you mean to say that "space world" leads to this area? Last time I checked you go through the sky to get to space.


Fix interface mapping

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

10 of 12 (83.33%)  changed or added relevant lines in 3 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.1%) to 84.704%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/eve2cml/eve/node.py
7
9
77.78%

Files with Coverage Reduction
New Missed Lines
%

src/eve2cml/eve/node.py
1
94.52%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
659

Relevant Lines:
778

💛  - Coveralls


How can I establishing communication between an app and this React Native AndroidWear app?

can you please provide more details?

related issue #25

@robinparadise Closing due to lack of activity, feel free to reopen if needed.


[Change Request] codeunit 64 "Sales-Get Shipment" - CopyDocumentAttachments

Hello, can you please change the lokal procedure CopyDocumentAttachments to a global procedure?
    procedure CopyDocumentAttachments(var SalesShipmentLine: Record "Sales Shipment Line"; var SalesLine2: Record "Sales Line")
    var
        OrderSalesLine: Record "Sales Line";
        DocumentAttachmentMgmt: Codeunit "Document Attachment Mgmt";
    begin
        if (SalesShipmentLine."Order No." = '') or (SalesShipmentLine."Order Line No." = 0) then
            exit;
        if not AnyLineHasAttachments(SalesShipmentLine."Order No.") then
            exit;
        OrderSalesLine.ReadIsolation := IsolationLevel::ReadCommitted;
        OrderSalesLine.SetLoadFields("Document Type", "Document No.", "Line No.");
        if OrderSalesLine.Get(OrderSalesLine."Document Type"::Order, SalesShipmentLine."Order No.", SalesShipmentLine."Order Line No.") then
            DocumentAttachmentMgmt.CopyAttachments(OrderSalesLine, SalesLine2);
    end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 13402.


we can enhance Gold price prediction by using LSTM model

Gold Price Prediction
Project Overview
This project aims to predict the price of gold using various machine learning models. The project includes data exploration, feature engineering, model training, and evaluation.
Requirements

Data Retrieval:

Collect historical gold price data from a reliable source.

Data Exploration:

Perform exploratory data analysis (EDA) to understand the data distribution and patterns.
Visualize the data to identify trends and relationships.

Feature Engineering:

Create new features that could improve the model's performance.
Handle missing values and perform data normalization/standardization.

Model Training:

Train various machine learning models (e.g., Linear Regression, Decision Trees, Random Forest, LSTM).
Use train_test_split to create training and testing datasets.
Apply techniques like cross-validation to improve model robustness.

Model Evaluation:

Evaluate model performance using appropriate metrics (e.g., RMSE, MAE, R2 score).
Compare the performance of different models.
Visualize the model predictions against actual values.

Libraries and Tools

pandas
numpy
matplotlib
seaborn
scikit-learn
tensorflow and keras (for LSTM model)
Any other relevant libraries for data processing and model training

Approach

Data Retrieval:

Collect historical gold price data from a reliable source such as Yahoo Finance, Quandl, or other financial data providers.

Data Exploration:

Load the data into a DataFrame and inspect the first few rows.
Check for missing values and handle them appropriately.
Perform statistical analysis to summarize the data.
Create visualizations (e.g., line plots, histograms, box plots) to understand data trends and distributions.

Feature Engineering:

Generate additional features like moving averages, rolling statistics, and lag features.
Normalize or standardize the data to improve model performance.
Encode categorical features if any.

Model Training:

Split the data into training and testing sets using train_test_split.
Train various machine learning models:

Linear Regression
Decision Trees
Random Forest
LSTM (using TensorFlow and Keras)

Perform cross-validation to ensure model robustness.

Model Evaluation:

Evaluate each model using metrics like RMSE, MAE, and R2 score.
Visualize the predictions of each model against actual gold prices.
Select the best-performing model based on evaluation metrics.

Tasks

[ ] Collect historical gold price data.
[ ] Perform exploratory data analysis (EDA).
[ ] Conduct feature engineering.
[ ] Normalize or standardize the data.
[ ] Split the data into training and testing sets.
[ ] Train machine learning models (Linear Regression, Decision Trees, Random Forest, LSTM).
[ ] Perform cross-validation.
[ ] Evaluate model performance using RMSE, MAE, and R2 score.
[ ] Visualize model predictions.
[ ] Compare the performance of different models.

Additional Information
Use the following libraries to assist with model training and evaluation:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout

** Expected Outcomes**

A comprehensive analysis of historical gold price data.
Effective feature engineering to enhance model performance.
Training and evaluation of multiple machine learning models.
Selection of the best model for gold price prediction.
Visualization of model predictions to provide insights.

give your opinion

already completed


Improve location list behavior on split windows of same buffer.

Currently, there are no tests.
Intended to fix w0rp/ale#2214.

How might I be able to help move this through?

I'll just merge this. It might not be possible to write Vader test for this.

Cheers! :beers:


[sway/language] Improve sway/language

Inpired by https://github.com/Alexays/Waybar/pull/659

Thanks!
Can you do another PR with just Dockerfiles changes?

Yes, I'll do.

Would be great also for this module to support tooltip-format

@eternal-sorrow
Ok, no problem.

@eternal-sorrow
Implemented in https://github.com/Alexays/Waybar/pull/1178


browser appears to cache the html

You can update the images but the rendered html does not change - I assume this is because I am using generic filenames for the local files and therefore the browser appears to cache the html. Solution is to use private tab in chrome

This can be avoid by simply giving the saved images a unique name each, but does mean the number of images saved to disc will grow over time

Hi, thanks for pointing this out. I had the same problem and fixed it by generating a random string for the image namees during each run.


No Method for _memcmp

There doesn't seem to be a matching signature for _memcmp. Usually this occurs when I save new changes while the server is running.

     Use `show(err)` to see complete types.

Hello @limarta you're on Julia 0.10 right? this was flagged here: https://github.com/tlienart/Xranklin.jl/issues/261
I think I know how to fix it, just haven't had the occasion to implement and test it but should now


ignoreAltitude now working

Ignore altitude feature now working, but I noticed that there is a problem when SceneLocationView is loaded.
Main Thread Checker: UI API called on a background thread: -[UIApplication applicationState]
PID: 6827, TID: 1912939, Thread name: com.apple.CoreMotion.MotionThread, Queue name: com.apple.root.default-qos.overcommit, QoS: 0
Backtrace:
4   libobjc.A.dylib                     0x000000023242f6f4 <redacted> + 56
5   CoreMotion                          0x0000000238ba9d9c CoreMotion + 294300
6   CoreMotion                          0x0000000238baa2cc CoreMotion + 295628
7   CoreMotion                          0x0000000238baa1dc CoreMotion + 295388
8   CoreMotion                          0x0000000238bd801c CoreMotion + 483356
9   CoreMotion                          0x0000000238bd8060 CoreMotion + 483424
10  CoreFoundation                      0x00000002331be27c <redacted> + 28
11  CoreFoundation                      0x00000002331bdb64 <redacted> + 276
12  CoreFoundation                      0x00000002331b8e58 <redacted> + 2276
13  CoreFoundation                      0x00000002331b8254 CFRunLoopRunSpecific + 452
14  CoreFoundation                      0x00000002331b8f88 CFRunLoopRun + 84
15  CoreMotion                          0x0000000238bd79f4 CoreMotion + 481780
16  libsystem_pthread.dylib             0x0000000232e36908 <redacted> + 132
17  libsystem_pthread.dylib             0x0000000232e36864 _pthread_start + 48
18  libsystem_pthread.dylib             0x0000000232e3edcc thread_start + 4

@lrsb - that thread checker issue is not new.  That's apparently from the ARKit library itself.  😞 I'll take a look at your PR, thanks for getting back to me so quickly


Develop

Hi @josh-nw can you explain the reason for the pull request? Were you trying to submit this PR to the original Docker repository? If so, can you sign your work? More info in our contribution guide: https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work

Hello Mano -  Totally my mistake. We had a laptop crash on a demo and I was working with a fork of this repo and was testing a ci/cd pipeline when I had to resolve a merge issue with the forked repo on the github web ui (don't use it that much) and the pull request got submitted here by mistake.
I will close now and apologies for any time lost on this.


other appearance customizations are not displayed(except Prism-Theme) in Style-Setting Plugin since update

@damiankorcz hello again
Problem
one day i notice i couldn't customize ITS-Checkboxes snippets(#20) in Style-Setting Plugin. then i open another vault with an older version of Prism-Theme and find it works normally.

v1.3.0

v1.5.2
it seems all appearance customizations disappear except Prism-Theme
Environment
os: win 10
ob ver: 0.13.19
prism ver: v1.5.1
Last
hope for your reply asap 😊

and i try to update the Style-Setting Plugin to the latest version and it still remains the bug

Hi @AiurArtanis, I don't seem to have this issue. I'm on Obsidian v0.13.24, Prism v1.5.2 and Style Settings v0.4.9. Since you are on an older version of Obsidian try updating to the latest public build. Should be v0.13.23. See if that helps at all. Not sure what might be causing this.

emmm i try to update ob to ver23 then i create a new vault with only Prism-Theme, css snippet and Style-Setting, and the bug still stays:

btw, where is this ver24 ob 🤣
the release from official page remains ver23:

It seems that we are using different snippets for the ITS Checkboxes. This is where the latest one is: https://github.com/SlRvb/Obsidian--ITS-Theme/blob/main/S - Checkboxes.css
Also version Obsidian v0.13.24 is the current insider version. I'll test more on this tomorrow.

It seems that we are using different snippets for the ITS Checkboxes. This is where the latest one is: https://github.com/SlRvb/Obsidian--ITS-Theme/blob/main/S - Checkboxes.css

i think it is nothing with this CSS snippet because i find the same case in plugin Buttons

prism:v1.3.0

prism:v1.5.2

Hi @AiurArtanis, I've tried again on a new vault with a fresh copy of Prism v1.5.2, Style Settings v0.4.9, Buttons v0.4.16 and the latest version of the ITS Checkboxes snippet. The Buttons Style Settings didn't show up for me initially so I made sure to comment out elements of the theme and slowly rebuild it by re-adding each .scss back to it. Although it's showing correctly now I see no changes in git (?) Really don't understand what is going on. I've pushed the rebuilt version of the theme to the main branch so you should be able to update and see if that does anything. Note I haven't changed the theme version number so don't worry about that.

I've pushed the rebuilt version of the theme to the main branch so you should be able to update and see if that does anything. Note I haven't changed the theme version number so don't worry about that.

i checked the commit history and didn't find the fix scss😂. or you try to fix it in v1.6.0 altogether?

it's fixed on the latest OB though i'm not sure if this update indeed works or other reason


Scala extension method that converts Map[CharSequence, String] to Iterator<Entry<>>

It is currently quite inconvenient when a Scala user tries to put multiple headers into HttpHeadersBuilder from a Map:
val headers: Map[String, String] = ...
val builder = HttpHeaders.builder()
headers.foreach { tuple => builder.add(tuple._1, tuple._2) }

It'd be nice if we add an extension method to Iterable[(CharSequence, String)] (and Iterable[(String, String)]?) so it's easier to convert it into an Iterator[Map.Entry[CharSequence, String]]:
builder.add(headers.mapEntryIterator)

Could we add a default method for java.util.Map<String, String> to HettpHeadersBuilder instead of adding a new extension method?
interface HttpHeadersBuilder {
    default HttpHeadersBuilder add(Map<? extends CharSequence, String> entries) {
        return add(entries.entrySet());
    }
}

Users could use scala.immutable.Map[String, String].asJava with:
import com.linecorp.armeria.scala.implicits._
val headers: Map[String, String] = Map("foo" -> "bar")
val builder = HttpHeaders.builder()
val header = headers.asJava
builder.add(header)